Two separate toggles: "Pop up on stdout" and "Pop up on stderr". Either or both can be checked.
Not sure where these options should live, though. Edit > Options > Console would work I suppose.