FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Enable transcription of native commands on non-Windows by SteveL-MSFT · Pull Request #4871 · PowerShell/PowerShell · GitHub

Enable transcription of native commands on non-Windows - #4871

Merged
Dongbo Wang (daxian-dbw) merged 7 commits into
PowerShell:masterfrom
SteveL-MSFT:mac-transcript
Sep 29, 2017
Merged

Enable transcription of native commands on non-Windows#4871
Dongbo Wang (daxian-dbw) merged 7 commits into
PowerShell:masterfrom
SteveL-MSFT:mac-transcript

Conversation

Copy link
Copy Markdown
Member

Transcription was relying on reading the screen buffer to record output from native commands.
This resulted in an unhandled exception calling an unimplemented api on non-Windows.
Fix is to default to using redirected output if reading the screen buffer is not supported.

Fix #1920

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Also need to check that PS error stream is empty.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

will add

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Actually, there's no reason to follow that pattern, I'm rewriting it as a scriptblock and any errors should just show up in the test run.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Need newline in the end of file.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

will add

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This can be made a static check so that we do it only once. And _scrapeHostOutput can be renamed to s_supportScreenScrape (or something similar).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

will change

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Why do we want redirectInput to be true by default? Even if we are transcribing and screen scrape is not supported, we shouldn't need to always redirect the input.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

And form line 442 below, it indicates that we transcribe for native command only if _runStandAlone is true. Now it looks like you are assuming we can do transcribe regardless of _runStandAlone. That seems not right.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Yes, redirectInput shouldn't be true by default. Will fix.

As for _runStandAlone, my understanding of the code is that if not running standalone, output is redirected and will be captured in the transcript. However, if running standalone, the output is screen scraped and put into the transcript. So it seems that regardless if running standalone, if screen scraping is not supported, the only way to transcribe the output is to redirect it.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Suggestion: this.Command.Context.EngineHostInterface.UI.IsTranscribing is used at 3 places in this class. Maybe we can have an instance field to cache the value? Like _isTranscribing = this.Command.Context.EngineHostInterface.UI.IsTranscribing in the constructor?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Will fix

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

from line 444 below, it looks like a HostException also indicates The host doesn't support scraping via its RawUI interface ...

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think the catch for HostException was a mistake. Going through the code in the debugger, that exception isn't thrown.

Dongbo Wang (daxian-dbw) Sep 20, 2017
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

By putting the call to GetBufferContents in the constructor, it's possible that an arbitrary exception will be thrown from GetBufferContents in the constructor. When that happens, it would be wrapped to an ApplicationFailedException properly prior to this change, but after this change, the NativeCommandProcessor would just fail to be constructed, and thus I guess the error would look obscure too. So, I'm not sure if it's appropriate to put this in the constructor ...

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

will fix

Copy link
Copy Markdown
Member

Fix is to default to using redirected output if reading the screen buffer is not supported.

Steve Lee (@SteveL-MSFT) Could you please explain a little why redirecting the output/error would support transcribing in case that screen scraping is not supported? (just curious how it's done in that case)

Copy link
Copy Markdown
Member Author

Dongbo Wang (@daxian-dbw) from what I can tell in the code, anything that gets redirected gets transcribed anyways

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think setting redirections for transcribing purpose should be done in the method CalculateIORedirection, and IMO it should be done at the end of that method -- after the existing logic of that method has run. For example, in case of running on the server side, redirectInput should always set to be true. Code like that should run even though screen scraping is not supported.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Makes sense, will fix.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think there is a change of behavior here. Assuming _isTranscribing == $true and screen scraping is supported,

  • before the change, _scrapeHostOutput is true only if _runStandAlone is true, so this block will run only the native command is running standalone.
  • after the change, even if it's not running standalone (i.e. standard input/error redirected), this block of code still will run when transcribing, and in such case, _startPosition is not even initialized.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Will fix

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Maybe remove the else? The else if (Platform.IsWindowsDesktop && IsConsoleApplication(this.Path)) block doesn't necessarily set redirectOutput and redirectError to true.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

And please add a comment that explains why we need to set output/error redirection in such case.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

yup

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

These changes now can be reverted.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Will revert

Copy link
Copy Markdown
Member

Andrew (@anmenaga) Can you please take a look again?

Copy link
Copy Markdown
Member Author

Mark Kraus (@markekraus) the test failures are indicating the WebListener didn't start within the timeout. Is this something you're familiar with?

Copy link
Copy Markdown
Contributor

Steve Lee (@SteveL-MSFT) News to me. I will take a look.

Mark Kraus (markekraus) commented Sep 26, 2017
edited
Loading

Copy link
Copy Markdown
Contributor

Steve Lee (@SteveL-MSFT) This appears to be related to the change in this PR. It looks native execution inside a job is broken. WebListener runs dotnet WebListener.dll in a Start-Job block. From your PR I can manually run the WebListener outside of a job in PowerShell. But inside a job it fails. But it's not limited to WebListener:

$Job = Start-Job {cmd.exe /c echo "test" }
$Job | Wait-Job | fl * -force

$Job.ChildJobs[0].Output

expected behavior (working in beta.7):

State         : Completed
HasMoreData   : True
StatusMessage :
Location      : localhost
Command       : cmd.exe /c echo "test"
JobStateInfo  : Completed
Finished      : System.Threading.ManualResetEvent
InstanceId    : 7d654dd9-ba42-48f0-be72-9e91863e7286
Id            : 1
Name          : Job1
ChildJobs     : {Job2}
PSBeginTime   : 9/26/2017 3:41:02 PM
PSEndTime     : 9/26/2017 3:41:02 PM
PSJobTypeName : BackgroundJob
Output        : {}
Error         : {}
Progress      : {}
Verbose       : {}
Debug         : {}
Warning       : {}
Information   : {}

test

Actual behavior:

State         : Failed
HasMoreData   : False
StatusMessage :
Location      : localhost
Command       : cmd.exe /c echo "test"
JobStateInfo  : Failed
Finished      : System.Threading.ManualResetEvent
InstanceId    : 58c30632-c4a1-41e3-b276-5aa9b24db3b0
Id            : 3
Name          : Job3
ChildJobs     : {Job4}
PSBeginTime   : 9/26/2017 3:42:41 PM
PSEndTime     : 9/26/2017 3:42:42 PM
PSJobTypeName : BackgroundJob
Output        : {}
Error         : {}
Progress      : {}
Verbose       : {}
Debug         : {}
Warning       : {}
Information   : {}


Steve Lee (SteveL-MSFT) commented Sep 26, 2017
edited
Loading

Copy link
Copy Markdown
Member Author

Mark Kraus (@markekraus) thanks! I'll take a look.

The problem is that when run as a job obviously there's no console, so some of the console calls even on Windows throws other exceptions (and it's not just the Host.HostException from the old code) when the code is checking if screen scraping is supported. Added a test to test start-job with native command to cover this.

Dongbo Wang (@daxian-dbw) Andrew (@anmenaga) can you take another look?

…ot supported

since screen buffer reading isn't supported, default to using redirected output
moved setting redirection due to transcription to within CalculateIORedirection() function
ensure screen scrapping only happens when running standalone
address PR feedback from Dongbo
added test for start-job and native commands
catch all exceptions trying to read the buffer
catch (Exception)
{
// screen scraping not supported on non-Windows or as job
}

Dongbo Wang (daxian-dbw) Sep 27, 2017
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think this check can still be done only once -- we do this check in CalculateIORedirection and only if _runStandAlone is true. Here the the code I propose:

private void CalculateIORedirection(out bool redirectOutput, out bool redirectError, out bool redirectInput)
{
    ...

    if (NativeCommandProcessor.IsServerSide)
    {
        redirectInput = true;
        redirectOutput = true;
        redirectError = true;
    }
    else if (Platform.IsWindowsDesktop && IsConsoleApplication(this.Path))
    {
        // On Windows desktops, if the command to run is a console application,
        // then allocate a console if there isn't one attached already...
        ConsoleVisibility.AllocateHiddenConsole();

        if (ConsoleVisibility.AlwaysCaptureApplicationIO)
        {
            redirectOutput = true;
            redirectError = true;
        }
    }
    _runStandAlone = !redirectInput && !redirectOutput && !redirectError;

    // !!!!! NOTE !!!!! -- Unchanged until here

    if (_runStandAlone)
    {
        if (s_supportScreenScrape == null)
        {
            // Call GetBufferContents() to check and set s_supportScreenScrape to true or false accordingly.
        }

        // if screen scraping isn't supported, we enable redirection so that the output is still transcribed
        // as redirected output is always transcribed
        if (_isTranscribing && (false == s_supportScreenScrape))
        {
            redirectOutput = true;
            redirectError = true;
            _runStandAlone = false;
        }
    }
}

So the check of screen scraping support is done only if we are running the native command as standalone application.

In case of running it in a Job, I think the standard input and error are already redirected, so nothing needs to be done.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I see. I'll try that.

refactor code

Dongbo Wang (daxian-dbw) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

LGTM. Thanks!

Dongbo Wang (daxian-dbw) merged commit 99e3fe5 into PowerShell:master Sep 29, 2017
Steve Lee (SteveL-MSFT) deleted the mac-transcript branch October 26, 2018 21:33
Thatgfsj (Thatgfsj) pushed a commit to Thatgfsj/PowerShell that referenced this pull request Aug 6, 2026
Transcription was relying on reading the screen buffer to record output from native commands.
This resulted in an unhandled exception calling an unimplemented API on non-Windows.
The fix is to use redirected output/error if reading the screen buffer is not supported.
We check whether screen scraping is supported or not only when the application is running standalone.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants


Back | FazBrowse Home | New Git URL