FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Allow for the use of a proxy server during Python Language Server download by d3r3kk · Pull Request #2418 · microsoft/vscode-python · GitHub

Allow for the use of a proxy server during Python Language Server download - #2418

Merged
Derek Keeler (d3r3kk) merged 16 commits into
microsoft:masterfrom
d3r3kk:issue2385_proxy_download
Aug 24, 2018
Merged

Allow for the use of a proxy server during Python Language Server download#2418
Derek Keeler (d3r3kk) merged 16 commits into
microsoft:masterfrom
d3r3kk:issue2385_proxy_download

Conversation

Copy link
Copy Markdown

Fixes #2385

  • Title summarizes what is changing
  • Includes a news entry file (remember to thank yourself!)
  • Unit tests & code coverage are not adversely affected (within reason)
  • Works on all actively maintained versions of Python (e.g. Python 2.7 & the latest Python 3 release)
  • Works on Windows 10, macOS, and Linux (e.g. considered file system case-sensitivity)

this.platform = this.services.get<IPlatformService>(IPlatformService);
this.platformData = new PlatformData(this.platform, this.fs);
constructor(
private readonly output: IOutputChannel,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Why are we changing the di pattern? I thought we (you and i )agreed to keep using a simple parameter and user the service locator pattern?
Has anything changed since, to change your decision?

Derek Keeler (d3r3kk) Aug 21, 2018
edited
Loading

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This particular class was not using DI, so I didn't add it. There was no new decision here, that was all there was to it 😄.

I felt that there was no need to add it to this particular class, seemed unnecessary for what it does. But I am certainly willing to have that discussion!

Should we want to expand this class's design, perhaps that's a different issue deserving of a longer conversation.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

See #2421

Comment thread src/client/activation/downloader.ts Outdated
private readonly fs: IFileSystem,
private readonly platformData: PlatformData,
readonly workspace: IWorkspaceService,
private requestHandler: IRequestWrapper | undefined,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality
  • Optional parameter should be at the end.
  • Why is this optional?

Derek Keeler (d3r3kk) Aug 21, 2018
edited
Loading

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Agreed, I will move it - not sure why I missed that!

  • LanguageServerDownloader move optional arguments to the end.

Optional in the sense that you can supply your own implementation only. If you do not supply one, one is supplied for you. Allows us to test the implementation of the request 'wrapper'.

Comment thread src/client/activation/downloader.ts Outdated
}
// Set file to executable (nothing happens in Windows, as chmod has no definition there)
const executablePath = path.join(installFolder, this.platformData.getEngineExecutableName());
fileSystem.chmodSync(executablePath, '0764'); // -rwxrw-r--

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Please use an async version, i.e. always use async instead of sync. This is what made node.js popular and faster. We need to remember that and not write old.net style sync code .

Derek Keeler (d3r3kk) Aug 21, 2018
edited
Loading

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Again, just using what was there before. I can improve on the code of course, but this change was supposed to be a simple one.

I will add an issue to redesign the downloader class to make it more generic/improve the design.

  • Create a new issue to track the redesign of the LanguageServerDownloader class. We should improve upon its design by adding DI, removing any synchronous calls, and making it generally more robust for other use cases.

  • Make the use of chmod async. This is the right thing to do.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

See #2421

proxy: this.proxyUri
};
}
return undefined;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Just use return

Derek Keeler (d3r3kk) Aug 21, 2018
edited
Loading

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Ok, will do!

  • RequestWithProxy::getRequestOptions remove return undefined and make it just return instead.

public downloadFileRequest(uri: string): request.Request {
const requestOptions = this.getRequestOptions();
if (requestOptions) {
return request(uri, requestOptions!);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

! is not necessary due to type guards.
Infact you should be able to use return request(uri, requestOptions)

Derek Keeler (d3r3kk) Aug 21, 2018
edited
Loading

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Ok, I'll try that. I believe the linter was complaining about this but I may be mistaken.

  • Remove unnecessary ! in downloadFileRequest

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I was mistaken. All good.

Comment thread src/client/activation/types.ts Outdated
}

export interface IRequestWrapper {
downloadFileRequest(uri: string): RequestResult;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

It provides a service of downloading a file, it does not wrap anything!
Please change the method to downliadFile.

Derek Keeler (d3r3kk) Aug 21, 2018
edited
Loading

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Sure thing.

  • Reconsider the names IRequestWrapper (not a wrapper) and downloadFileRequest.

Comment thread src/client/activation/downloader.ts Outdated
private engineFolder: string) {

if (!this.requestHandler) {
this.requestHandler = new RequestWithProxy(workspace.getConfiguration('http').get('proxy', ''));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This shouldn't be passed in as an argument. Assume you have to use this class in 10 places, now you have to read the proxy and pass it in 10 places.
It should be accessed by the downloader in the class itself.

Please inject this via DI, we shouldn't be manually constructing concrete classes, goes against IOC pattern.

Derek Keeler (d3r3kk) Aug 21, 2018
edited
Loading

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Assume you have to use this class in 10 places

We use it in 1, and I didn't think this issue deserved the time to make it more generic. I feel that is a different issue.

now you have to read the proxy and pass it in 10 places

It felt odd coupling the RequestHandler to the WorkspaceService. The downloader doesn't care that you have a Workspace object, it only cares that you have a proxy setting (or not).

Please inject this via DI

...not this change. Let's push that to another.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

See issue #2421

Copy link
Copy Markdown
Author

Don Jayamanne (@DonJayamanne) please re-review.

codecov Bot commented Aug 21, 2018
edited
Loading

Copy link
Copy Markdown

Codecov Report

Merging #2418 into master will decrease coverage by 0.48%.
The diff coverage is 68.29%.

@@            Coverage Diff             @@
##           master    #2418      +/-   ##
==========================================
- Coverage   75.63%   75.14%   -0.49%     
==========================================
  Files         310      310              
  Lines       14511    14352     -159     
  Branches     2567     2540      -27     
==========================================
- Hits        10975    10785     -190     
- Misses       3528     3558      +30     
- Partials        8        9       +1
Flag Coverage Δ
#MacOS ?
#Windows ?
Impacted Files Coverage Δ
src/client/common/platform/types.ts 100% <ø> (ø) ⬆️
src/client/activation/types.ts 100% <ø> (ø) ⬆️
src/client/activation/languageServer.ts 26.08% <0%> (+0.4%) ⬆️
src/client/activation/platformData.ts 85% <100%> (ø) ⬆️
src/client/common/platform/fileSystem.ts 70.96% <66.66%> (-0.47%) ⬇️
src/client/activation/requestWithProxy.ts 66.66% <66.66%> (ø)
src/client/activation/downloader.ts 34.56% <72.22%> (-0.8%) ⬇️
src/client/common/platform/registry.ts 40.42% <0%> (-51.07%) ⬇️
src/client/common/platform/pathUtils.ts 66.66% <0%> (-33.34%) ⬇️
src/client/common/utils.ts 57.89% <0%> (-16.79%) ⬇️
... and 29 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e3a6bc2...26fbd00. Read the comment docs.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

At a high level, LGTM. However there are a bunch of little things I noticed that might be worth cleaning up. Some of them are nits, but some should probably be addressed.

Comment thread src/client/activation/downloader.ts Outdated
private readonly platformData: PlatformData,
readonly workspace: IWorkspaceService,
private engineFolder: string,
private requestHandler?: IDownloadFileService) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

FWIW, I find it clearer to put the opening brace on its own line:

    private engineFolder: string,
    private requestHandler?: IDownloadFileService
) {

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think either way is good, and I have tried to follow what I see across the rest of the classes in the extension thus far.

Personally, I like the:

constructor
(
    some: string,
    params: number,
    here: object
)
{
    some_code_here();
}

...way of doing things, but I'm learning to adapt.

deactivate(): Promise<void>;
}

export interface IDownloadFileService {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Doesn't this need the following?

export const IDownloadFileService = Symbol('IDownloadFileService');

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Our use of injectify makes use of the Symbol type to identify implementations of interfaces. Since we don't inject the IDownloadFileService into DI (yet) it doesn't need the Symbol definition.

Comment thread src/client/activation/downloader.ts Outdated

let localTempFilePath = '';
try {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

superfluous blank line

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Agreed.


const deferred = createDeferred();
const fileStream = fileSystem.createWriteStream(tempFile.filePath);
const fileStream = this.fs.createWriteStream(tempFile.filePath);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Ha! We weren't even using this.fs already?

Comment thread src/client/activation/downloader.ts Outdated
private readonly output: IOutputChannel,
private readonly fs: IFileSystem,
private readonly platformData: PlatformData,
readonly workspace: IWorkspaceService,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This doesn't need readonly, right?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I felt this was appropriate, since I am not changing the state of that object and am only reading from it.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Ah, I didn't realize you could mark parameters as readonly. Cool!

if (requestOptions) {
return request(uri, requestOptions);
}
return request(uri);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

FWIW, if both cases terminate the function then it's often clearer to use an else:

    if (requestOptions) {
        return request(uri, requestOptions);
    } else {
        return request(uri);
    }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

6 or 1/2 dozen.
Personally I find the current approach more readable, less nesting. I like an else for such simple statements only when using IIF (immediate ifs)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Agreed.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Wow Don Jayamanne (@DonJayamanne) jumped in between Eric Snow (@ericsnowcurrently)'s comments and my responses. Fun timez.

I am going with Eric's suggestion here, I likey.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I definitely agree with Don in cases where there's a clear common path. I try to keep those less indented than the code for the "exceptional" cases. However, when they are equally likely or equally relevant I prefer to keep indentation the same. It helps inform the reader of the nature of the cases.

That said, if either case is more than a few lines it often makes sense to factor it out into a separate private function/method, so that the logic of the original function isn't obscured by handling of the different cases.

}

public chmod(filePath: string, mode: string): Promise<void> {
return new Promise<void>((resolve, reject) => {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Blech! This async stuff is so gross! Kudos for knowing how to do this. :)

Mikhail Arkhipov (MikhailArkhipov) Aug 23, 2018
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

There is also createDeferred() that simplifies things. Rough equivalent of the TaskCompletionSource in C#

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I've not learned why/how to use the createDeferred method yet, and was simply following the pattern from the methods above!

const link = languageServerDownloader.getDownloadUri();
assert.equal(link, DownloadLinks[PlatformName.Linux64Bit]);
});
test('Supports download via proxy', async () => {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Maybe I'm missing something, but languageServerDownloader isn't used in this test. So doesn't the test belong in src/test/activation/requestWithProxy.unit.test.ts (or at least in its own suite)?

Also, there probably should be separate tests for LanguageServerDownloader.downloadLanguageServer() that use different languageServerDownloader.requestHandler values.

Derek Keeler (d3r3kk) Aug 23, 2018
edited
Loading

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

You found me out! Lazy me.

  • Move tests into appropriate test suites.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Let's ensure we add far more robust testing as part of #2421, as it will need a bit of fine-tuning to make it more accessible to tests.

Comment thread src/client/activation/downloader.ts Outdated
}, (progress) => {

requestProgress(request(uri))
requestProgress(this.requestHandler!.downloadFile(uri))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Splitting up this line might be more readable:

    requestProgress(
        this.requestHandler!.downloadFile(uri))

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Agreed

const fileName = files[0].replace(/\\/g, '/');
expect(fileName).to.equal(expectedFileName);
});
test('Ensure creating a temporary file results in a unique temp file path', async () => {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

smart tests!

fileSystem.chmodSync(executablePath, '0764'); // -rwxrw-r--
}
// Set file to executable (nothing happens in Windows, as chmod has no definition there)
const executablePath = path.join(installFolder, this.platformData.getEngineExecutableName());

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

It worked on Windows as is but OK

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I took the opportunity to decouple this class from the IPlatformService (I know we still use PlatformData to get a hold of some names, but those two classes have separate use cases).

Copy link
Copy Markdown
Author

Eric Snow (@ericsnowcurrently), Don Jayamanne (@DonJayamanne): All PR comments are addressed, if someone could find the time to re-review? Thanks!

Copy link
Copy Markdown
Author

Retry the CI on VSTS - uncertain why this would fail! (Cannot reproduce it on Ubuntu linux...)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

LGTM

I left a few comments, but they're not that big a deal.

this.fs,
this.platformData,
this.workspace,
new RequestWithProxy(this.workspace.getConfiguration('http').get('proxy', '')),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Mashing this into one line is okay if the risk of exceptions from getConfiguration() and get('proxy') are relatively nil (which AFAICS they are). Otherwise stack traces are a little harder to follow. I'll also argue that this is a bit less readable. :)

I'll leave it up to you.

proxy: this.proxyUri
};
} else {
return;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I though you were going with return {};. It's fine either way. Mostly this syntax (mixed bare return and non-bare return in the same function) weirds me out! that and the whole return-CoreOptions-OR-undefined makes the function feel much more complex than it actually is.

That said, it's not a huge deal. :) I'll leave this up to you.

}
return request(uri);
const requestOptions: request.CoreOptions | undefined = this.requestOptions;
return request(uri, requestOptions);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

+1

Comment thread src/test/mocks/vsc/extHostedTypes.ts Outdated
return edits ? edits.slice() : undefined;
}

createFile(uri: vscUri.URI, options?: { overwrite?: boolean, ignoreIfExists?: boolean }): void {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Whoa! I didn't realize we had any pre-built testing doubles! (This is more of a fake than a mock, BTW.) I thought we only use mocks.

See:

Derek Keeler (d3r3kk) dismissed Don Jayamanne (DonJayamanne)’s stale review August 24, 2018 19:01

All PR comments have been addressed or forwarded to #2421.

Derek Keeler (d3r3kk) merged commit 3ab005e into microsoft:master Aug 24, 2018
Derek Keeler (d3r3kk) deleted the issue2385_proxy_download branch August 24, 2018 20:09
lock Bot locked as resolved and limited conversation to collaborators Jul 31, 2019
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support downloading the language server through proxy settings

4 participants


Back | FazBrowse Home | New Git URL