Enable recursion into OneDrive by SteveL-MSFT · Pull Request #9509 · PowerShell/PowerShell · GitHub

PR Summary

The current logic of Get-ChildItem -Recurse for the FileSystemProvider is that if it's a symlink, we don't recurse into it unless -FollowSymLink is specified. However, OneDrive reparse points aren't symlinks (in that they don't link to the local filesystem). There is another flag (on Windows) to indicate if the reparse point is a named surrogate which means it IS a symlink. So the fix is to check for this flag and if it's not a named surrogate (aka symlink), it is ok to recurse into it.

Manually verified with OneDrive.

PR Context

Fix #9461

PR Checklist

PR has a meaningful title
- Use the present tense and imperative mood when describing your changes
Summarized changes
Change is not breaking
Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
This PR is ready to merge and is not Work in Progress.
- If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.
User-facing changes
- Not Applicable
- OR
- Documentation needed
  - Issue filed:
Testing - New and feature
- N/A or can only be tested interactively
- OR
- Make sure you've added a new test if existing tests do not effectively test the code changed

Impressive fix... relying on very undocumented stuff if you ask me... unless "named surrogates" and "data.dwReserved0 & 0x20000000" is documented somewhere. Steve Lee (@SteveL-MSFT)?

my understanding is that there won't be a 6.3 before 7.0, correct?

Stéphane BARIZIEN (@sba923) correct, next version is 7.0-preview.1. As noted in the comments in the change, the use of dwReserved0 & 0x20000000 is documented via the macro published in the Windows SDK header. I agree that it would have been better as an API (or even better just another FileAttribute), but that's what we have to work with right now.

Steve Lee (@SteveL-MSFT) Thanks for the confirmation about 7.0. I had missed the comment with the reference to the IsReparseTagNameSurrogate macro, sorry for that. Reparse point tags are documented here, this documents the N (Name surrogate) bit with value 0x20000000, and links to another documentation for the macro.

The documentation for WIN32_FIND_DATAA documents that symlinks use a reparse point tag of IO_REPARSE_TAG_SYMLINK (0xA000000C) that has the N bit set. To satisfy my curiosity, I will check what reparse point tag (with the N bit not set) is used by OneDrive (and request that it be added to the documentation).

Perhaps we could use method with fewer parameters
https://github.com/dotnet/corefx/blob/87089ef4559605f8a34ca8959a0ef9c71aa1e02e/src/System.IO.FileSystem/src/System/IO/FileSystem.Windows.cs#L241

I do very wonder that CoreFX uses IsNameSurrogateReparsePoint() method only for recurse directory remove but not for recurse directory read.

CoreFx uses FindFirstFileEx() as well, they just name their wrapper FindFirstFile()

On the second point, I agree it might be an oversight that they don't have symmetric behavior.

I've collected my findings about the reparse tags used by OneDrive in this spreadsheet.

https://docs.microsoft.com/en-us/windows/desktop/w8cookbook/placeholder-files is definitely outdated, as Windows 10 OneDrive doesn't use IO_REPARSE_TAG_FILE_PLACEHOLDER anymore. Instead, it uses a number of tags from IO_REPARSE_TAG_CLOUD_1 to IO_REPARSE_TAG_CLOUD_F (on my system I found only 3 values being used), with a IO_REPARSE_TAG_CLOUD_MASK that apparently can be used to compute (tag & IO_REPARSE_TAG_CLOUD_MASK) >> 12 and get some value between 1 and 15 that the OneDrive sync client probably uses as information about the item...

I'm not sure where to post the request to document all this. Steve Lee (@SteveL-MSFT): can you help?

For now I've opened MicrosoftDocs/feedback#1484 and MicrosoftDocs/feedback#1483

This returns a SafeFileHandle, which eventually calls CloseHandle.
According to the FindFirstFileExW documentation, the handle should be closed with FindClose, not CloseHandle.

Nice catch! Will submit a new PR to address.

#10405

Enable recursion into OneDrive

d014114

Steve Lee (SteveL-MSFT) requested review from Bruce Payette (BrucePay) and Andrew (anmenaga) as code owners May 1, 2019 18:33

Steve Lee (SteveL-MSFT) requested a review from Aditya Patwardhan (adityapatwardhan) May 1, 2019 18:34

Steve Lee (SteveL-MSFT) assigned Aditya Patwardhan (adityapatwardhan) May 1, 2019

Ilya (iSazonov) reviewed May 2, 2019

View reviewed changes

Comment thread src/System.Management.Automation/namespaces/FileSystemProvider.cs Outdated

Comment thread src/System.Management.Automation/namespaces/FileSystemProvider.cs Outdated

Ilya (iSazonov) added the CL-General Indicates that a PR should be marked as a general cmdlet change in the Change Log label May 2, 2019

Ilya (iSazonov) added this to the 7.0.0-preview.1 milestone May 2, 2019

address Ilya's feedback

1df6183

Ilya (iSazonov) reviewed May 2, 2019

View reviewed changes

Comment thread src/System.Management.Automation/namespaces/FileSystemProvider.cs Outdated

removed unused method

5032769

Ilya (iSazonov) approved these changes May 3, 2019

View reviewed changes

Ilya (iSazonov) reviewed May 4, 2019

View reviewed changes

Aditya Patwardhan (adityapatwardhan) approved these changes May 7, 2019

View reviewed changes

Aditya Patwardhan (adityapatwardhan) merged commit 1d28144 into PowerShell:master May 7, 2019

Steve Lee (SteveL-MSFT) deleted the ondrive-recurse branch May 12, 2019 20:04

Steve Lee (SteveL-MSFT) mentioned this pull request Jun 13, 2019

Fix auto-downloading of files in OneDrive when doing dir #9895

Merged

12 tasks

Kalle Olavi Niemitalo (KalleOlaviNiemitalo) reviewed Aug 21, 2019

View reviewed changes

Steve Lee (SteveL-MSFT) mentioned this pull request Aug 21, 2019

Minor fix for recursion into OneDrive - Change FindFirstFileEx() to use SafeFindHandle type #10405

Merged

14 tasks

Ilya (iSazonov) mentioned this pull request Aug 31, 2021

Add internal AppExecLink support to link APIs dotnet/runtime#58233

Closed

Thatgfsj (Thatgfsj) pushed a commit to Thatgfsj/PowerShell that referenced this pull request Aug 6, 2026

Enable recursion into OneDrive (PowerShell#9509 )

2c2912c

+                      {
+              #if !UNIX
+                          var data = new WIN32_FIND_DATA();
+                          using (SafeFileHandle handle = FindFirstFileEx(filePath, FINDEX_INFO_LEVELS.FindExInfoBasic, ref data, FINDEX_SEARCH_OPS.FindExSearchNameMatch, IntPtr.Zero, 0))

                           IntPtr hTemplateFile);
+                      [DllImport(PinvokeDllNames.FindFirstFileDllName, EntryPoint = "FindFirstFileExW", SetLastError = true, CharSet = CharSet.Unicode)]
+                      private static extern SafeFileHandle FindFirstFileEx(string lpFileName, FINDEX_INFO_LEVELS fInfoLevelId, ref WIN32_FIND_DATA lpFindFileData, FINDEX_SEARCH_OPS fSearchOp, IntPtr lpSearchFilter, int dwAdditionalFlags);

Enable recursion into OneDrive by SteveL-MSFT · Pull Request #9509 · PowerShell/PowerShell · GitHub

Uh oh!

Enable recursion into OneDrive - #9509

Enable recursion into OneDrive#9509Aditya Patwardhan (adityapatwardhan) merged 3 commits intoPowerShell:masterPowerShell/PowerShell:masterfrom SteveL-MSFT:ondrive-recurseSteveL-MSFT/PowerShell:ondrive-recurseCopy head branch name to clipboard

Conversation

Steve Lee (SteveL-MSFT) commented May 1, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

PR Context

PR Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Stéphane BARIZIEN (sba923) commented May 3, 2019

Uh oh!

Steve Lee (SteveL-MSFT) commented May 3, 2019

Uh oh!

Stéphane BARIZIEN (sba923) commented May 3, 2019

Uh oh!

Ilya (iSazonov) May 4, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Steve Lee (SteveL-MSFT) May 4, 2019

Choose a reason for hiding this comment

Uh oh!

Steve Lee (SteveL-MSFT) May 4, 2019

Choose a reason for hiding this comment

Uh oh!

Stéphane BARIZIEN (sba923) commented May 12, 2019

Uh oh!

Stéphane BARIZIEN (sba923) commented May 12, 2019

Uh oh!

Kalle Olavi Niemitalo (KalleOlaviNiemitalo) Aug 21, 2019

Choose a reason for hiding this comment

Uh oh!

Steve Lee (SteveL-MSFT) Aug 21, 2019

Choose a reason for hiding this comment

Uh oh!

Steve Lee (SteveL-MSFT) Aug 21, 2019

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Enable recursion into OneDrive#9509
Aditya Patwardhan (adityapatwardhan) merged 3 commits into
PowerShell:masterfrom
SteveL-MSFT:ondrive-recurse

Steve Lee (SteveL-MSFT) commented May 1, 2019 •
edited
Loading

Ilya (iSazonov) May 4, 2019 •
edited
Loading