FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

$PSVersionTable.GitCommitId based on generated constant by iSazonov · Pull Request #3690 · PowerShell/PowerShell · GitHub

$PSVersionTable.GitCommitId based on generated constant - #3690

Closed
Ilya (iSazonov) wants to merge 5 commits into
PowerShell:masterfrom
iSazonov:gitcommitid
Closed

$PSVersionTable.GitCommitId based on generated constant#3690
Ilya (iSazonov) wants to merge 5 commits into
PowerShell:masterfrom
iSazonov:gitcommitid

Conversation

Ilya (iSazonov) commented May 3, 2017
edited
Loading

Copy link
Copy Markdown
Collaborator

Resolve #3676
Fix #3739

Generate C# file with a constant containing current version GitCommitId.
Correct a test for GitCommitId.

Jason Shirk (lzybkr) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Maybe in another PR, or this one if you prefer, but it would be nice to also include the commit hash in the version info similar to what the clr does:

@ C:\Program Files\PowerShell\6.0.0-alpha.18
#293 PS> (dir .\System.Collections.dll).VersionInfo.ProductVersion
4.6.24705.01. Commit Hash: 4d1af962ca0fede10beb01d197367c2f90e92c97

Comment thread build.psm1 Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

We should stop generating this file.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I use the file in a test. Can we leave the file in repo but exclude it from release and publish?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I don't see that file being used in any tests. i do see the string powershell.version in a test, but it sort of looks like a broken test to me.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Yes, it is a broken test and I added a fix in the PR.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I see. It might be fine if you generated the file alongside other test artifacts, but I would not create it in $PSHome.

Copy link
Copy Markdown
Collaborator Author

Jason Shirk (@lzybkr) Please clarify - currently git --git-dir=".git" describe --dirty --abbrev=60 return v6.0.0-alpha.18-60-gb09e9b570c757b9869f454276aa708eac307f5b1 where last part is a hash - do you want change formatting? v6.0.0-alpha.18-60 Commit Hash: gb09e9b570c757b9869f454276aa708eac307f5b1

Copy link
Copy Markdown
Contributor

I'm always a fan of better formatting.

I suggest reading the documentation on describe, and we might consider using --long to always get the hash.

Note that the g is not part of the hash, the format (with --long) is {tag}-{# of commits past tag}-g{hash of head}.

Copy link
Copy Markdown
Collaborator Author

We would parse the long formatted "git describe" and output in the desired form, ex.:
Version: v6.0.0-alpha.18 Commits: 60 Commit Hash: b09e9b570c757b9869f454276aa708eac307f5b1

Copy link
Copy Markdown
Contributor

Sounds good, though I would omit Commits if it is 0 (which will be the common case) and maybe use the phrase Additional Commits.

Copy link
Copy Markdown
Collaborator Author

Jason Shirk (@lzybkr) powershell.version is still used in Install-PowerShellRemoting.ps1.

I see #2671 Should we fix the script here? If yes how can we fix this? Can we use there PSVersionTable.PSVersion?

/cc Mike Richmond (@mirichmo)

Copy link
Copy Markdown
Collaborator Author

I added commit to fix #3739

Dongbo Wang (daxian-dbw) commented May 10, 2017
edited
Loading

Copy link
Copy Markdown
Member

Ilya (@iSazonov) thanks for fixing this. The hard-coded alpha string caused me huge effort in beta.1 release.
I have 2 comments:

  1. Could you please resolve the conflicts?
  2. If you search the usage of powershell.version, you will find it appears in following files:
E:\PowerShell\.gitignore
E:\PowerShell\.spelling
E:\PowerShell\build.psm1
E:\PowerShell\build.sh
E:\PowerShell\build.sh
E:\PowerShell\src\powershell-native\Install-PowerShellRemoting.ps1
E:\PowerShell\src\powershell-unix\powershell-unix.csproj
E:\PowerShell\src\powershell-win-core\powershell-win-core.csproj
E:\PowerShell\src\System.Management.Automation\engine\PSVersionInfo.cs
E:\PowerShell\src\System.Management.Automation\engine\PSVersionInfo.cs
E:\PowerShell\test\powershell\Host\PSVersionTable.Tests.ps1

Some of those files are not updated in this PR. I think all appearances of powershell.version need to be reviewed and updated.

Copy link
Copy Markdown
Collaborator Author

This is the third conflict over the past 24 hours in build.psm1 😄 - I'll resolve tomorrow.

I use powershell.version file in tests. So you might have to leave it in .gitignore

I'm afraid that I won't be able to fix build.sh - can I delegate this anybody?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I'm not sure if this test is really useful. The content contained in powershell.version is guaranteed (by the build script) to be the same as in PSVersionInfo.generated.cs. The only possible failure I can see for this test is that we are running tests outside of where powershell was being built and thus powershell.vesrion is not available.

I believe it's more useful to check if both GitCommitId and PSVersion are nonempty and GitCommitId starts with PSVersion, or something like that.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I tried to do the test with direct checking. It seems indirect validation will catch a bug too.

Fixed.

Comment thread build.psm1 Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This section should be moved to a separate function (better to make the function invisible from outside the module, but not required) so that Start-PSBuild won't keep expanding its size 😄
And also, move the call to that function after we check/do ResGen, so that the System.Management.Automation\gen folder is guaranteed to exist and the New-Item line can be removed.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Fixed.

Copy link
Copy Markdown
Member

Ilya (@iSazonov) Sorry that I have to change build.psm1 many times in the past day -- obviously our release process and test infrastructure are still far away from perfect 😦

As for build.sh, writing an equivalent bash script to do the same would be (1) not easy (2) hard to maintain 2 scripts for the same thing in future. One option is to extract this logic and write C# code to do it, like the existing ResGen and TypeCatalogGen. Then we can just call dotnet run from both powershell script and bash script.

I'm not sure how many people are using build.sh honestly, and not really sure if this worth the effort.
Steve Lee (@SteveL-MSFT) Joey Aiello (@joeyaiello) Jason Shirk (@lzybkr) do we have any data to show how important build.sh is? Can we just remove that support?

Ilya (iSazonov) commented May 11, 2017
edited
Loading

Copy link
Copy Markdown
Collaborator Author

appveyor temporaryly failed - I restart.

Copy link
Copy Markdown
Collaborator Author

I only fix but don't test 'Install-PowerShellRemoting.ps1'.

Copy link
Copy Markdown
Contributor

Ilya (@iSazonov) - I ran across this relevant blog post which suggests an assembly attribute that we should also use.

I think we can ignore the part in that blog post about finding the sha without git - we've been using git to do this already with no complaints.

Copy link
Copy Markdown
Collaborator

This is exactly what we use. Used to use GitVersion but now we're doing our own computation of build metadata:

Get-FileVersionInfo P:\Repositories\BuildResults\Foo_Master_CompleteBuild\5.2.0.105\64BitDevelopment
\Release\Foo.dll | ForEach-Object ProductVersion

5.2.0+105.Branch.master.Sha.31e45082cd1093133b6687dd172daa04b2a10940.Built.20170505T1739

And we do use git to get the SHA1, branch, etc. This is the function we use:

function GetGitBuildMetadata([string]$SemVer, [string]$Revision) {
    $timestamp = Get-Date -Format yyyyMMddTHHmm
    $sha1   = git rev-parse --verify HEAD
    $branch = git rev-parse --abbrev-ref HEAD
    if (!$branch -or ($branch -eq 'HEAD')) {
        if ($env:BUILD_SOURCEBRANCH) {
            $branch = $env:BUILD_SOURCEBRANCH.Trim().Replace("refs/heads/", "").Replace("refs/","")
        }
        else {
            $branch = 'unknown'
        }
    }

    $prereleaseTag = if ($branch -notmatch 'master|release') { '-prerelease' } else { '' }

    $metadata = "${SemVer}${prereleaseTag}+${Revision}.Branch.${branch}.Sha.${sha1}.Built.${timestamp}"
    $metadata
}

Copy link
Copy Markdown
Member

Closed the PR by accident 🤕

Copy link
Copy Markdown
Collaborator Author

Dongbo Wang (@daxian-dbw) GetVersion isn't still ported 😕 GitTools/GitVersion#1175

Copy link
Copy Markdown
Contributor

Dongbo Wang (@daxian-dbw) I don't believe that tampering with version attributes of a file by itself is a security concern, since are many other ways of file tampering for malicious purposes. The way to guard against file tampering is with application white listing (and black listing for deprecated versions) where approved file versions are validated through file signing or catalog signing.

Copy link
Copy Markdown
Collaborator Author

I took the next step. Now all projects have common properties and assemblies get version from Start-PSBuild.

Copy link
Copy Markdown
Member

This PR is getting too complex, and I think we'd better break it down.
We need a conclusion about how to dynamically generate PSVersion and GitCommitId. There are three proposals so far:

  1. Add script in Start-PSBuild to parse git describe string, and generate c# source file to keep the information. Pros: easy to implement. Cons: (1) hard to bring build.sh up-to-date (e.g. hard to write the equivalent bash script, hard to maintain two scripts that do the same thing) (2) make it harder to eventually build the powershell core in Visual Studio as we have another necessary step burnt into the script.
  2. Add a C# project, maybe named PSVersionGen, like ResGen and TypeCatalogGen to do the code generation. Pros: (1) easy to update build.sh, and only one implementation to maintain. (2) it may be easier to eventually build the powershell core in Visual Studio 2017. Cons: a bit harder to implement comparing using the script.
  3. Dynamically generate AssemblyFileVersionAttribute and AssemblyInformationalVersionAttribute via MSBuild, and change code in PSVersionInfo.cs to parse the FileVersion string to get PSVersion and GitCommitId when PSVersionTable is generated for the first time. Pros: clean, no magical code generation at build time, maybe the easiest to move to Visual Studio 2017 in future. Cons: hard to implement -- csproj changes + more robust logic to parse FileVersion in production code. The MSBuild part definitely need extra research and trial/error (build + NuGet package geneation both depend on csproj. A side topic, .NET Core team use different csproj files for build and package, maybe we should go that direction.)

We need to draw to a conclusion first to know what work items would be. Then we break down the work items so that they can be addressed in multiple PRs, instead of one PR with too many changes.

I vote for NO.3 proposal. Jason Shirk (@lzybkr) Ilya (@iSazonov) please share your opinions.

Copy link
Copy Markdown
Collaborator Author

In brief I agree with NO.3

Perhaps we should create a Meta Issue to split the work on small pieces:
Meta Issue - "Move Start-PSBuild to MSBuild":

  • Clean up Start-PSBuild from FullClr code and old code.
  • Generate MSBuild variables with version strings based on Git tags
  • Move common csproj properties to a common targets file.
  • Unify AssemblyFileVersionAttribute and AssemblyInformationalVersionAttribute for all assemblies.
  • Fix PSVersionInfo.cs to parse the FileVersion string or/and use MSBuild variables with version.
  • Use different csproj files for build and package.
  • ???

Generate C# file with a constant containing current version GitCommitId.
Correct test for GitCommitId.
+ Format GitCommitId version string
Fix Install-PowerShellRemoting.ps1
Fix test
Fix build.psm1
Comment thread build.psm1
if ($($matchVersion[1]) -ne "0") {
$Global:formattedVersion += " Additional commits: $($matchVersion[2])"
}
$Global:formattedVersion += " Commit Hash: $($matchVersion[3])"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

James Truher (@JamesWTruher) the telemetry you're receiving is from Ilya (@iSazonov) running a build of PowerShell with this (not yet committed to master) change.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

the new version string format breaks our telemetry. I'm not really sure that the additional data provides more benefit. Is is possible to go back to the original format?

Comment thread build.psm1
Stop-Process -Verbose
}

# Generate version constant for $PSVersionTable

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This change in formatting breaks our telemetry

Copy link
Copy Markdown
Collaborator Author

The PR shouldn't be merged - it is only for tracking multiple changes. We split the PR on some light-weight PRs.
There we are trying set assembly attributes by MSBuild based on git version tag and also expose the version to PSVersion table and PowerShell startup banner.
Some Microsoft projects (Roslin, CoreFX/CoreCLR ...) use such version format.

Jason Shirk (@lzybkr) Dongbo Wang (@daxian-dbw) Could you please comment? How can we resolve the conflict with telemetry?

Copy link
Copy Markdown
Member

The work is covered b #4863. So close this PR.

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants


Back | FazBrowse Home | New Git URL