Improve cache for match, split, replace operators and WildcardPattern by iSazonov · Pull Request #10657 · PowerShell/PowerShell · GitHub

PR Summary

Fix #8941

Add new cache for case-sensitive Regex-s
Use StringComparer.Ordinal in the cache (that will slightly speed up and reduce allocations)
Use the cache in regex.cs (that will slightly speed up cmdlets)
Make the cache two-level to take in account all regex option combinations.

PR Context

Before the change we cached only case-insensitive Regex-s (with RegexOptions.IgnoreCase).

PR Checklist

PR has a meaningful title
- Use the present tense and imperative mood when describing your changes
Summarized changes
Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
This PR is ready to merge and is not Work in Progress.
- If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.
Breaking changes
- None
- OR
- Experimental feature(s) needed
  - Experimental feature name(s):
User-facing changes
- Not Applicable
- OR
- Documentation needed
  - Issue filed:
Testing - New and feature
- N/A or can only be tested interactively
- OR
- Make sure you've added a new test if existing tests do not effectively test the code changed
Tooling
- I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
- OR
- I have considered the user experience from a tooling perspective and enumerated concerns in the summary. This may include:
  - Impact on PowerShell Editor Services which is used in the PowerShell extension for VSCode (which runs in a different PS Host).
  - Impact on Completions (both in the console and in editors) - one of PowerShell's most powerful features.
  - Impact on PSScriptAnalyzer (which provides linting & formatting in the editor extensions).
  - Impact on EditorSyntax (which provides syntax highlighting with in VSCode, GitHub, and many other editors).

/cc Michael Klement (@mklement0) if you are interested to see and measure results.

Add new cache for ignore case Regex-s Use StringComparer.Ordinal in the cache Use the cache in regex.cs

Co-Authored-By: Steve Lee <slee@microsoft.com>

Having this lambda expression in this method will result in an instance of the helper class to be created every time NewRegex gets called, even if the query can be served by the cache.

This should be changed to the following:

else
{
    if (subordinateRegexCache.Count > MaxRegexCache)
    {
        subordinateRegexCache.Clear();
    }

    var regex = new Regex(patternString, options);
    return subordinateRegexCache.GetOrAdd(patternString, regex);
}

Note that, the ValueFactory delegate is called outside the locks and thus it's not atomic, meaning there is no difference clearing the dictionary this way from doing it in the ValueFactory delegate.

Thanks for great comment!

Done.

LGTM

🎉v7.0.0-preview.6 has been released which incorporates this pull request.:tada:

Handy links:

Release Notes

…0657)

Ilya (iSazonov) added the CL-Performance Indicates that a PR should be marked as a performance improvement in the Change Log label Oct 1, 2019

Ilya (iSazonov) added this to the 7.0.0-preview.5 milestone Oct 1, 2019

Ilya (iSazonov) requested review from Steve Lee (SteveL-MSFT) and Dongbo Wang (daxian-dbw) October 1, 2019 07:41

PoshChan-Bot (PoshChan) reviewed Oct 1, 2019

View reviewed changes

Comment thread src/System.Management.Automation/engine/lang/parserutils.cs Outdated

PoshChan-Bot (PoshChan) reviewed Oct 1, 2019

View reviewed changes

Comment thread src/System.Management.Automation/engine/lang/parserutils.cs Outdated

PoshChan-Bot (PoshChan) reviewed Oct 1, 2019

View reviewed changes

Comment thread src/System.Management.Automation/engine/lang/parserutils.cs Outdated

Ilya (iSazonov) changed the title Improve cache for match, split and replace operators and WildcardPattern Improve cache for match, split, replace operators and WildcardPattern Oct 1, 2019

Steve Lee (SteveL-MSFT) requested changes Oct 1, 2019

View reviewed changes

Comment thread src/System.Management.Automation/engine/lang/parserutils.cs Outdated

Comment thread src/System.Management.Automation/engine/lang/parserutils.cs Outdated

Comment thread src/System.Management.Automation/engine/lang/parserutils.cs Outdated

ghost added Waiting on Author The PR was reviewed and requires changes or comments from the author before being accept and removed Waiting on Author The PR was reviewed and requires changes or comments from the author before being accept labels Oct 1, 2019

Ilya (iSazonov) and others added 6 commits October 21, 2019 17:10

Improve cache for match, split and replace operators and WildcardPattern

c8e98d1

Add new cache for ignore case Regex-s Use StringComparer.Ordinal in the cache Use the cache in regex.cs

Fix CodeFactor issues

86757a1

Add 2-level cache

df42f0c

Fix some CodeFactor issues

d4b468d

Update src/System.Management.Automation/engine/lang/parserutils.cs

6b4f05a

Co-Authored-By: Steve Lee <slee@microsoft.com>

Address feedback

c3fee9c

Ilya (iSazonov) force-pushed the perf-cmatch-add-cache branch from 8799f38 to c3fee9c Compare October 21, 2019 12:11

PoshChan-Bot (PoshChan) reviewed Oct 21, 2019

View reviewed changes

Comment thread src/System.Management.Automation/engine/lang/parserutils.cs

Steve Lee (SteveL-MSFT) assigned Dongbo Wang (daxian-dbw) Oct 21, 2019

Steve Lee (SteveL-MSFT) approved these changes Oct 21, 2019

View reviewed changes

Ilya (iSazonov) modified the milestones: 7.0.0-preview.5, 7.0.0-preview.6 Oct 24, 2019

Dongbo Wang (daxian-dbw) requested changes Oct 31, 2019

View reviewed changes

ghost added the Waiting on Author The PR was reviewed and requires changes or comments from the author before being accept label Oct 31, 2019

Address Dongbo's feedback

9c315e4

ghost removed the Waiting on Author The PR was reviewed and requires changes or comments from the author before being accept label Nov 1, 2019

Dongbo Wang (daxian-dbw) approved these changes Nov 1, 2019

View reviewed changes

Dongbo Wang (daxian-dbw) merged commit 54edaee into PowerShell:master Nov 1, 2019

Ilya (iSazonov) deleted the perf-cmatch-add-cache branch November 1, 2019 18:28

ghost mentioned this pull request Nov 21, 2019

Performance problem: -cmatch prevents automatic caching of on-demand-compiled regexes #8941

Closed

Thatgfsj (Thatgfsj) pushed a commit to Thatgfsj/PowerShell that referenced this pull request Aug 6, 2026

Improve the caching when creating new Regex instances (PowerShell#1…

0b364f8

…0657)

Improve cache for match, split, replace operators and WildcardPattern by iSazonov · Pull Request #10657 · PowerShell/PowerShell · GitHub

Uh oh!

Improve cache for match, split, replace operators and WildcardPattern - #10657

Improve cache for match, split, replace operators and WildcardPattern#10657
Dongbo Wang (daxian-dbw) merged 7 commits into
PowerShell:masterfrom
iSazonov:perf-cmatch-add-cache

Ilya (iSazonov) commented Oct 1, 2019 •
edited
Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Ilya (iSazonov) commented Oct 1, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Dongbo Wang (daxian-dbw) Oct 31, 2019 •
edited
Loading

Uh oh!

Uh oh!

Ilya (iSazonov) Nov 1, 2019

Uh oh!

Dongbo Wang (daxian-dbw) left a comment

Uh oh!

Deleted user (ghost) commented Nov 21, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

+                          }
+                          else
+                          {
+                              return subordinateRegexCache.GetOrAdd(patternString, key =>

Improve cache for match, split, replace operators and WildcardPattern by iSazonov · Pull Request #10657 · PowerShell/PowerShell · GitHub

Uh oh!

Improve cache for match, split, replace operators and WildcardPattern - #10657

Improve cache for match, split, replace operators and WildcardPattern#10657Dongbo Wang (daxian-dbw) merged 7 commits intoPowerShell:masterPowerShell/PowerShell:masterfrom iSazonov:perf-cmatch-add-cacheiSazonov/PowerShell:perf-cmatch-add-cacheCopy head branch name to clipboard

Conversation

Ilya (iSazonov) commented Oct 1, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

PR Context

PR Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Ilya (iSazonov) commented Oct 1, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Dongbo Wang (daxian-dbw) Oct 31, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ilya (iSazonov) Nov 1, 2019

Choose a reason for hiding this comment

Uh oh!

Dongbo Wang (daxian-dbw) left a comment

Choose a reason for hiding this comment

Uh oh!

Deleted user (ghost) commented Nov 21, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Improve cache for match, split, replace operators and WildcardPattern#10657
Dongbo Wang (daxian-dbw) merged 7 commits into
PowerShell:masterfrom
iSazonov:perf-cmatch-add-cache

Ilya (iSazonov) commented Oct 1, 2019 •
edited
Loading

Dongbo Wang (daxian-dbw) Oct 31, 2019 •
edited
Loading