FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Unified StripedHeap model, size-class chunk recycling and allocation-driven purge by franz1981 · Pull Request #17151 · netty/netty · GitHub

/ netty Public

Unified StripedHeap model, size-class chunk recycling and allocation-driven purge - #17151

Open
franz1981 wants to merge 10 commits into
netty:4.2from
franz1981:4.2_chunk_recycling
Open

franz1981 wants to merge 10 commits into
netty:4.2from
franz1981:4.2_chunk_recycling

Conversation

franz1981 commented Jul 27, 2026
edited
Loading

Copy link
Copy Markdown
Contributor

Motivation:

Existing performance tests shows adaptive striping model and sharing across size classes to under-perform compared to other alternatives.
In addition, the existing striped model is too eager to both allocate and throw away existing slots under contention instead lazily allocate and eventually reuse them, keeping them both stable and more cache-friendly for existing threads.
The existing shared size-class chunk cache degrade severely under heavy contention - and its algorithm is not really made to be effective under concurrent usage.
The chunk cache is not able to purge idle chunks if a specific size class end up allocating from the same magazine's current chunk, causing memory waste.

Modification:

  • Lifts the striped logic up, sharing it across a variety of allocation sizes, to better react to contention
  • Make local heaps lazily allocated and "stable" vs their stripe slot position
  • Exposed mpsc recyclers thanks to the new ownership model
  • Drop the shared size class chunk cache implementation
  • Recycle free lists (limited) and size-class chunks across "compatible" classes
  • Implemented per-class tick to drive purging, enabling recycling/deallocation of idle chunks

Result:

  • better scalability
  • (more) prompt release and reuse of memory across different size classes

Fixes #17028


Update 2026-09-22 (head 82bd38c, rebased onto current 4.2, history squashed to 10 commits; the previous head 004e4cc is kept locally as fine history):

  • Unified StripedHeap model, two-list chunk cache, release notifications and size-class chunk recycling
  • Split Chunk and Magazine by kind; keep the size-class magazine's active chunk inside its cache
  • Give each stripe its own buddy chunk cache and move the buddy tree into BuddyTree
  • Extract ChunkQueue and PendingChunks; decide a size-classed chunk's queue in one place
  • File buddy chunks by their largest free block, without a concurrent map
  • Bound the recycler and the idle buddy chunks in bytes; release buddy blocks in place
  • Family chunk sizes, one retained chunk per size class, one recycler pool per chunk size
  • Build the chunk pool tables from named steps, and test them on any size class table
  • Test the ways a segment comes back to a size-classed chunk
  • Register the allocator's updater-backed fields for native-image, and restore the LocalPool ratio comment

User-visible property changes: io.netty.allocator.chunkPurgePollsShared and io.netty.allocator.chunkPurgeThreshold are removed; io.netty.allocator.chunkPurgePollsThreadLocal default 16 -> 4; new io.netty.allocator.buddyIdleBytes and io.netty.allocator.recycledChunkBytes.

Verification: full buffer module tests on JDK 21 (14,196 run, 0 failures); mvn -pl common,buffer -am verify -DskipTests on JDK 11.0.29 with revapi ("API checks completed without failures") and checkstyle clean; the branch is rebased onto current 4.2 (conflict-free, byte-identical replay).

Numbers, head-to-head against the mimalloc Java port on its own harness (84 cells, 3 forks, 32 threads, Ryzen 9 7950X one NUMA node at 2300 MHz, JDK 21): latency geomean 0.873 and peak RSS 0.955 (ADAPTIVE / MIMALLOC, below 1 = this branch better); this branch is ahead by more than 10% in 47 cells and behind in 12. All cells, sortable: https://franz1981.github.io/netty-allocator-h2h/results/2026-09-22-x86/index.html - raw JMH JSON, RSS logs and the reduction script: https://github.com/franz1981/netty-allocator-h2h

Copy link
Copy Markdown
Contributor Author

This closes #17047 and replace it.

Copy link
Copy Markdown
Contributor Author

052ce63 has introduced a tiny performance regression which I'm still investigating.

In the meantime I'm adding other changes to make naming and classes more clear.
I'll add them as separate commits.

In order to perform tests without chunk churning, I've added 6ab4716 too, which I'll send a separate PR as well FYI @chrisvest

mags[i] = new Magazine(this, true, chunkManagementStrategy.createController(this));
}
magazines = mags;
private static final int FREELIST_POOL_COUNT; // number of distinct freelist capacity buckets

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I need to verify it yet

private final MpscIntQueue[] freelistSlots = new MpscIntQueue[FREELIST_POOL_COUNT];
private final IntStack[] localFreelistSlots = new IntStack[FREELIST_POOL_COUNT];

private static final int TARGET_RECYCLED_BYTES = 4 * 1024 * 1024;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

The "shared" bin target 4 MB per local heap


private Magazine createMagazine(int sizeClassIndex, AdaptivePoolingAllocator allocator) {
if (recycler == null) {
recycler = Magazine.AdaptiveRecycler.sharedMpsc(MAGAZINE_BUFFER_QUEUE_CAPACITY);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

bad name, I prefer exclusiveAcquire or something similar: mpsc is way too anonymous

static {
NEXT_IN_LINE = AtomicReferenceFieldUpdater.newUpdater(Magazine.class, Chunk.class, "nextInLine");
}
private static int threadIndex(Thread t) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

we want some better shuffling than what we had before

// In any case we will store the Chunk as the current so it will be used again for the next allocation and
// thus be "reserved" by this Magazine for exclusive usage.
curr = NEXT_IN_LINE.getAndSet(this, null);
curr = nextInLine;

franz1981 Jul 27, 2026
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

no reason anymore to have a xchg here as it is always guarded by the stripe lock; we don't allow allocating outside the lock as it can severely escape the reason why the striped lock exist in primis

@@ -279,7 +279,7 @@ void purgeScanShouldEvictIdleChunks(boolean threadLocal) throws Exception {
AdaptiveByteBufAllocator allocator = new AdaptiveByteBufAllocator(false, threadLocal);
long purgePolls = threadLocal ?
AdaptivePoolingAllocator.CHUNK_PURGE_POLLS_THREAD_LOCAL :

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

this is ugly; need fixing it!

}

@SuppressWarnings("unchecked")
protected Recycler(int maxCapacity, boolean unguarded, boolean mpsc) {

franz1981 Jul 27, 2026
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

mpsc is ugly, better sharedGet or something similar
and need tests to hit this too

Copy link
Copy Markdown
Contributor Author

I see that the current model on this pr would enable a minalloc style local release path which would benefit to prevent chunk churning and the artificial floor of 8M per size class cache which is applied to 4.2.
@laosijikaichele I think we have to rethink the cache management to look more like what minalloc does.
Any suggestion is welcome to keep a better balance between memory usage and chunk churning with many live buffers

Copy link
Copy Markdown
Contributor

I see that the current model on this pr would enable a minalloc style local release path which would benefit to prevent chunk churning and the artificial floor of 8M per size class cache which is applied to 4.2.

@laosijikaichele I think we have to rethink the cache management to look more like what minalloc does.

Any suggestion is welcome to keep a better balance between memory usage and chunk churning with many live buffers

Thanks for reaching out. I'm currently away on vacation and will take a look once I'm back.

Copy link
Copy Markdown
Contributor Author

@laosijikaichele you can ignore the big amount of changes I have here, please: in the final version I plan to split into separate files the allocator key components too 🙏
This is using a mechanism very similar to mimalloc (withou biasing) - and notification-based, in order to avoid having floors/ceiling on the live set of allocations.
Is not as optimal as it should, because I've implement freelists recycling, not as simple as mimalloc does with recycled "blocks", but is quite effective as it is shared among many size-classes.

In any case this grant me a very very good performance for the event loop case, and the platform one, fixing some of the shortcomings of #17166

franz1981 commented Aug 26, 2026
edited
Loading

Copy link
Copy Markdown
Contributor Author

Just for myself: the epoch based mechanism on top of new striped model was impl at 191f507
while the changes we have so far move this to a notification based approach which remove the ceil of the size class chunk cache, allowing buffers releases to promptly made available or retire a chunk (into a shared recycling parking lot).
There are two reasons for this change:

  • if the ceil is too high, a cache retain too much memory, which requires further allocations of the same size to slowly purge them (back to the OS)
  • If the ceil is too small, it cause excessive OS malloc/free to respect the limit and still make purged chunks unable to further satisfy incoming allocations, in a vicious circle which cause other temporary fresh chunk allocations

In the new notification based model, instead, chunks will be allocated to serve the incoming requests and accomodate for the live required buffers, but, once released, will be made available or, if fully released, recycled in the shared parking lot, to serve future demand.
If a chunk is fully released and recycled in the parking lot but exceeding its capacity, it will be freed back to the OS.
That means that the total available cached memory is still bounded, but once a buffer is allocated it has the chance to be cached and reused as many time as required

This comment has been minimized.

…s and size-class chunk recycling

Squash of:
- Unified StripedHeap model with cross-size-class chunk recycling
- Replace ring buffer cache with mimalloc-style two-list chunk cache
- Add shared-path signal detection via offerAndGetSize + tryWriteLock
- Attempt tryWriteLock on every cross-thread release (mimalloc-style)
- Filter tryWriteLock to transition boundaries, cleanup
- Make owningCache final — set once at construction, never changes
- Add local freelists for shared chunks, tryWriteLock-first release
- Remove dead localFreeList null checks, simplify
- Extract shared eviction logic, fix stale javadocs
- Tiered purge: frequent bounded + rare unbounded sweep
- Fix chunk leak on locked release path; bound post-hit scan walk
- MpscIntQueue: resetAndFill must wait for already-claimed slots
- Stage 1: notification queue for cross-thread segment returns
- Stage 2: delete the exhausted-list scans, keep fullSweep as a detector
- Bounded probe of the exhausted list before allocating a fresh chunk
- Make the cache Javadoc precise about what is trustworthy and what is not
- fullSweep is a third recovery route, not a violation detector
- Do not fire a FreeChunk event for a chunk whose buffer was recycled
- AdaptivePoolingAllocator: drop redundant fully-qualified java.util references
- Magazine.allocate: extract slow path into allocateSlow
- SizeClassedChunk.releaseSegment: shrink the owner-thread fast path
- Drop dead isReallocation/reallocate parameter from the chunk-capacity path
- Remove the write-only RESCUED_BY_FULL_SWEEP counter
- Remove the exhausted-list sweep from the purge tick
- Make the release-lock contract explicit in ThreadLocalSizeClassedChunkCache
- Fold the single-subclass cache base into its implementation
- Return the purge-tick signal instead of storing it on the magazine
- Pool every recycled buffer on the thread-local magazine path
- Ignore revapi additions to the MpscIntQueue interface
- Tick the purge on the size-classed path only, and let the magazine sweep its own siblings
- Add size classes up to 128 KiB to the adaptive allocator
- Carry a chunk's free lists with its buffer through the SizeClassChunkRecycler
…ve chunk inside its cache

Squash of:
- Make Chunk abstract and give the unpooled fallback its own chunk class
- Split Magazine into a size-class magazine and a buddy magazine
- Drop buddy-shaped leftovers from the size-class allocation path
- Remove native-image reflection entries for fields no longer accessed reflectively
- Keep the fallback to a fresh chunk if the size-class cache ever hands out a full chunk
- Release a size-class chunk that fails an allocation instead of retrying it
- Let the size-class cache hold an active chunk at the head of its reusable list
- Keep a size-class magazine's current chunk inside its cache
- Describe the active chunk in the size-class cache documentation
- Test the size-class magazine's active chunk inside its cache
- Drop the size-classed chunk's magazine back-reference
- Tighten the size-class cache's assertions and comments
- Test the size-class magazine's active chunk by behaviour
- Pin chunk allocations and used memory on a seeded allocation trace
- Pin the unpooled fallback: accounted while alive, replaced on growth, freed on release
- Make the unpooled fallback a one-shot BuddyChunk
…to BuddyTree

Squash of:
- Pin buddy chunk reuse across rounds of the same large allocations
- Give each stripe its own buddy chunk cache
- Move the buddy tree of BuddyChunk into its own BuddyTree
- Test BuddyTree against a frozen copy of its current search
- Store the largest free order in each BuddyTree node
…ueue in one place

Squash of:
- Keep the size-class cache's active chunk off its lists
- Extract ChunkQueue from the size-class cache's two lists
- Record a size-classed chunk's cache membership as its ChunkQueue
- Extract PendingChunks from the size-class cache
- Decide a size-classed chunk's queue in one place, SizeClassedChunkCache.refile
- Fix comments left behind by the size-class queue refactor
- Keep ChunkQueue and PendingChunks links on Chunk
Squash of:
- Expose a BuddyTree's largest free order and whether it is wholly free
- Pin buddy chunk reuse after foreign releases, the idle bound, and cross-thread content
- File buddy chunks by their largest free block, without a concurrent map
- Bound only the wholly free buddy chunks by CHUNK_REUSE_QUEUE
- Describe the allocator as it is now, and derive chunk geometry in the buddy tests
- Allocate from the buddy chunk with the largest free block
…blocks in place

Squash of:
- Bound the idle size-classed chunks by the retention floor, not every chunk
- Revert "Bound the idle size-classed chunks by the retention floor, not every chunk"
- Bound the chunk recycler by one byte budget per heap
- Bound a buddy magazine's idle chunks in bytes
- Let a buddy release act in place when the stripe lock is free
…ool per chunk size

Squash of:
- Give the size classes from 32 KiB up chunks of 8 segments
- Skip this magazine's own cache when draining the heap's pending notes
- Retain one chunk per size class instead of a byte-budgeted floor
- Share a recycler pool between all size classes with the same chunk size
- Give the size classes from 16 KiB up the chunk size of their family
…ze class table

Motivation:

Which size classes share a recycler pool was computed by one nested loop in the static initialiser. An earlier
version of it merged only adjacent size classes with the same chunk size, and nothing noticed when a change of the
chunk geometry made equal chunk sizes non-adjacent: the pools silently stopped being shared.

Modification:

The tables are built by distinctChunkSizes(sizeClasses) and chunkPools(sizeClasses, chunkSizes), which take the
size class table as a parameter. One test checks that today's size classes share a pool exactly when their chunk
sizes are equal; another checks the same on 1000 seeded random tables, where equal chunk sizes are rarely adjacent.

Result:

The same tables (the recycler's bytecode is unchanged), and a geometry change that breaks pool sharing fails a test.
Motivation:

Four behaviours of a size-classed chunk had no test of their own: what it reports about its free segments
through every way a segment comes back, the deallocation of a freed allocator's chunk by a release that cannot take
the stripe lock, the segment a growing buffer leaves behind, and allocation racing releases from other threads.

Modification:

Add capacityQueriesFollowEveryWayASegmentComesBack, segmentReturnedExternallyAfterFreeMustStillDeallocateChunk,
aBufferThatOutgrowsItsSegmentGivesItBack and segmentsReleasedByOtherThreadsAreNeverHandedOutTwice, and share the
reflective freeHeap helper with the existing test that inlined it.

Result:

Each test fails when the code path it names is broken: the external-release deallocation skipped, an external
offer duplicated, the old segment kept on growth, hasFullCapacity ignoring the external list.
franz1981 force-pushed the 4.2_chunk_recycling branch 2 times, most recently from b3f70fd to 82bd38c Compare September 22, 2026 19:04
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AdaptivePoolingAllocator: Performance gap in latency/RSS in several benchmark cases compared with other allocators

2 participants


Back | FazBrowse Home | New Git URL