| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
|
This closes #17047 and replace it. |
Sorry, something went wrong.
|
052ce63 has introduced a tiny performance regression which I'm still investigating. In the meantime I'm adding other changes to make naming and classes more clear. In order to perform tests without chunk churning, I've added 6ab4716 too, which I'll send a separate PR as well FYI @chrisvest |
Sorry, something went wrong.
| mags[i] = new Magazine(this, true, chunkManagementStrategy.createController(this)); | ||
| } | ||
| magazines = mags; | ||
| private static final int FREELIST_POOL_COUNT; // number of distinct freelist capacity buckets |
There was a problem hiding this comment.
I need to verify it yet
Sorry, something went wrong.
| private final MpscIntQueue[] freelistSlots = new MpscIntQueue[FREELIST_POOL_COUNT]; | ||
| private final IntStack[] localFreelistSlots = new IntStack[FREELIST_POOL_COUNT]; | ||
|
|
||
| private static final int TARGET_RECYCLED_BYTES = 4 * 1024 * 1024; |
There was a problem hiding this comment.
The "shared" bin target 4 MB per local heap
Sorry, something went wrong.
|
|
||
| private Magazine createMagazine(int sizeClassIndex, AdaptivePoolingAllocator allocator) { | ||
| if (recycler == null) { | ||
| recycler = Magazine.AdaptiveRecycler.sharedMpsc(MAGAZINE_BUFFER_QUEUE_CAPACITY); |
There was a problem hiding this comment.
bad name, I prefer exclusiveAcquire or something similar: mpsc is way too anonymous
Sorry, something went wrong.
| static { | ||
| NEXT_IN_LINE = AtomicReferenceFieldUpdater.newUpdater(Magazine.class, Chunk.class, "nextInLine"); | ||
| } | ||
| private static int threadIndex(Thread t) { |
There was a problem hiding this comment.
we want some better shuffling than what we had before
Sorry, something went wrong.
| // In any case we will store the Chunk as the current so it will be used again for the next allocation and | ||
| // thus be "reserved" by this Magazine for exclusive usage. | ||
| curr = NEXT_IN_LINE.getAndSet(this, null); | ||
| curr = nextInLine; |
There was a problem hiding this comment.
no reason anymore to have a xchg here as it is always guarded by the stripe lock; we don't allow allocating outside the lock as it can severely escape the reason why the striped lock exist in primis
Sorry, something went wrong.
| @@ -279,7 +279,7 @@ void purgeScanShouldEvictIdleChunks(boolean threadLocal) throws Exception { | |||
| AdaptiveByteBufAllocator allocator = new AdaptiveByteBufAllocator(false, threadLocal); | |||
| long purgePolls = threadLocal ? | |||
| AdaptivePoolingAllocator.CHUNK_PURGE_POLLS_THREAD_LOCAL : | |||
There was a problem hiding this comment.
this is ugly; need fixing it!
Sorry, something went wrong.
| } | ||
|
|
||
| @SuppressWarnings("unchecked") | ||
| protected Recycler(int maxCapacity, boolean unguarded, boolean mpsc) { |
There was a problem hiding this comment.
mpsc is ugly, better sharedGet or something similar
and need tests to hit this too
Sorry, something went wrong.
|
I see that the current model on this pr would enable a minalloc style local release path which would benefit to prevent chunk churning and the artificial floor of 8M per size class cache which is applied to 4.2. |
Sorry, something went wrong.
Thanks for reaching out. I'm currently away on vacation and will take a look once I'm back. |
Sorry, something went wrong.
|
@laosijikaichele you can ignore the big amount of changes I have here, please: in the final version I plan to split into separate files the allocator key components too 🙏 In any case this grant me a very very good performance for the event loop case, and the platform one, fixing some of the shortcomings of #17166 |
Sorry, something went wrong.
|
Just for myself: the epoch based mechanism on top of new striped model was impl at 191f507
In the new notification based model, instead, chunks will be allocated to serve the incoming requests and accomodate for the live required buffers, but, once released, will be made available or, if fully released, recycled in the shared parking lot, to serve future demand. |
Sorry, something went wrong.
…s and size-class chunk recycling Squash of: - Unified StripedHeap model with cross-size-class chunk recycling - Replace ring buffer cache with mimalloc-style two-list chunk cache - Add shared-path signal detection via offerAndGetSize + tryWriteLock - Attempt tryWriteLock on every cross-thread release (mimalloc-style) - Filter tryWriteLock to transition boundaries, cleanup - Make owningCache final — set once at construction, never changes - Add local freelists for shared chunks, tryWriteLock-first release - Remove dead localFreeList null checks, simplify - Extract shared eviction logic, fix stale javadocs - Tiered purge: frequent bounded + rare unbounded sweep - Fix chunk leak on locked release path; bound post-hit scan walk - MpscIntQueue: resetAndFill must wait for already-claimed slots - Stage 1: notification queue for cross-thread segment returns - Stage 2: delete the exhausted-list scans, keep fullSweep as a detector - Bounded probe of the exhausted list before allocating a fresh chunk - Make the cache Javadoc precise about what is trustworthy and what is not - fullSweep is a third recovery route, not a violation detector - Do not fire a FreeChunk event for a chunk whose buffer was recycled - AdaptivePoolingAllocator: drop redundant fully-qualified java.util references - Magazine.allocate: extract slow path into allocateSlow - SizeClassedChunk.releaseSegment: shrink the owner-thread fast path - Drop dead isReallocation/reallocate parameter from the chunk-capacity path - Remove the write-only RESCUED_BY_FULL_SWEEP counter - Remove the exhausted-list sweep from the purge tick - Make the release-lock contract explicit in ThreadLocalSizeClassedChunkCache - Fold the single-subclass cache base into its implementation - Return the purge-tick signal instead of storing it on the magazine - Pool every recycled buffer on the thread-local magazine path - Ignore revapi additions to the MpscIntQueue interface - Tick the purge on the size-classed path only, and let the magazine sweep its own siblings - Add size classes up to 128 KiB to the adaptive allocator - Carry a chunk's free lists with its buffer through the SizeClassChunkRecycler
…ve chunk inside its cache Squash of: - Make Chunk abstract and give the unpooled fallback its own chunk class - Split Magazine into a size-class magazine and a buddy magazine - Drop buddy-shaped leftovers from the size-class allocation path - Remove native-image reflection entries for fields no longer accessed reflectively - Keep the fallback to a fresh chunk if the size-class cache ever hands out a full chunk - Release a size-class chunk that fails an allocation instead of retrying it - Let the size-class cache hold an active chunk at the head of its reusable list - Keep a size-class magazine's current chunk inside its cache - Describe the active chunk in the size-class cache documentation - Test the size-class magazine's active chunk inside its cache - Drop the size-classed chunk's magazine back-reference - Tighten the size-class cache's assertions and comments - Test the size-class magazine's active chunk by behaviour - Pin chunk allocations and used memory on a seeded allocation trace - Pin the unpooled fallback: accounted while alive, replaced on growth, freed on release - Make the unpooled fallback a one-shot BuddyChunk
…to BuddyTree Squash of: - Pin buddy chunk reuse across rounds of the same large allocations - Give each stripe its own buddy chunk cache - Move the buddy tree of BuddyChunk into its own BuddyTree - Test BuddyTree against a frozen copy of its current search - Store the largest free order in each BuddyTree node
…ueue in one place Squash of: - Keep the size-class cache's active chunk off its lists - Extract ChunkQueue from the size-class cache's two lists - Record a size-classed chunk's cache membership as its ChunkQueue - Extract PendingChunks from the size-class cache - Decide a size-classed chunk's queue in one place, SizeClassedChunkCache.refile - Fix comments left behind by the size-class queue refactor - Keep ChunkQueue and PendingChunks links on Chunk
Squash of: - Expose a BuddyTree's largest free order and whether it is wholly free - Pin buddy chunk reuse after foreign releases, the idle bound, and cross-thread content - File buddy chunks by their largest free block, without a concurrent map - Bound only the wholly free buddy chunks by CHUNK_REUSE_QUEUE - Describe the allocator as it is now, and derive chunk geometry in the buddy tests - Allocate from the buddy chunk with the largest free block
…blocks in place Squash of: - Bound the idle size-classed chunks by the retention floor, not every chunk - Revert "Bound the idle size-classed chunks by the retention floor, not every chunk" - Bound the chunk recycler by one byte budget per heap - Bound a buddy magazine's idle chunks in bytes - Let a buddy release act in place when the stripe lock is free
…ool per chunk size Squash of: - Give the size classes from 32 KiB up chunks of 8 segments - Skip this magazine's own cache when draining the heap's pending notes - Retain one chunk per size class instead of a byte-budgeted floor - Share a recycler pool between all size classes with the same chunk size - Give the size classes from 16 KiB up the chunk size of their family
…ze class table Motivation: Which size classes share a recycler pool was computed by one nested loop in the static initialiser. An earlier version of it merged only adjacent size classes with the same chunk size, and nothing noticed when a change of the chunk geometry made equal chunk sizes non-adjacent: the pools silently stopped being shared. Modification: The tables are built by distinctChunkSizes(sizeClasses) and chunkPools(sizeClasses, chunkSizes), which take the size class table as a parameter. One test checks that today's size classes share a pool exactly when their chunk sizes are equal; another checks the same on 1000 seeded random tables, where equal chunk sizes are rarely adjacent. Result: The same tables (the recycler's bytecode is unchanged), and a geometry change that breaks pool sharing fails a test.
Motivation: Four behaviours of a size-classed chunk had no test of their own: what it reports about its free segments through every way a segment comes back, the deallocation of a freed allocator's chunk by a release that cannot take the stripe lock, the segment a growing buffer leaves behind, and allocation racing releases from other threads. Modification: Add capacityQueriesFollowEveryWayASegmentComesBack, segmentReturnedExternallyAfterFreeMustStillDeallocateChunk, aBufferThatOutgrowsItsSegmentGivesItBack and segmentsReleasedByOtherThreadsAreNeverHandedOutTwice, and share the reflective freeHeap helper with the existing test that inlined it. Result: Each test fails when the code path it names is broken: the external-release deallocation skipped, an external offer duplicated, the old segment kept on growth, hasFullCapacity ignoring the external list.
…restore the LocalPool ratio comment
| Back | FazBrowse Home | New Git URL |
Motivation:
Existing performance tests shows adaptive striping model and sharing across size classes to under-perform compared to other alternatives.
In addition, the existing striped model is too eager to both allocate and throw away existing slots under contention instead lazily allocate and eventually reuse them, keeping them both stable and more cache-friendly for existing threads.
The existing shared size-class chunk cache degrade severely under heavy contention - and its algorithm is not really made to be effective under concurrent usage.
The chunk cache is not able to purge idle chunks if a specific size class end up allocating from the same magazine's current chunk, causing memory waste.
Modification:
Result:
Fixes #17028
Update 2026-09-22 (head 82bd38c, rebased onto current 4.2, history squashed to 10 commits; the previous head 004e4cc is kept locally as fine history):
User-visible property changes: io.netty.allocator.chunkPurgePollsShared and io.netty.allocator.chunkPurgeThreshold are removed; io.netty.allocator.chunkPurgePollsThreadLocal default 16 -> 4; new io.netty.allocator.buddyIdleBytes and io.netty.allocator.recycledChunkBytes.
Verification: full buffer module tests on JDK 21 (14,196 run, 0 failures); mvn -pl common,buffer -am verify -DskipTests on JDK 11.0.29 with revapi ("API checks completed without failures") and checkstyle clean; the branch is rebased onto current 4.2 (conflict-free, byte-identical replay).
Numbers, head-to-head against the mimalloc Java port on its own harness (84 cells, 3 forks, 32 threads, Ryzen 9 7950X one NUMA node at 2300 MHz, JDK 21): latency geomean 0.873 and peak RSS 0.955 (ADAPTIVE / MIMALLOC, below 1 = this branch better); this branch is ahead by more than 10% in 47 cells and behind in 12. All cells, sortable: https://franz1981.github.io/netty-allocator-h2h/results/2026-09-22-x86/index.html - raw JMH JSON, RSS logs and the reduction script: https://github.com/franz1981/netty-allocator-h2h