The streaming counterpart of ktxTexture2_TranscodeBasis agreed in issue 1224:
an object created for a Basis-compressed source (typically a
metadata-only texture constructed from the serialized file prefix) and a
chosen target, that will process mip levels one at a time from
caller-provided payload bytes into caller-provided buffers, with layout
queries describing the processed output. This commit lands the public
surface, creation/validation and the layout queries; ProcessLevel
validates its arguments per the final contract and returns
KTX_UNSUPPORTED_FEATURE until the codec paths land in the next commit of
this series.
ktxLevelProcessor_CreateBasis validates the source exactly as
TranscodeBasis does (transcodable color model, SGD presence for
BasisLZ/6x6-intermediate; unknown color models fail the same gate),
rejects video sources with KTX_UNSUPPORTED_FEATURE (agreed follow-up),
and builds a private KTX_TEXTURE_CREATE_NO_STORAGE prototype — the same
prototype technique TranscodeBasis uses, minus the full-mip-chain
allocation. The queries delegate to it: GetLevelSize/GetImageSize are
the existing size calculations, GetImageOffset is rebased to be relative
to the level's destination buffer, GetOutputVkFormat reports the
resolved format.
So the processor validates sources, resolves targets and initializes
identically to TranscodeBasis, three pieces of it are factored out and
shared via the new internal basis_transcode.h: the source gate
(transcodable color model, presence of the supercompression global data
the scheme requires) as ktxTexture2_validateBasisSource, the target
resolution logic (alpha-content detection, automatic-selection mapping,
VkFormat resolution with the source's transfer function, PVRTC1
power-of-two validation and the transcoder-availability check) as
ktxTexture2_resolveBasisTargetFormat, and the guarded process-wide
basisu_transcoder_init() call as ktxInitBasisTranscoder. Creating a
processor performs that initialization once, never per processor. The
colorModel to basis_tex_format mapping both users need is a file-local
helper. No behavior change to TranscodeBasis.
CreateBasis requires the source to retain its serialized-source state:
a texture whose image data has been loaded, or one created with
ktxTexture2_Create, is rejected with KTX_INVALID_OPERATION, because
ProcessLevel consumes the serialized level payload that only such a
source can describe (loading a Zstd/Zlib source even rewrites its
level index to the inflated sizes). ProcessLevel re-checks that state
through ktxTexture2_GetLevelFileInfo, propagating its result, and
validates srcSize against the byteLength it reports. The three layout
queries validate their arguments themselves so an out-of-range level,
layer or faceSlice is KTX_INVALID_VALUE from all of them.
The processor struct lives in a private header (level_processor.h, as
texture2.h does for ktxTexture2_private) and level_processor.cpp is
added to the libktx Doxygen inputs, so the reference documents the
ktxLevelProcessor class with its functions and nothing internal.
Tests verify the resolved format and every layout query against the
actual output of ktxTexture2_TranscodeBasis on the same ETC1S 2D-array
source, plus creation validation and the ProcessLevel
validate-then-report contract.
First part of the per-level streaming API discussed in #1224. This draft carries the truncated-stream regression test, the source-side query and the public surface of the level processor, so the header can be reviewed against what was agreed in the thread before the codec work lands. ktxLevelProcessor_ProcessLevel is intentionally a stub here: argument validation per the final contract, then KTX_UNSUPPORTED_FEATURE.
To answer the question in #1224 first: yes, the truncated-stream regression test was successful. A ktxTexture2 constructs from exactly the serialized metadata prefix without KTX_TEXTURE_CREATE_LOAD_IMAGE_DATA_BIT on unmodified main, for BasisLZ, Zstd-supercompressed and unsupercompressed sources, and the constructor never reads past that prefix. No prerequisite fix to the constructor was needed; the test is the first commit.
What is in the three commits
Metadata-only construction tests (test-only). Construction from the prefix through the end of SGD/KVD, as applicable, checked against an instrumented bounded custom ktxStream on all platforms and, on POSIX, a PROT_NONE guard page immediately after the prefix. Also covers the clean failure modes: LOAD bit on a prefix, late ktxTexture2_LoadImageData on a shell, metadata cut one byte into the last section. Files are generated in-process; no new binary resources.
ktxTexture2_GetLevelFileInfo — where a level's stored payload lives in the serialized source: absolute byteOffset, byteLength, and the Level Index uncompressedByteLength. A thin public exposure of the existing rebased level index plus _firstLevelFileOffset. Per the thread, ktxTexture2_levelFileOffset() itself is left private and unchanged; the public query wraps its logic and returns KTX_INVALID_OPERATION once the texture no longer has serialized-source state (created with ktxTexture2_Create, or after image data has been loaded). If you would rather expose ktxTexture2_levelFileOffset() directly, in addition or instead, that is a small change.
ktxLevelProcessor skeleton — the header as drafted in Feature request: transcode individual mip levels from a partially-downloaded KTX2 file (progressive texture streaming) #1224: opaque type, ktxLevelProcessor_CreateBasis, GetOutputVkFormat, the three layout queries (GetLevelSize / GetImageSize / GetImageOffset, level-relative), ProcessLevel, Destroy. CreateBasis validates the source the same way ktxTexture2_TranscodeBasis does, rejects video with KTX_UNSUPPORTED_FEATURE (follow-up, as agreed), builds the private KTX_TEXTURE_CREATE_NO_STORAGE prototype the queries delegate to, and performs the transcoder's process-wide initialization through the same guarded mechanism TranscodeBasis uses — once, never per processor. To make both paths validate sources, resolve targets and initialize identically, three pieces of TranscodeBasis are factored into internal helpers declared in a new basis_transcode.h: the source gate (transcodable color model, SGD presence), the target resolution (alpha content, automatic-selection mapping, VkFormat with transfer function, PVRTC1 pow2 check, transcoder-availability check) and the guarded basisu_transcoder_init() call. Mechanical; no behavior change to TranscodeBasis. CreateBasis requires the source to retain its serialized-source state and rejects a texture whose image data has been loaded (or one created with ktxTexture2_Create) with KTX_INVALID_OPERATION: ProcessLevel consumes the serialized level payload that only such a source can describe, and loading a Zstd/Zlib source rewrites its level index to the inflated sizes. ProcessLevel re-checks that state through GetLevelFileInfo and validates srcSize against the byteLength it reports; the three layout queries report out-of-range arguments uniformly as KTX_INVALID_VALUE. The processor struct lives in a private header, as texture2.h does for ktxTexture2_private, and level_processor.cpp is added to the libktx Doxygen inputs, so the generated reference documents the ktxLevelProcessor class with its functions and nothing internal.
Settled in #1224 and reflected here
Still open — your call
Tests
32 new gtest cases in transcodetests, all generated in-process: the construction/over-read/truncation matrix × 3 source variants; GetLevelFileInfo values byte-for-byte against the serialized index, availability before and invalidation after load, invalid arguments, no-serialized-source; processor creation validation and every layout query cross-checked against the actual output of ktxTexture2_TranscodeBasis on an ETC1S 2D-array source. Full library/tools/tests build clean on Linux; libktx.doc builds without new warnings.
Out of scope here (per the thread)
Batch level ranges, cancellation, caching, single-level GL/Vulkan upload, format-property queries, JS/Java/Python bindings (follow-ups), video (follow-up before release), XUASTC sources (rejected by the same source-family gate as TranscodeBasis).