| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
# Conflicts: # core/src/main/java/org/apache/datafusion/SessionContextBuilder.java # proto/session_options.proto
| Back | FazBrowse Home | New Git URL |
Which issue does this PR close?
Rationale for this change
DataFusion's RuntimeEnv accepts a CacheManagerConfig with three independent caches: the file-embedded metadata cache (parquet footers / page metadata), the list-files cache (object-store LIST results), and the file-statistics cache (per-file row counts and column stats used by the planner). The Rust API is RuntimeEnvBuilder::with_cache_manager(CacheManagerConfig). The Java binding has no surface for any of it — every SessionContext ends up with the no-op upstream defaults today, so a parquet workload reading the same footer thousands of times across queries goes back to the object store every single time, and statistics-driven planners can't persist their stats across queries.
This PR adds a typed cacheManager(CacheManagerOptions) setter on SessionContextBuilder that exposes the three caches independently:
Semantics, all matching upstream (datafusion).
Three independent toggles, all matching upstream:
What changes are included in this PR?
Are these changes tested?
Yes, 18 new tests cross CacheManagerOptionsTest and SessionContextCacheManagerTest.
Are there any user-facing changes?
Yes, but additive only — no breaking changes:
No behavior change for callers that do not invoke the new setter — the cache_manager field is absent on the wire and the native side leaves upstream's RuntimeEnvBuilder defaults in place.