| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Quantifying where time goes in end-to-end agentic LLM inference — and why your CPU might be stealing 15% and more of your GPU throughput.
📝 Blog post: andyluo7.github.io
├── BLOG.md # Full blog post (markdown)
├── PLAN.md # Original experiment plan
├── RESULTS_SUMMARY.md # Raw results summary
├── scripts/
│ ├── 01_tokenizer_benchmark.py # Tokenizer CPU micro-benchmark
│ ├── 02_request_profiler.py # E2E request time decomposition
│ ├── 03_gpu_monitor.sh # GPU utilization sampling
│ ├── 04_vllm_internal_profiler.py # vLLM internal component profiler
│ ├── 05_load_generator.py # Concurrent load generator
│ ├── 06_server_decomposition.py # HBM-PC server-side decomposition
│ └── 07_lmcache_decomposition.py # LMCache server-side decomposition
└── results/
├── phase1_request_profiler.json
├── phase2_tokenizer_benchmark.json
├── phase2_cpu_component_benchmark.json
├── phase2-deepdive/
│ └── server_decomposition.json # HBM prefix cache arm
├── phase2-deepdive-lmcache/
│ └── server_decomposition.json # LMCache DRAM arm
└── phase3/
├── load_test_results.json
└── cpu_flame_worker_tp0.svg
This analysis accompanies our LMCache multi-turn agentic benchmark on MI300X, which compared KV-cache strategies (no cache, HBM prefix cache, LMCache CPU DRAM) using the same hardware and workload traces.
MIT
| Back | FazBrowse Home | New Git URL |