| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
|
I noticed several drifts between this guide and the metrics definition (like the error code etc) and I think it will become more programatic in the future because it will be very hard to keep the guide the metrics definition synced. I think we should make the guide very thin. The durable structure is: guide owns the decision tree and interpretation, registry owns definitions, and the guide links rather than paraphrases. Every value taxonomy or default it copies inline is future debt. |
Sorry, something went wrong.
The metric registry says what each instrument measures, and the observability guide says how telemetry is collected, but neither takes an operator from a symptom to a cause. A user reporting a slow request or a 503 had no path through the data. These three guides each start at a symptom and end at a component. Every step gives a query, says how to read the result, and names the next step or the guide to switch to. The shared index holds what all three need: the two spellings of the instrument names and which ingest path produces each, the four reasons a query returns nothing, and the pointer to the subsystems that emit no metrics at all. Every query was run against a live cluster, and the readings drove the text: the resume label is only meaningful next to the outcome, a fleet gauge that every replica reports needs max rather than sum, and a phase histogram has to exclude failures before its quantile means anything.
The guides copied label taxonomies and flag defaults inline. A copy goes out of date without a signal, and one already had: the guide said that ate.imagecache.requests reports _OTHER for each status that is not 401, 403 or 429, but the registry and internal/imagecache/metrics.go also pass 404, 500, 502, 503 and 504 through as literal values. Give the guides one job. They own the decision tree: which query to run, how to read the result, and where to go next. The registry owns the definitions. Drop the value tables for ate.snapshot.kind, ate.failure.reason, ate.scheduling.constraint and the parking outcome, and link to the registry group instead. Drop the --parked-request-max default and link to request-parking.md, which owns the flag. Drop the histogram bucket ceilings and the two sample measurements, which the surrounding argument does not need. The phase, reason and outcome tables that remain hold routing and not definitions. Fix two drifts on the way: three guides said "three reasons" for an empty query where the README lists four, and capacity-is-full.md said "two states" above a three-row table.
Agreed, and updated to remove some metrics details from the guide. |
Sorry, something went wrong.
Review found decision-tree arrows that send an operator to the wrong component, and queries that contradict the paragraph beside them. Routing. The router maps a shed request and an exhausted park budget onto the same no_capacity outcome, so that outcome can no longer send the reader straight to the capacity guide; step 3 now splits the two on parking.rejected, which is the only signal that separates them. The timeout row claimed an ended park budget, which never classifies that way: an ended budget surfaces the condition that blocked the resume. The capacity guide sent the common path to "make spec.replicas larger" without passing the step that asks whether the workers are doing any work. Hidden capacity has two causes the guides did not name: a paused actor is pinned to the node holding its snapshot and reports required_nodes rather than selector, and the idle worker count includes draining workers that the scheduler will never pick. Queries. The CPU rate summed across the stats source that the text below it says never to add together. The snapshot-size quantile dropped the file name that the next line says to compare on. The step 1 restore quantile counted failed phases, which hold their timers, while step 2 excludes them. The desired-minus-ready subtraction used a bare on() join that fails once a second replica reports the same pool. A paused restore was also documented as never downloading, which is untrue for the data_on_golden scope: the golden files still come from object storage.
| Back | FazBrowse Home | New Git URL |
Fixes #1294
It's converted from this public doc. Please feel free to drop comments in the doc and I can update this PR.