FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

exit: what must survive a failed teardown is saved before it, and the crash handler outlives it by aurelienpierre · Pull Request #1496 · aurelienpierreeng/ansel · GitHub

Repository navigation

exit: what must survive a failed teardown is saved before it, and the crash handler outlives it - #1496

Merged
Jiyone merged 3 commits into
masterfrom
exit-persists-before-teardown
Oct 26, 2026
Merged

Jiyone merged 3 commits into
masterfrom
exit-persists-before-teardown

Conversation

Copy link
Copy Markdown
Member

Third PR of the thumbnail/cache cluster (#1472–#1479). Closes #1476.

Three symptoms in the Windows reports behind this cluster — no thumbnail cache on disk, preferences
not saved, "it closes on a crash" — are one event, and the ordering inside dt_cleanup() is what
couples them.

Nothing was persisted until the very end

dt_conf_set_*() writes only the in-memory table. The settings file is written by dt_conf_save(),
whose sole caller on the normal path was dt_conf_cleanup(), at the tail of dt_cleanup(). And a
thumbnail reaches the disk cache only when its cache entry is released, which for everything still
resident happened inside dt_mipmap_cache_cleanup() — a hundred lines earlier, but still well past
the risky part.

So a session's preferences and its whole thumbnail cache both depended on reaching the end of a
long serial teardown, behind a window hidden at the start of it — a teardown that includes
dt_ctl_switch_mode_to(""), i.e. the darkroom leave() path this tree has documented hazards in.

before after
dt_sentry_shutdown() first — crash handler gone for the whole teardown moved to just before dt_conf_cleanup()
preferences saved only at the tail dt_conf_flush() before anything that can fail (and again at the tail)
thumbnails written only at the tail dt_mipmap_cache_flush_to_disk() before anything that can fail (and again at the tail)

The later calls stay on purpose: they catch what the teardown itself writes (window geometry, panel
sizes, sentry's session counters) and any thumbnail another thread still held.

Details worth reviewing

  • dt_mipmap_cache_flush_to_disk() is the existing eviction walk at fill ratio 0, not a second
    loop. dt_cache_gc() already skips entries it cannot trywrlock, honours _lock_demoting, and
    runs the cleanup callback that does the writing. A bespoke "flush but keep" pass would need its own
    answer to each of those, plus a way not to write every entry twice at teardown.
  • It needed dt_cache_gc_locked(), because dt_cache_gc() is documented "will never lock" and
    means it: both callers inside cache.c already hold cache->lock, so a caller outside the file
    had none. Calling it unlocked would have been a data race on cache->lru.
  • dt_conf_save() now takes cf->mutex. It never did — survivable only while its one normal
    caller ran with every other thread joined. The table is read key by key there and written under the
    mutex by every dt_conf_set_*(), so an unlocked walk concurrent with a write reads a GHashTable
    mid-rehash. The pre-existing call in opencl.c:1011 had the same exposure, so this is a fix in
    its own right, not just enabling work.
  • dt_conf_flush() is dt_conf_save() for the singleton. Every other public getter and setter
    there already takes no dt_conf_t *; save and cleanup were the exceptions, which forced a caller
    that merely wants the settings on disk to name darktable.conf — and src/gui/preferences.c, the
    most obvious such caller, deliberately does not include darktable.h.
  • The preferences dialog now saves when it closes. This is the direct answer to "my settings are
    not saved": until this commit a preference set at the start of a session lived only in memory for
    the whole of it, and any abnormal exit lost it.

What was measured

  • Build clean, 0 warnings, in build and build-nofeatures. All six local gates pass, cycles 0.
  • A CLI run with an isolated --configdir/--cachedir reports the flush once, in the new position,
    and writes a 5570-byte anselrc.
  • Checked against the defect, not only against the fix. Injecting abort() into the teardown
    immediately after the two new calls: the process dies with SIGABRT (exit 134) and anselrc is
    still on disk (5564 bytes). On master the only dt_conf_save() on that path runs after the
    injection point, so it leaves none — grep confirms dt_conf_save had exactly two callers,
    dt_conf_cleanup() and opencl.c.

Not measured: the thumbnail half needs a populated mip_thumbs at exit, which only the GUI
lighttable produces — an ansel-cli export uses DT_MIPMAP_FULL only, so the flush correctly
reports 0.00 MB banked there. The mechanism is the same eviction path PR #1495 measured directly.

What this does not do

The disk cache still depends on eviction to be written; this only stops it depending on surviving the
teardown. Writing a thumbnail when it is generated (write-through) would remove the exit flush
entirely and is the better end state, but it changes the cache's write policy and overlaps #1446's
"skip the eviction write when the file already holds this buffer" and #1086's plan to record what
each thumbnail was rendered from — so it wants your call rather than my guess.

Depends on nothing; branched off origin/master like the other two, so the three can land in any
order.

🤖 Generated with Claude Code

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

… crash handler outlives it

Three symptoms in the Windows reports behind #1476 -- no thumbnail cache on disk, preferences not saved, "it closes on a crash" -- are one event, and the ordering inside dt_cleanup() is what couples them.

Neither thing was persisted until the very end. dt_conf_set_*() writes only the in-memory table; the settings file is written by dt_conf_save(), whose sole caller on the normal path was dt_conf_cleanup(), at the tail of dt_cleanup(). And a thumbnail reaches the disk cache only when its cache entry is released, which for everything still resident happened inside dt_mipmap_cache_cleanup(), a hundred lines earlier but still well past the risky part. So a session's preferences AND its whole thumbnail cache both depended on reaching the end of a long serial teardown, behind a window hidden at the start of it -- a teardown that includes dt_ctl_switch_mode_to(""), i.e. the darkroom leave() path this tree has documented hazards in.

Both are now banked immediately after the last point that can ask the user anything, before any of that runs. The later calls stay: they catch what the teardown itself writes (window geometry, panel sizes, sentry's session counters) and any thumbnail another thread still held.

dt_mipmap_cache_flush_to_disk() is the new entry point, and it is the existing eviction walk at a fill ratio of 0 rather than a second loop: the walk already skips entries it cannot trywrlock, honours _lock_demoting, and runs the cleanup callback that does the writing. The walk never locked, because both its callers inside cache.c already hold cache->lock, and it had no caller outside the file. It is now _cache_gc_locked(), private, and the public dt_cache_gc() takes cache->lock around it and returns the cost reclaimed: a public function that touches the LRU unlocked was a trap for the first outside caller, and the _locked suffix means "the caller holds the lock" elsewhere in the tree (conf.c's _dt_conf_get_var_locked()).

dt_sentry_shutdown() moves from the FIRST statement of dt_cleanup() to just before dt_conf_cleanup(). sentry_close() uninstalls the crash handler, so running it first left the entire teardown unreported: an exit crash produced no event at all, which is why #1476 had no crash data to work from. It has to stay ahead of dt_conf_cleanup(), because it writes the clean-session counter and the session duration through dt_conf_set_*() and that save is what puts them on disk.

Two supporting changes the above made necessary:

- The save now takes cf->mutex. It never did, which was survivable only while its one normal caller ran with every other thread joined: the table is read key by key there and written under the mutex by every dt_conf_set_*(), so an unlocked walk concurrent with a write reads a GHashTable mid-rehash. The pre-existing call in opencl.c had the same exposure.
- dt_conf_flush() saves the singleton, and is the only public way to save. Every other public getter and setter here already takes no dt_conf_t *. opencl.c, the last caller that named darktable.conf to save, calls it too, which closes conf's last cross-module reference (doc/globals-migration.md); the save itself is private to conf.c.

And the preferences dialog now saves when it closes, which is the direct answer to "my settings are not saved": until this commit, a preference set at the start of a session lived only in memory for the whole of it.

Measured: build clean, 0 warnings. A CLI run with an isolated --configdir/--cachedir reports the flush once, in the new position, and writes a 5570-byte anselrc. Injecting abort() into the teardown immediately after the two new calls still leaves anselrc on disk, where on master the only dt_conf_save() on that path runs after the injection point and leaves none.

The thumbnail half, measured in the GUI with a shim that pages through the lighttable and then quits (Xephyr 1920x1080, default 1000 MB budget): 400 thumbnails, 124 MB, flushed in 259 ms; 400 at 494 MB in 817 ms, all written by the flush and none left for the end; 60 NEF quit while rendering, 8 of 12 written by the flush and the 4 still rendering written by dt_mipmap_cache_cleanup(). No thread waited on the cache lock during a flush. doc/shutdown.md has the table; doc/sentry.md places dt_sentry_shutdown() at its new position.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Jiyone force-pushed the exit-persists-before-teardown branch 2 times, most recently from 5fa236a to 89af5cc Compare October 26, 2026 20:56

Jiyone commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Rebased on master (f6f82cbf45) and reworked after a review against AGENTS.md. The branch is now three commits; the PR description above predates them, so this is what changed.

1a1a9a279b — the original commit, amended

  • dt_cache_gc() takes the lock. It is public, takes cache->lock, and returns the cost reclaimed. The unlocked walk is private, _cache_gc_locked(). In this tree the _locked suffix means "the caller holds the lock" (_dt_conf_get_var_locked() in conf.c), and dt_cache_gc_locked() did the opposite. dt_cache_gc() had no caller outside cache.c, so changing what it does breaks nothing. dt_cache_gc_locked() is gone.
  • One public way to save the settings. The save is private to conf.c (_conf_save()), and dt_conf_flush() is the only public way in, without its NULL check. common/opencl.c:1011 calls it too, which removes the last place outside conf.c that named darktable.conf (doc/globals-migration.md updated).
  • Comments. Shortened; issue numbers and history moved out of the code. They stay in the commit message. In darktable.c, only the comments changed.
  • Docs.
    • doc/sentry.md and doc/shutdown.md still put dt_sentry_shutdown() at the start of dt_cleanup(); fixed.
    • doc/shutdown.md gains What reaches the disk before the teardown, with the measurements below.

776f1b4c81 — the settings file is replaced atomically

  • The problem. The save opened anselrc with "wb", which empties it before writing. Under gdb, with ansel-cli killed inside the save after the open, anselrc went from 12825 bytes to 0. This PR saves more often, including once just before the part of the teardown that can fail, so a truncated file is more likely than before.
  • The fix. The table is serialised under cf->mutex and written with g_file_set_contents(), which writes a temporary file and renames it over anselrc. Killed at the rename(), anselrc stays whole.
  • The cost. GLib's fsync() holds the mutex for 4–7 ms per save, against 0.2–0.7 ms before (10 runs, ext4 on a SATA SSD). The write stays under the mutex so that two saves land in the order they read the table.
  • A failed save is reported. dt_conf_flush() returns 1, with the reason on stderr. The preferences dialog then raises a Settings not saved alert, with the configuration folder as its item. The other callers (startup, teardown) only print on stderr.
  • New test: tests/unittests/test_conf_save.c.
    • It holds a handle on the old file across a flush, and that handle must still read the old content. Run against the old "wb" save, this test fails.
    • It also checks that the value set is in the file, that no temporary file is left, and that a write that cannot happen returns 1.

00f5fc2070 — what the flush log reports

The log said "MB banked", but the number is what left RAM. With 400 thumbnails read back from the disk cache, it printed 494.4 MB banked and wrote nothing: an entry read from its file is dropped without being written. It now says "evicted from RAM".

The thumbnail half, measured

These are GUI runs. An LD_PRELOAD shim pages through the lighttable and then quits, and temporary probes time the flush and count what the cleanup callback did with each entry. Setup: Xephyr at 1920×1080, default 1000 MB thumbnail budget.

At the quit In RAM Flush Written by the flush Left for the end
400 JPEG, 4 per row, empty disk cache 400, 124 MB 259 ms 400 0
400 JPEG, 2 per row, empty disk cache 400, 494 MB 817 ms 400 0
the same, quit 0.2 s after the last page 400, 494 MB 776 ms 400 0
the same, thumbnails read from the disk cache 400, 494 MB 64 ms 0 0
60 NEF with a history, quit while rendering 12, 14.8 MB 30 ms 8 4
  • Lock time. The flush holds the thumbnail cache lock throughout, about 1.65 ms per MB written.
  • No extra work. The work moves earlier rather than being added: in the JPEG rows, the end-of-teardown cleanup wrote nothing.
  • No contention. No thread waited more than 2 ms on that lock during a flush. After the quit, workers take no new job.
  • Thumbnails still rendering. The 4 thumbnails still rendering at the quit were skipped by the flush and written by dt_mipmap_cache_cleanup(), as intended.

Checks

  • Built with clang-19, RelWithDebInfo and Debug: no new warnings.
  • All eight gates of ci.yml pass, and include_graph reports cycles 0.
  • ctest passes 31/31 in Debug under Xephyr.
  • The Settings not saved alert, on screen. Ansel ran on a scratch configdir, and anselrc was replaced by a directory while it was running, so the rename fails. Closing the preferences dialog raised the alert: title, message, the configuration folder as its item. Closing it twice gave one window, with the folder listed once; GLib removed the temporary file both times. With the directory removed, the next close saved anselrc (13205 bytes) with no alert, and Ansel exited with status 0.

Jiyone self-assigned this Oct 26, 2026
Jiyone and others added 2 commits October 26, 2026 22:29
…a failed save is reported

The save opened anselrc with "wb", which empties it before the first line is written: a process that died during a save left a 0-byte file and lost every setting. Measured under gdb with ansel-cli, killed inside the save after the open: anselrc went from 12825 bytes to 0. This branch saves more often, and once right before the part of the teardown that can fail, so the window matters more than it did.

The table is now serialised into a buffer under cf->mutex and written with g_file_set_contents(), which writes a temporary file and renames it over the old one. Killed at the rename, the same run leaves anselrc intact at 12825 bytes, plus an orphan anselrc.XXXXXX. The write stays under the mutex so that two saves land in the order they read the table; with the fsync GLib does before the rename, the mutex is held 4-7 ms per save on ext4 (10 runs), against 0.2-0.7 ms before.

A failed write was silent. It is printed on stderr with its reason, and dt_conf_flush() returns 1. The preferences dialog, the one caller with a user in front of it who has just changed settings, raises a "Settings not saved" alert with the configuration folder as its item. Checked under gdb by pointing the write at a missing directory: the stderr line, a return of 1, and a normal exit.

tests/unittests/test_conf_save.c pins the replacement: a handle opened on the old file before a flush must still read the old content after it. Run against the save put back to fopen("wb"), that test fails and the two others in the file (what is written, what a failure returns) pass, as they should.

doc/preferences.md describes when the settings reach the disk, the atomic replacement, its cost, and how a failure is reported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dt_mipmap_cache_flush_to_disk() printed the cost the eviction walk reclaimed as "MB banked", but the cleanup callback drops an entry read from its file without writing it. Measured on the branch with probes in the callback, 400 thumbnails read back from the disk cache and flushed at exit: the log said 494.4 MB banked, and 0 were written. The message now says what the number is; each write is already logged by the callback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Jiyone force-pushed the exit-persists-before-teardown branch from 89af5cc to 00f5fc2 Compare October 26, 2026 21:29

sonarqubecloud Bot commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

Jiyone merged commit b0f1052 into master Oct 26, 2026
22 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

Bug: everything that persists state at exit runs after crash reporting is off, behind a long serial teardown, with the window already hidden

2 participants


Back | FazBrowse Home | New Git URL