FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

subscription_thread_dispatcher: guard against duplicate room events by stephen-derosa · Pull Request #195 · livekit/client-sdk-cpp · GitHub

subscription_thread_dispatcher: guard against duplicate room events - #195

Open
stephen-derosa wants to merge 4 commits into
livekit:mainfrom
stephen-derosa:fix/subscription_thread_dispatcher_cleanup
Open

subscription_thread_dispatcher: guard against duplicate room events#195
stephen-derosa wants to merge 4 commits into
livekit:mainfrom
stephen-derosa:fix/subscription_thread_dispatcher_cleanup

Conversation

stephen-derosa commented Jul 6, 2026
edited
Loading

Copy link
Copy Markdown
Collaborator

Overview

Fixes silent callback-replacement failures and duplicate-subscription churn in SubscriptionThreadDispatcher. No public API changes — setOn*FrameCallback now does what it always documented.

The bug

Reader threads capture the frame callback by value. setOn*FrameCallback documented itself as "register or replace", but when a reader was already running, overwriting the registration map only replaced the dispatcher's copy — the running thread kept invoking the old callback indefinitely. Replacement was a silent no-op: no error, no warning, frames just kept going to the callback the caller thought they had discarded.

Changes

  • Replace in place. setOn* extracts the stale reader under the lock, installs the new callback, and joins the old thread outside it; Room then restarts a reader bound to the new callback. The extract also clears the dedup guard's precondition, so no change to that logic was needed.
  • Duplicate subscription events. Readers record the subscribed track SID and skip redundant restarts, so a repeated track_subscribed no longer rebuilds a healthy reader.
  • Data track teardown. In-flight RemoteDataTrack::subscribe() calls are marked cancelled, so unpublish/republish cannot leave a stale reader or duplicate FFI subscription behind.

Behavioral notes (documented, not enforced)

  • setOn*/clearOn* block until any in-flight callback invocation returns. On return, the previous callback has finished and its copy is destroyed — but a callback that never returns blocks registration indefinitely.
  • Re-entrant registration is unsupported. Calling setOn*/clearOn* from inside a frame callback for the same key would self-join. Media readers are detached with a logged error (safe — those lambdas capture no this); data readers cannot be detached, so the removal is refused and the reader is reaped at teardown.

Public API

No signature changes. include/livekit/room.h changes are documentation only.

Test-only: RemoteDataTrack::testFfiHandleId() removed in favor of RemoteDataTrackTestAccess; new RoomTestAccess exposes reader counts to integration tests.

Testing

  • Unit (67 tests): replacement semantics across audio/video and both directions of the shared video slot; that the stored callback and its stream options actually swap; isolation across keys; registration surviving unsubscribe; SID dedup; data reader cancellation.
  • Integration (12 tests, new file): delivery switching on replacement, slow-callback blocking, 20 sequential replacements and 4-thread concurrent churn asserting exactly one live reader, deferred start, unpublish/republish, and all three re-entrancy paths. Blocking calls are watchdogged so regressions fail rather than hang CI.
  • Mutation-tested. Reintroducing the original defect fails 3 tests; skipping the callback install fails 4. The latter class of bug was caught by zero tests before this PR.

Integration tests compile and link but were not run locally (no LIVEKIT_TOKEN_A/LIVEKIT_TOKEN_B); they need CI or a live server.

stephen-derosa self-assigned this Jul 6, 2026
Copilot AI review requested due to automatic review settings July 6, 2026 21:39
stephen-derosa marked this pull request as draft July 6, 2026 21:39

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Pull request overview

This PR refines SubscriptionThreadDispatcher’s reader-thread lifecycle handling to avoid redundant restarts and to ensure data reader threads reliably tear down (especially when a data-track subscription is still in flight), aligning with the SDK’s threading model by preventing teardown hangs.

Changes:

  • Add cancellation signaling for active data readers and ensure cancellation is set before closing/joining during unpublish and teardown.
  • Introduce a self-cleanup path (eraseDataReaderIfCurrent) so data reader threads can safely remove stale slots when they exit.
  • Skip redundant audio/video/data reader restarts when the same track SID is already active.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
src/tests/unit/test_subscription_thread_dispatcher.cpp Adds unit tests covering data reader cancellation semantics and self-erase behavior.
src/subscription_thread_dispatcher.cpp Implements cancellation-before-close, self-erase cleanup, and redundant reader-start suppression.
include/livekit/subscription_thread_dispatcher.h Adds per-reader state (track_sid, cancelled) and declares the new cleanup helper.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

This comment was marked as resolved.

stephen-derosa changed the title rm unneeded function, better cleanup subscription_thread_dispatcher: guard against duplicate room events Jul 7, 2026
stephen-derosa force-pushed the fix/subscription_thread_dispatcher_cleanup branch from 2f313fe to f1ff6e6 Compare July 7, 2026 18:17
stephen-derosa requested a review from Copilot July 7, 2026 22:11
stephen-derosa marked this pull request as ready for review July 7, 2026 22:11

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

Comment thread include/livekit/remote_data_track.h Outdated
Comment thread include/livekit/room.h Outdated
/// @return @c true if the callback was registered; @c false if a reader is
/// already active for the key (call @ref clearOnAudioFrameCallback
/// first) or the room has no dispatcher.
[[nodiscard]] bool trySetOnAudioFrameCallback(const std::string& participant_identity, const std::string& track_name,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

From what I can tell, these new "try" methods have the same inputs as the deprecated prior ones, just a new return type (and name).

This is actually a situation where I think maintaining the API and using exceptions is cleaner than deprecation/new API just to get the bool return type.

Thoughts on this? Could we not move the new implementations from room.cpp into the older versions and throw instead of returning false?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

isnt that still changing the public API though? maybe im wrong but i would think adding a throw to a function that previously didnt would require a major bump?

alan-george-lk Jul 8, 2026
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Fair point, technically speaking those methods were never marked noexcept and anything within them could throw (STL container exceptions, runtime exceptions from FFI stuff, etc.) so I think it's the least of the evils?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

hm yeah im a little torn here. I dont want to just start throwing underneath applications, especially when our mantra has somewhat been team #nothrow 😅

but the point of other things can throw inside the function are valid 🤔

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Also torn, I don't love my suggestion, to me keeping the API clean/avoiding deprecations is the least of the evils. Maybe @xianshijing-lk can chime in

Comment thread include/livekit/room.h Outdated
/// @return @c true if the callback was registered; @c false if a reader is
/// already active for the key (call @ref clearOnVideoFrameCallback
/// first) or the room has no dispatcher.
[[nodiscard]] bool trySetOnVideoFrameCallback(const std::string& participant_identity, const std::string& track_name,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I guess I didn't understand why these callback events need to be in the room ?
couldn't they be in the remote participants?

And does this frame callbacks work on both local / remote participants ?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

the room has access to the room events which are needed to set callbacks to incoming track messages -- the remote participant doesnt really.

This callback only works for local participants, the intention is to provide a nice ergonomic way for the user to set callbacks for incoming messages.

Comment thread src/room.cpp
} else {
// The track is not subscribed yet. The callback is registered; the reader
// starts when the track is subscribed (see kTrackSubscribed in onEvent).
LK_LOG_DEBUG(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

should you still return true here ?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

good question, yes since the user has successfully set the callback. It doesnt necessarily mean that a track has been published or that they will get messages on said track. If we were to only return true to a callback being set after a track has been published, then early joining participants that want incoming messages will effectively have to periodically set callbacks until success, or listen to room events -- both of which defeat the purpose of the ergonomic functionality of these setOn*Callback functions.

Comment thread src/room.cpp
// If we've already subscribed to the track, handle it immediately
auto track = findSubscribedRemoteTrack(participant_identity, track_name);
if (track) {
subscription_thread_dispatcher_->handleTrackSubscribed(participant_identity, track_name, track);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think I forgot some technical details here, is this subscription_thread_dispatcher_ a manager that manages all the subscription thread ?
and does one callback has one corresponding subscription thread ? or one thread that handles all the callbacks ?

stephen-derosa Jul 9, 2026
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think I forgot some technical details here, is this subscription_thread_dispatcher_ a manager that manages all the subscription thread ?

it manages subscribing to tracks and providing incoming frames to the users callbacks

and does one callback has one corresponding subscription thread ? or one thread that handles all the callbacks ?

Each subscription gets its own thread

This comment was marked as resolved.

stephen-derosa force-pushed the fix/subscription_thread_dispatcher_cleanup branch from 2179d4c to fbdcbd4 Compare July 9, 2026 16:21

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

@alan-george-lk
In this work, i realized subscription_thread_manager is marked public:

I missed this in the review for the LIVEKIT_API PR. any thoughts on making this private?

I ran a query on this and it doesnt look like:

  1. this file is actually included in any of the other public includes
  2. the SubscriptionThreadDispatcher doesnt really work without the room wrapper
  3. its only used by the room

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

discussed in the standup need to confirm that:

  • cpp examples build with this private
  • it is in fact not included/usable outside the room.cpp/hpp

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

consider moving to src/ so that this isn't reachable

Copy link
Copy Markdown
Collaborator Author

ensure that this also works for local publishing of video/audio

stephen-derosa force-pushed the fix/subscription_thread_dispatcher_cleanup branch from 4f18b6f to 2aa6d4c Compare July 17, 2026 19:56
stephen-derosa requested a review from a team as a code owner August 28, 2026 20:46
…plicate room events test

SubscriptionThreadDispatcher: proper replacing of audio/video callbacks. Deprecate a setOn*Callback(), replace with trySetOn*Callback()

fix thread detaching
stephen-derosa force-pushed the fix/subscription_thread_dispatcher_cleanup branch from 4b75ba8 to 0dbdc64 Compare August 28, 2026 20:48

devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Devin Review found 1 new potential issue.

4 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

Comment on lines +604 to +616
if (it->second && isSelfThread(it->second->thread_id)) {
// The caller IS this reader, so it reached us from inside its own data frame
// callback. Joining would be a self-join, and unlike media readers a data
// reader cannot be detached: it re-enters the dispatcher after the callback
// returns. Leave the slot in place -- the reader exits on its own once its
// stream closes, and stopAll() reaps it.
LK_LOG_ERROR(
"Data reader for callback id={} tried to tear itself down from inside its "
"own data frame callback; leaving the reader in place. Removing a data "
"callback from within that callback is not supported",
id);
return {};
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

🟡 Removing a data callback from inside itself never stops delivery

When the removal runs from inside the data callback it targets, the self-join guard (extractDataReaderThreadLocked) returns without cancelling the reader or closing its stream, so the reader keeps reading and keeps invoking the removed callback until the track is unpublished or the room is torn down. The unpublish path handles the same case by closing the stream.

Prompt for agents
In extractDataReaderThreadLocked (src/subscription_thread_dispatcher.cpp), the self-thread branch returns early to avoid a self-join, but it does not stop the reader: it neither sets cancelled nor closes the reader's stream. As a result a removeOnDataFrameCallback call made from inside the target data callback leaves the reader looping on stream->read() and invoking its own captured copy of the callback indefinitely. The sibling case in handleDataTrackUnpublished handles this correctly by setting reader->cancelled = true and closing reader->stream under sub_mutex before leaving the slot in place. Consider mirroring that here: mark the reader cancelled and close its stream (guarded by sub_mutex; lock_ is already held so the lock_->sub_mutex order is preserved) before returning the empty thread, so the reader exits on its next read and stops delivering to the removed callback. Note the comment claiming 'the reader exits on its own once its stream closes' is currently inaccurate because nothing closes the stream in this branch.

Was this helpful? React with 👍 or 👎 to provide feedback.

/// Registering again for a key that already has an active reader replaces the
/// callback in place: the previous reader's stream is closed and its thread
/// is joined before this call returns, and @ref Room then starts a fresh
/// reader bound to the new callback. When this call returns, the previous

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

at first i thought this was a bit verbose, but i actually think this is appropriate

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants


Back | FazBrowse Home | New Git URL