feat(history-sync): emit messaging-history.set event on sync completion and fix race condition - #2442
Merged
DavidsonGomes merged 3 commits intoFeb 24, 2026
Conversation
Copy link
Copy Markdown
Sorry, something went wrong.
…on and fix race condition Reorder webhook emissions (CHATS_SET, MESSAGES_SET) to fire after database persistence, fixing a race condition where consumers received the event before data was queryable. Emit a new MESSAGING_HISTORY_SET event when progress reaches 100%, allowing consumers to know exactly when history sync is complete and messages are available in the database. Register the new event across all transport types (Webhook, WebSocket, RabbitMQ, NATS, SQS, Kafka, Pusher) and validation schemas.
Track message, chat and contact counts across all history sync batches using instance-level counters, so the final event reports accurate totals instead of only the last batch counts. Addresses Sourcery review feedback on PR evolution-foundation#2440.
Copy link
Copy Markdown
Contributor
Reviewer's GuideImplements a reliable history sync completion signal by reordering WhatsApp Baileys history sync webhooks to fire after database persistence and introducing a new messaging-history.set event with cumulative sync counters, wired through all event transports, config, and validation schemas. Sequence diagram for WhatsApp history sync with messaging-history.set and race-condition fixsequenceDiagram
participant WhatsAppBaileys as WhatsApp_Baileys_SDK
participant BaileysService as BaileysStartupService
participant PrismaChat as PrismaRepositoryChat
participant PrismaMessage as PrismaRepositoryMessage
participant ContactHandler as ContactHandle
participant EventController
participant ExternalApp as External_Application
WhatsAppBaileys->>BaileysService: historySyncBatch(progress, isLatest, contacts, chats, messages)
BaileysService->>BaileysService: build chatsRaw from chats
BaileysService->>BaileysService: historySyncChatCount += chatsRaw.length
alt SAVE_DATA.HISTORIC enabled for chats
BaileysService->>PrismaChat: createMany(chatsRaw, skipDuplicates)
PrismaChat-->>BaileysService: chats persisted
end
BaileysService->>EventController: sendDataWebhook(CHATS_SET, chatsRaw)
EventController-->>ExternalApp: CHATS_SET webhook
BaileysService->>BaileysService: build messagesRaw from messages
BaileysService->>BaileysService: historySyncMessageCount += messagesRaw.length
alt SAVE_DATA.HISTORIC enabled for messages
BaileysService->>PrismaMessage: createMany(messagesRaw, skipDuplicates)
PrismaMessage-->>BaileysService: messages persisted
end
BaileysService->>EventController: sendDataWebhook(MESSAGES_SET, messagesRaw, isStatus, instanceId, metadata(progress, isLatest))
EventController-->>ExternalApp: messages.set webhook
BaileysService->>BaileysService: filteredContacts = contacts with name or notify
BaileysService->>BaileysService: historySyncContactCount += filteredContacts.length
BaileysService->>ContactHandler: contacts.upsert(filteredContacts)
ContactHandler-->>BaileysService: contacts upserted
alt progress == 100
BaileysService->>EventController: sendDataWebhook(MESSAGING_HISTORY_SET, totals(messageCount, chatCount, contactCount))
EventController-->>ExternalApp: messaging-history.set webhook
BaileysService->>BaileysService: reset historySyncMessageCount, historySyncChatCount, historySyncContactCount to 0
end
ExternalApp->>ExternalApp: query API for chats, messages, contacts (data already persisted)
Class diagram for BaileysStartupService and event config types with MESSAGING_HISTORY_SETclassDiagram
class BaileysStartupService {
- historySyncMessageCount int
- historySyncChatCount int
- historySyncContactCount int
+ handleHistorySyncBatch(progress int, isLatest boolean, contacts any, chats any, messages any) void
+ sendDataWebhook(event Events, data any, isStatus boolean, instanceId string, metadata any) void
}
class EventsRabbitmq {
+ MESSAGE boolean
+ ACK boolean
+ CALL boolean
+ TYPEBOT_START boolean
+ TYPEBOT_CHANGE_STATUS boolean
+ MESSAGING_HISTORY_SET boolean
}
class EventsWebhook {
+ MESSAGE boolean
+ ACK boolean
+ CALL boolean
+ TYPEBOT_START boolean
+ TYPEBOT_CHANGE_STATUS boolean
+ MESSAGING_HISTORY_SET boolean
+ ERRORS boolean
+ ERRORS_WEBHOOK string
}
class EventsPusher {
+ MESSAGE boolean
+ ACK boolean
+ CALL boolean
+ TYPEBOT_START boolean
+ TYPEBOT_CHANGE_STATUS boolean
+ MESSAGING_HISTORY_SET boolean
}
class SqsEvents {
+ SEND_MESSAGE boolean
+ TYPEBOT_CHANGE_STATUS boolean
+ TYPEBOT_START boolean
+ MESSAGING_HISTORY_SET boolean
}
class Sqs {
+ EVENTS SqsEvents
}
class ConfigService {
+ getDatabaseConfig() Database
+ getEventsRabbitmq() EventsRabbitmq
+ getEventsWebhook() EventsWebhook
+ getEventsPusher() EventsPusher
+ getSqs() Sqs
}
class EventController {
+ events string[]
}
BaileysStartupService --> ConfigService : uses
BaileysStartupService --> EventController : emits_events
ConfigService --> EventsRabbitmq : provides
ConfigService --> EventsWebhook : provides
ConfigService --> EventsPusher : provides
ConfigService --> Sqs : provides
Sqs --> SqsEvents : aggregates
EventController --> EventsRabbitmq : references
EventController --> EventsWebhook : references
EventController --> EventsPusher : references
EventController --> SqsEvents : references
File-Level Changes
Tips and commands Interacting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Sorry, something went wrong.
Copy link
Copy Markdown
Contributor
There was a problem hiding this comment.
Hey - I've found 1 issue, and left some high level feedback:
- The cumulative historySync*Count fields are stored as mutable state on the service instance; if multiple history syncs can run concurrently for the same service (or if batches interleave), these counters could mix data between syncs—consider scoping them to a specific sync session or deriving totals from per-sync context instead.
- Counters are only reset when progress === 100; if a sync aborts, errors, or never reaches 100%, the next sync will emit incorrect cumulative totals—consider adding a timeout/error-path reset or tying reset logic to the lifecycle of a sync rather than just progress.
Please address the comments from this code review:
## Overall Comments
- The cumulative `historySync*Count` fields are stored as mutable state on the service instance; if multiple history syncs can run concurrently for the same service (or if batches interleave), these counters could mix data between syncs—consider scoping them to a specific sync session or deriving totals from per-sync context instead.
- Counters are only reset when `progress === 100`; if a sync aborts, errors, or never reaches 100%, the next sync will emit incorrect cumulative totals—consider adding a timeout/error-path reset or tying reset logic to the lifecycle of a sync rather than just `progress`.
## Individual Comments
### Comment 1
<location path="src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts" line_range="255-258" />
<code_context>
private logBaileys = this.configService.get<Log>('LOG').BAILEYS;
private eventProcessingQueue: Promise<void> = Promise.resolve();
+ // Cumulative history sync counters (reset on sync completion)
+ private historySyncMessageCount = 0;
+ private historySyncChatCount = 0;
+ private historySyncContactCount = 0;
+
// Cache TTL constants (in seconds)
</code_context>
<issue_to_address>
**issue (bug_risk):** History sync counters are global to the instance and may leak across multiple or aborted sync runs.
Because these counters live on the instance and are only reset when `progress === 100`, an aborted history sync (error, disconnect, restart, etc.) will leave them non-zero and the next run will report combined counts from multiple runs. Overlapping syncs on the same instance would also interleave their counts. Please scope these counters to a single sync run (e.g., locals or keyed by sync ID), or reset them at sync start as well as on completion.
</issue_to_address>Sourcery is free for open source - if you like our reviews please consider sharing them ✨ Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Sorry, something went wrong.
Comment thread
src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts
Outdated
Detect new sync runs by tracking lastProgress — when progress resets or decreases, counters are zeroed before accumulating. This prevents stale counts from aborted syncs leaking into subsequent runs. Addresses Sourcery review feedback on PR evolution-foundation#2442.
alexandrereyes
added a commit
to alexandrereyes/evolution-api
that referenced
this pull request
Feb 24, 2026
Detect new sync runs by tracking lastProgress — when progress resets or decreases, counters are zeroed before accumulating. This prevents stale counts from aborted syncs leaking into subsequent runs. Addresses Sourcery review feedback on PR evolution-foundation#2442.
p3dr0574
pushed a commit
to p3dr0574/evolution-api
that referenced
this pull request
Jul 3, 2026
Detect new sync runs by tracking lastProgress — when progress resets or decreases, counters are zeroed before accumulating. This prevents stale counts from aborted syncs leaking into subsequent runs. Addresses Sourcery review feedback on PR evolution-foundation#2442. (cherry picked from commit 6f75944)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When an external application subscribes to messages.set events and tries to query messages from the API upon receiving the event, it hits a race condition: the webhook is emitted before the data is persisted to the database via createMany. This means consumers receive the notification but can't find the messages yet.
Additionally, there is no dedicated event signaling that history sync is complete. Consumers receive N partial messages.set events with incremental progress and must guess when it's safe to start querying.
Solution
1. Fix race condition: persist before emitting
Reorder CHATS_SET and MESSAGES_SET webhook emissions to fire after prismaRepository.*.createMany(), ensuring data is queryable when the event reaches consumers.
2. Emit messaging-history.set when sync completes
When progress === 100, emit a new MESSAGING_HISTORY_SET event with cumulative counts across the entire sync:
{ "event": "messaging-history.set", "instance": "my-instance", "data": { "messageCount": 1420, "chatCount": 350, "contactCount": 500 } }Counts are accumulated via instance-level counters (historySyncMessageCount, historySyncChatCount, historySyncContactCount) that increment on each batch and reset after the completion event is emitted, so they represent the full sync totals — not just the last batch.
3. Register event across all transports
Added MESSAGING_HISTORY_SET to:
🔗 Related Issue
Builds on top of #2260 which added isLatest and progress to the messages.set payload.
🧪 Type of Change
🧪 Testing
✅ Checklist
📝 Files Changed
Summary by Sourcery
Add a completion event for WhatsApp history sync and ensure data is persisted before emitting related events.
New Features:
Bug Fixes:
Enhancements:
Build:
Chores: