Problem
The repo's queue-contract convention (doc/rfc/messagequeue-contract.md) is proto3 payloads serialized as protojson, with a contract package owning both the messages and their topic_keys bindings — api/runway/messagequeue/ is the reference example and stovepipe/core/messagequeue/ follows it. The submitqueue domain has not been ported: there is no submitqueue/core/messagequeue/ package, and 11 of its 12 topics still carry hand-rolled JSON via ToBytes/FromBytes methods on entity types.
Current state per topic:
| Topic key |
Payload |
Serialization |
| start |
RequestID |
entity JSON |
| cancel |
CancelRequest |
entity JSON |
| validate |
RequestID |
entity JSON |
| batch |
RequestID |
entity JSON |
| score |
BatchID |
entity JSON |
| speculate |
BatchID |
entity JSON |
| prioritize |
QueueID |
entity JSON |
| build |
BatchID |
entity JSON |
| buildsignal |
BuildID |
entity JSON |
| conclude |
BatchID |
entity JSON |
| log |
RequestLog |
entity JSON |
| merge |
MergeRequest |
✅ protojson (api/runway/messagequeue — correctly borrowed; runway owns that queue's contract) |
Consequences of the split: no additive-evolution guarantees (protojson's unknown-field discard, UPPER_SNAKE enums, int64-as-string are only enforced on merge), no topic_keys binding or contract test for internal topics, and queue-payload serialization concerns living on domain entities (which are supposed to be pure data).
Work
- Create submitqueue/core/messagequeue/ mirroring the reference layout: proto/ sources, committed protopb/, and the generic protojson glue (Marshal / Unmarshal[T] / TopicKeys) — same as stovepipe/core/messagequeue/.
- Define proto messages for each payload shape (request-ID carrier, batch-ID carrier, build-ID carrier, queue-ID carrier, cancel request, request log) and bind each topic key to its message via the topic_keys option from api/base/messagequeue.
- Add the contract test: protojson round-trip per message + every topic key bound to exactly one message.
- Migrate producers and consumers topic by topic (gateway Land/Cancel publishes, all orchestrator stage controllers, core/request.PublishLog, and the DLQ controllers/republish tooling — anything touching msg.Payload).
- Once no queue payload uses them, remove ToBytes/FromBytes from the submitqueue entity types (RequestID, BatchID, BuildID, QueueID, CancelRequest, RequestLog, and the full-entity variants) so entities go back to being pure data.
- Bazel visibility stays domain-scoped per the internal-contract rule.
Caveats
- Wire-format flip: protojson output (snake_case fields, int64-as-string) differs from the current encoding/json entity output, so in-flight messages from before a cutover won't parse after it. Port topic-by-topic with drained queues (fine at the current pre-prod stage), or give consumers a transitional dual-format read if needed.
- The log topic crosses gateway ↔ orchestrator but both are submitqueue-domain services, so the internal contract location (submitqueue/core/messagequeue/) is correct — it must not go under api/.
- The message-ID convention is a separate concern tracked in messagequeue: entity-ID message IDs + publish dedup silently drop wake-up events; use intent-scoped message IDs #352 (intent-scoped message IDs); porting payloads to proto neither depends on nor resolves it, but call sites will be touched by both — coordinate to avoid churn.
Related
Reactions are currently unavailable
Problem
The repo's queue-contract convention (doc/rfc/messagequeue-contract.md) is proto3 payloads serialized as protojson, with a contract package owning both the messages and their topic_keys bindings — api/runway/messagequeue/ is the reference example and stovepipe/core/messagequeue/ follows it. The submitqueue domain has not been ported: there is no submitqueue/core/messagequeue/ package, and 11 of its 12 topics still carry hand-rolled JSON via ToBytes/FromBytes methods on entity types.
Current state per topic:
Consequences of the split: no additive-evolution guarantees (protojson's unknown-field discard, UPPER_SNAKE enums, int64-as-string are only enforced on merge), no topic_keys binding or contract test for internal topics, and queue-payload serialization concerns living on domain entities (which are supposed to be pure data).
Work
Caveats
Related