Broker bindings
The envelope is broker-agnostic — it is just the message body. A binding says how a given broker carries that body natively, so a consumer can route by URN and continue a trace without decoding the body, and so a message one SDK produces is consumed byte-for-byte by another over the same broker.
Two rules hold for every binding:
- The body is always the canonical envelope — compact UTF-8 JSON, byte-identical across SDKs. Native metadata is a redundant, routable projection of the body, never a replacement for it.
- The envelope is frozen at
schema_version: 1. A new broker is purely additive — it never changes the wire format.
Redis
Reliable-queue pattern: RPUSH to produce, a blocking move to a :processing list to
reserve, then remove on ack. Redis lists carry no per-message metadata, so routing and
tracing read the envelope body directly (job/urn, trace_id). At-least-once via the
processing list.
RabbitMQ (AMQP 0-9-1)
The body is the envelope; the contract is also projected onto AMQP properties and headers so a consumer can route on the broker’s metadata:
| AMQP field | Carries |
|---|---|
type |
the URN (job) |
correlation_id |
trace_id |
message_id |
meta.id |
header x-attempts |
attempts |
header x-schema-version |
meta.schema_version |
header x-source-lang |
meta.lang |
Messages are application/json, persistent delivery, on durable queues; consumed with
basic.get + manual ack (at-least-once).
Amazon SQS
The canonical envelope is the MessageBody. On produce, the transport projects the
envelope onto native MessageAttributes — a routable view of the body. Ids and
strings are DataType String; counters are DataType Number:
| MessageAttribute | DataType | Value |
|---|---|---|
bq-job |
String | the URN (job) |
bq-trace-id |
String | trace_id |
bq-message-id |
String | meta.id |
bq-schema-version |
Number | meta.schema_version (1) |
bq-source-lang |
String | meta.lang |
bq-created-at |
Number | meta.created_at as epoch milliseconds |
Consuming uses SQS’s native delivery semantics:
- A receive reserves the message for the visibility timeout; a successfully handled
message is removed with
DeleteMessage. A failing handler simply does not delete it — SQS redelivers after the visibility timeout (at-least-once). attemptsis reconciled tomax(body.attempts, ApproximateReceiveCount − 1). The first delivery reads0; a runtime-incremented count is never lowered; an absent or non-numericApproximateReceiveCountis ignored. A drop-in driver that instead surfaces the broker’s native count (e.g. Laravel’sSqsJob::attempts()) documents that divergence.
FIFO queues (.fifo) set MessageGroupId (the configured group, else the queue
name) and MessageDeduplicationId = meta.id — unless the queue uses content-based
deduplication. Delayed delivery uses DelaySeconds, capped at SQS’s 900-second maximum.
This binding is implemented identically across every SDK that ships an SQS transport
(Go, Python, Node, Java, PHP, .NET) and is locked by the
conformance suite, so the projected
attributes and the reconciled attempts are guaranteed to match.
Azure Service Bus
The canonical envelope is the Body. Azure Service Bus has first-class native slots
for almost every envelope concept, so the binding maps onto native message fields and
needs only two custom application properties:
| Field | Value |
|---|---|
Subject (a.k.a. Label) |
the URN (job) — route on Subject without reading the body |
CorrelationId |
trace_id |
MessageId |
meta.id (enables ASB duplicate detection) |
ContentType |
application/json |
DeliveryCount |
broker-maintained, 1-based — the native attempts source |
ApplicationProperties["bq-schema-version"] |
meta.schema_version (1) |
ApplicationProperties["bq-source-lang"] |
meta.lang |
ApplicationProperties["bq-created-at"] |
meta.created_at (epoch ms, convenience mirror) |
Application properties are native AMQP-typed values (numbers stay numbers), not the
DataType-wrapped strings SQS uses.
Consuming uses ASB’s PeekLock model:
- A receive reserves the message for the lock duration; a handled message is removed
with
Complete. A failing handlerAbandons it — the broker redelivers and incrementsDeliveryCount(at-least-once). AtMaxDeliveryCountASB auto-moves it to the native$DeadLetterQueuesub-queue. attemptsis reconciled tomax(body.attempts, DeliveryCount − 1):DeliveryCount(1-based) is the native floor (first delivery reads0), and a runtime-incremented body count is never lowered. The rule is identical for the native-consumer SDKs (.NET, Java, Node) and the runtime-transport SDKs (Python, Go).
Delayed delivery is native — set ScheduledEnqueueTime to now + delay. Auth is a
connection string or Azure AD (DefaultAzureCredential); transport is AMQP 1.0 over TLS
(or WebSockets). An optional SessionId (FIFO) is opt-in and not a contract field.
This binding is implemented identically across every SDK that ships an ASB transport (.NET, Java, Python, Node, Go) and is locked by the conformance suite. PHP is deferred (no modern official client).
Apache Pulsar
The canonical envelope is the message payload. Pulsar message properties are
string→string, so the binding projects a redundant, routable view of the body onto
bq- properties (every value stringified) and keeps the body authoritative:
| Property | Value |
|---|---|
bq-job |
the URN (job) — route on bq-job without reading the body |
bq-trace-id |
trace_id |
bq-message-id |
meta.id |
bq-schema-version |
meta.schema_version ("1") |
bq-source-lang |
meta.lang |
bq-attempts |
attempts — the authoritative count, carried in the body |
publishTime |
mirrors meta.created_at (broker-set; body authoritative) |
Properties are strings (Pulsar has no typed properties), so numbers are stringified — unlike ASB’s native AMQP-typed values.
Consuming receives one message at a time:
- A handled message is
acknowledged. A failing handlernegativeAcknowledges it — the broker redelivers it (at-least-once) and incrementsRedeliveryCount. With a nativeDeadLetterPolicyit eventually moves to the cross-language<queue>.dlqtopic. attemptsis reconciled tomax(body.attempts, RedeliveryCount).RedeliveryCountis 0-based (0 on first delivery), so it maps directly with no −1 — and a runtime-incremented body count is never lowered. The rule is identical for the native-consumer SDKs (.NET, Java, Node) and the runtime-transport SDKs (Python, Go).
Delayed delivery is native — deliverAfter (relative) or deliverAt (absolute), also
mirrored on a bq-delay property. The default subscription is Shared, named
babelqueue; topics default to persistent://public/default/<queue>. Auth is via the
service URL (pulsar:// or pulsar+ssl://) plus any client-configured TLS/token.
This binding is implemented identically across every SDK that ships a Pulsar transport
(.NET, Java, Python, Node, Go) and is locked by the
conformance suite. PHP reaches Pulsar over
Pulsar’s native WebSocket API (PulsarTransport to produce, PulsarConsumer to consume)
— the same envelope and bq- properties, just a different access path; it round-trips live
against the native SDKs.
Apache Kafka
Kafka is a partitioned, append-only log with consumer-group offset commits — not a
queue with per-message ack. It has no native per-message ack, delayed delivery,
dead-letter queue, or delivery counter, so this binding absorbs all four in the transport
layer (the envelope stays schema_version: 1). The canonical envelope is the record
value; the contract fields move to bq- record headers (UTF-8 byte strings, so
integers are stringified):
| Field | Value |
|---|---|
value |
the canonical envelope JSON |
timestamp |
mirrors meta.created_at (Unix ms) |
header bq-job |
the URN (job) — route on bq-job without reading the body |
header bq-trace-id |
trace_id |
header bq-message-id |
meta.id |
header bq-schema-version |
meta.schema_version ("1") |
header bq-source-lang |
meta.lang |
header bq-attempts |
attempts — the authoritative retry counter (Kafka has no native one) |
Consuming is process-then-commit (manual commit, enable.auto.commit = false): a
record is reserved by being polled, the handler runs, and only then is the offset committed
(Commit). A crash before commit redelivers it on the next poll — at-least-once; handlers
MUST be idempotent (dedupe on meta.id).
attemptsis reconciled to thebq-attemptsheader when present (authoritative), else the body’s ownattempts(the fallback for a non-BabelQueue producer). This is not a max — the header overrides the body.- Retry / delay is SDK-owned, since Kafka has neither: a failing handler republishes the
envelope to a tiered retry topic
<topic>.retry.<n>withbq-attempts + 1(andbq-delay/bq-original-topic), then commits; a retry-topic worker re-injects it after the tier delay. A delay or release with no retry topics configured raises rather than silently dropping. - Terminal failure → DLQ: at max-tries the envelope goes to
<topic>.dlqwith the additivedead_letterblock (opt-in; if disabled, terminal failures degrade to commit-and-drop).
Connection is bootstrap.servers + a group.id; auth is SASL_SSL / SSL via the
native client. This binding is implemented identically across Java, Go, Node, Python and .NET
and is locked by the conformance suite. PHP
ships both halves over ext-rdkafka (opt-in — the one binding that relaxes the
zero-extension rule): KafkaTransport to produce, KafkaConsumer (process-then-commit) to
consume, plus the SDK-owned retry-topic machinery (KafkaRetryRouter + KafkaRetryConsumer);
proven live Java→PHP.
Apache ActiveMQ Artemis
Artemis speaks AMQP 1.0 (not RabbitMQ’s 0-9-1), and gives the binding native primitives —
per-message settlement, scheduled delivery, a delivery counter and a dead-letter address — so
it maps onto them rather than re-implementing. The canonical envelope is the message body;
the contract fields ride the slots a JMS peer reads, plus the string-valued bq_ application
properties:
| Field | Value |
|---|---|
| body | the canonical envelope JSON |
annotation x-opt-jms-type |
the URN (job) → JMSType — route on it without reading the body |
correlation-id |
trace_id → JMSCorrelationID |
creation-time |
meta.created_at (Unix ms) → JMSTimestamp |
property bq_schema_version |
meta.schema_version ("1") |
property bq_source_lang |
meta.lang |
property bq_attempts |
attempts — a 0-based mirror (the body stays authoritative) |
property bq_app_id |
"babelqueue" |
These property names use underscores, not the hyphens of the Kafka/Pulsar bindings: a JMS
property name must be a valid Java identifier (no -), and the Java binding produces/consumes
over JMS, so every Artemis SDK uses the same JMS-legal bq_ form for cross-protocol parity.
Unlike Kafka/Pulsar, the URN / trace_id / message-id are not bq_ properties — they ride
the JMS-native slots (x-opt-jms-type, correlation-id, the broker-set message-id), so a plain
JMS consumer routes and correlates with no BabelQueue awareness.
Consuming reserves one message at a time and settles it per message:
- A handled message is
accepted (acknowledged). A failing handlerreleases it — the broker redelivers it (at-least-once) and increments the AMQPdelivery-count. At max-tries the envelope goes to the cross-language<queue>.dlqwith the additivedead_letterblock (alongside Artemis’s own dead-letter address). attemptsis reconciled tomax(body.attempts, delivery-count). The AMQPdelivery-countis 0-based (0 on first delivery), so it maps directly with no −1 — and a runtime-incremented body count is never lowered. The Java binding consumes over JMS and reads the 1-basedJMSXDeliveryCount, subtracting 1 to arrive at the same 0-basedattempts. The rule is identical for the native-consumer SDKs (.NET, Java, Node) and the runtime-transport SDKs (Python, Go).
Delayed delivery is native — JMS 2.0 setDeliveryDelay (Java) or the x-opt-delivery-time
annotation (AMQP), also mirrored on a bq_delay property. Connection is the broker’s AMQP
acceptor (amqp:// / amqps://, default port 5672); Java connects over JMS via the Artemis
client. This binding is implemented identically across Java (JMS), .NET, Python, Node and Go (all
AMQP 1.0) and is locked by the
conformance suite. PHP reaches Artemis over
STOMP (StompTransport to produce; Laravel ships the babelqueue-artemis drop-in consume
driver) — Artemis bridges STOMP ↔ AMQP 1.0 ↔ JMS on the same address, so a STOMP-produced
message is consumed natively by the JMS and AMQP-1.0 SDKs, and vice versa.
Other brokers
With Amazon SQS, Azure Service Bus, Apache Pulsar, Apache Kafka and Apache ActiveMQ Artemis all
shipped, the binding catalogue covers every broker on the expansion roadmap. New brokers follow
the same shape — body identical, contract fields projected onto native metadata — and ship as
additive MINOR releases. The envelope stays schema_version: 1.