Event-Driven Architecture on AWS: EventBridge, SQS, Kafka, and Reliable Integration Patterns

Event-driven architecture can remove synchronous dependencies, absorb traffic spikes, and let teams evolve systems independently. It can also create duplicate processing, invisible failure queues, incompatible schemas, and production incidents that are difficult to reconstruct.

The difference is not whether a system publishes events. The difference is whether event contracts, delivery semantics, ownership, retries, replay, and operations are designed as part of the architecture.

This matters for SaaS platforms, fintech systems, data pipelines, AI automation, and internal business integrations. In each case, one business change may need to update several independent consumers without turning the originating request into a fragile chain of API calls.

AWS provides several overlapping services for this work: Amazon EventBridge, SQS, SNS, Kinesis, and Managed Streaming for Apache Kafka. Choosing the right one requires understanding what an event means, how consumers fail, whether order matters, and who owns the integration.

This guide explains those decisions and presents a production design for reliable event-driven systems.

Why event-driven architecture is timely

AWS recently expanded Amazon EventBridge Custom Event Bus with retained events, subscriber-controlled routing, point-in-time starting positions, pause and resume, FIFO delivery within event groups, publish-time deduplication, and cross-account sharing through AWS Resource Access Manager.

These capabilities address several limitations that previously pushed teams toward additional queues, archives, or streaming infrastructure. They do not make every event-driven workload an EventBridge workload, but they change the trade-offs for enterprise integration and multi-account platforms.

The current EventBridge Custom Event Bus documentation distinguishes the newer subscriber model from Custom Event Bus - Classic, which continues to use rules and targets. Both remain available, so architecture and migration decisions need to be explicit.

At the same time, AI agents and workflow automation are increasing the number of system-to-system actions. More producers and consumers make event governance, authorization, and auditability more important, not less.

Events, commands, and messages are different

Many integration problems begin with unclear semantics.

Event

An event is a fact that already happened:

• OrderPlaced
• PaymentAuthorized
• CustomerAddressChanged
• ModelEvaluationCompleted

The producer does not instruct a specific consumer. Zero, one, or many consumers may react.

Command

A command requests that a particular capability perform work:

• AuthorizePayment
• GenerateInvoice
• ProvisionTenantEnvironment

It has an intended owner and may be accepted or rejected. Commands are often delivered through a queue or direct API rather than broadcast through a general event bus.

Message

A message is the transport envelope carrying an event, command, or other payload. Delivery guarantees apply to messages; business semantics apply to the event or command inside.

Naming every payload SomethingEvent does not make the architecture event-driven. If a producer publishes SendInvoiceEmailEvent and expects exactly one email service to execute it, the payload behaves like a command.

Clear semantics determine ownership, retry behavior, observability, and compatibility.

When asynchronous events are the right choice

Use events when:

• the producer should not know all downstream consumers;
• multiple systems react independently to the same business fact;
• consumers can tolerate eventual consistency;
• traffic must be buffered or processed at different rates;
• new consumers should be added without changing the producer;
• integrations cross account, team, or organizational boundaries;
• audit and replay of state changes are valuable.

Prefer a synchronous API when the caller needs an immediate answer to continue, the operation has one clear owner, and failure must be presented directly to the user.

Many systems need both. An API can validate and commit an order synchronously, then publish OrderPlaced for fulfillment, analytics, CRM, fraud monitoring, and notifications.

Do not use asynchronous messaging to hide an undefined transaction. If the user cannot understand whether an operation succeeded, the architecture has moved uncertainty rather than managed it.

Choosing between EventBridge, SQS, SNS, Kinesis, and Kafka

The services overlap, but their primary abstractions differ.

Use an event bus when the central need is routing facts by content to independent subscribers. Use a queue when the central need is assigning units of work and applying backpressure. Use a stream when consumers need an ordered log, independent offsets, substantial replay, or continuous event processing.

Combining them is common. EventBridge can route a business event into a consumer-owned SQS queue. The queue protects the consumer from traffic bursts and dependency outages. The consumer processes at its own rate.

A production reference architecture

Consider an order service that commits an order and needs to notify payment, fulfillment, analytics, and customer communication systems.

1. Commit business state

The order service validates the request and writes the authoritative order record in its database. The database transaction defines whether OrderPlaced is true.

2. Record an outbox event atomically

In the same transaction, the service writes an outbox row containing the event ID, type, aggregate ID, version, payload, timestamp, and publication status.

This prevents the dual-write failure where the database commit succeeds but event publication fails, or the event is published for a transaction that later rolls back. AWS documents this approach in its transactional outbox guidance.

3. Publish from the outbox

A relay reads committed outbox rows and publishes them to EventBridge, Kafka, or another broker. The relay marks progress only after the broker acknowledges the event.

The relay may publish the same event more than once if it crashes after publication but before updating its checkpoint. That is expected. The event ID remains stable across attempts.

Our change data capture guide covers polling publishers and log-based CDC options such as Debezium and AWS DMS.

4. Route to consumer-owned buffers

Subscribers filter events by type, domain, tenant class, region, or other stable attributes. Critical consumers normally receive events through their own SQS queue or stream subscription rather than sharing one queue.

Each consumer then controls throughput, retry policy, scaling, and dead-letter handling without affecting other consumers.

5. Process idempotently

The consumer records the event ID or applies a naturally idempotent state transition. It acknowledges the message only after the business effect succeeds.

6. Observe and reconcile

Metrics track publish lag, subscriber delivery failures, queue age, consumer errors, retries, and dead-letter volume. Reconciliation jobs compare critical business states across systems instead of assuming successful message delivery implies correct business outcomes.

The transactional outbox is usually the starting point

Publishing after a database commit looks simple:

1. Save the order.
2. Publish OrderPlaced.

The process can crash between those steps. Reversing the order creates the opposite problem: consumers may observe an event for data that never commits.

A distributed transaction across the application database and broker is rarely available or desirable. The outbox pattern uses the local database transaction as the reliable boundary.

An outbox implementation needs operational decisions:

• How quickly must events be published?
• Is polling acceptable, or is log-based capture required?
• How are rows partitioned and indexed?
• When are published rows archived or deleted?
• How is poison data isolated?
• How does the relay preserve aggregate ordering?
• How do operators replay a bounded set safely?

Do not let an unbounded outbox table become a hidden production bottleneck. Monitor oldest unpublished age, publication rate, failures, and table growth.

Design event contracts for evolution

An event is a public contract even when it is used only inside one company. Once several teams consume it, changing the payload becomes a distributed deployment problem.

A useful envelope includes:

• globally unique event ID;
• event type and schema version;
• occurrence time and publication time;
• producer and environment;
• aggregate or entity identifier;
• correlation and causation identifiers;
• tenant or business partition where appropriate;
• data classification or sensitivity marker;
• payload containing the business fact.

Prefer business names over database names. CustomerEmailChanged is more stable than CustomerRowUpdated. Do not publish an entire database row merely because it is convenient.

Compatibility rules

Add optional fields rather than renaming or changing meaning. Consumers should tolerate unknown fields. Required fields need stable semantics.

When a breaking change is unavoidable, publish a new event version or type and run both during a measured migration. Define who owns the schema, how compatibility is tested, and when old versions can be retired.

A schema registry helps with syntax and compatibility. It cannot decide whether a field's business meaning changed.

Idempotency is not optional

Most practical event systems provide at-least-once delivery somewhere in the path. Retries, publisher ambiguity, replay, and operator intervention can all create duplicates.

A consumer should be able to receive the same event repeatedly without creating an incorrect result.

Common patterns include:

• insert the event ID into a processed-events table with a unique constraint;
• use the event ID as the idempotency key for an external provider;
• update state only when the incoming aggregate version is newer;
• use compare-and-set conditions in the destination database;
• store a deterministic operation result and return it on retry.

Deduplication at the broker is useful but not sufficient. The newer EventBridge Custom Event Bus can suppress duplicate publishes within a documented five-minute window and can combine this with FIFO subscriber delivery. Events repeated outside that window, in a different scope, or during a business replay still require idempotent consumers. AWS explicitly recommends retaining consumer idempotency in its ordering and deduplication guidance.

Exactly-once business effects are an end-to-end property, not a checkbox on the transport.

Ordering requires a business scope

Global ordering is expensive and rarely necessary. Most businesses need order within an entity or account:

• events for one order;
• ledger changes for one account;
• commands for one device;
• lifecycle transitions for one tenant.

Use that identifier as the partition key, message group, or event group. Different groups can process concurrently.

Ordering introduces head-of-line blocking. If one event repeatedly fails, later events in the same group may wait. Keep groups bounded, classify permanent failures, and provide a remediation path.

Consumers should still validate state transitions. Ordered delivery does not guarantee that every event was published correctly or that an operator will never replay older data.

Retries, dead-letter queues, and poison events

Retries should match the failure.

Retry transient network errors, throttling, and temporary dependency failures with exponential backoff and jitter. Do not repeatedly retry invalid payloads, failed authorization, deleted resources, or business-rule rejection.

Set a retry budget. Unlimited retries turn one bad event into permanent load and can block ordered groups.

A dead-letter queue is not a resolution strategy. It is a quarantine boundary. A complete process includes:

• an alert based on age and volume;
• searchable failure context;
• redaction of sensitive payloads;
• an identified owner;
• tools to inspect, correct, discard, or replay;
• protection against replaying the same poison event indefinitely;
• an audit record of operator actions.

Monitor the age of the oldest message, not only queue depth. A small queue containing one week-old payment event is more serious than a larger queue draining normally.

Replay is a product capability

Replay supports new consumers, recovery from a consumer bug, data reconstruction, and changed business rules. It also repeats side effects if it is not designed safely.

Before replaying, define:

• the exact time or event-ID range;
• the target consumer and environment;
• whether external actions are suppressed;
• how replayed events are identified;
• expected throughput and cost;
• how duplicate state is prevented;
• how completion is validated.

The enhanced EventBridge Custom Event Bus retains events for a configured period and allows a subscriber to start at the horizon or a point in time. AWS describes this in its retained-event replay documentation. Kafka and Kinesis use consumer offsets and retention models for similar recovery scenarios.

Test replay in advance. A capability first exercised during an incident is an experiment.

Multi-account EventBridge architecture

AWS organizations commonly separate workloads by product, environment, or security boundary. Event routing then crosses accounts.

A centralized bus can simplify discovery and governance, but centralization creates a critical shared dependency. A fully decentralized mesh preserves team control but can produce duplicated routing, inconsistent policy, and unclear costs.

The newer EventBridge model supports sharing a bus through AWS RAM while allowing consumer accounts to manage subscribers. This can separate responsibilities:

• the platform team owns bus availability, retention, encryption, and base policy;
• producer teams own event contracts and publication quality;
• consumer teams own filters, targets, retries, scaling, and business processing;
• security governs which accounts and event sources may publish or subscribe.

Use resource policies and IAM conditions to restrict producers by organization, account, source, and event type. Do not grant broad publish access simply because the bus is internal.

For platform-level implementation and ownership practices, see our guide to internal developer platforms.

Observability for event-driven systems

Distributed traces are useful, but an event may be consumed hours later or replayed months later. Preserve both trace context and durable business correlation identifiers.

Key metrics include:

• outbox oldest unpublished age;
• publication errors and throttling;
• event bus delivery failures;
• subscriber lag or queue age;
• messages in flight;
• retry and dead-letter rates;
• consumer processing latency;
• duplicate and stale-event counts;
• reconciliation differences;
• cost per published and delivered business event.

Log the event ID, event type, aggregate ID, consumer, attempt, result, and correlation ID. Avoid logging full sensitive payloads by default.

Instrument producer, broker integration, and consumer spans with consistent attributes. Our OpenTelemetry production guide explains context propagation and cardinality controls for distributed systems.

Technical delivery metrics are not enough. Add business metrics such as orders awaiting fulfillment, payments missing settlement confirmation, or customer changes not reflected in downstream systems.

Security and data governance

Events often travel farther than API responses and may be retained for replay. Minimize payloads accordingly.

Production controls should include:

• encryption in transit and at rest;
• least-privilege publish and subscribe permissions;
• separate production and non-production buses;
• producer authentication and source restrictions;
• consumer-owned delivery roles;
• payload classification and field-level minimization;
• schema validation at publication or ingestion;
• retention aligned with legal and business requirements;
• immutable audit logs for policy and subscription changes;
• protection against event injection and oversized payloads.

Do not place secrets, access tokens, or unnecessary personal data in events. Where a consumer can retrieve authoritative data securely, publish an identifier and the minimum context needed to decide whether retrieval is appropriate.

Multi-tenant platforms must prevent one tenant's event from reaching another tenant's processing boundary. Tenant context should be validated, not merely copied from an untrusted payload. The isolation patterns in our multi-tenant SaaS architecture guide apply equally to asynchronous consumers.

Events and durable workflows solve different problems

An event bus distributes facts to independent consumers. A durable workflow coordinates a defined multi-step outcome.

Use choreography when consumers react independently and no single service needs to control the entire sequence. Use orchestration when steps have an explicit order, wait for callbacks, share a deadline, or require coordinated compensation.

For example, OrderPlaced can fan out as an event. The fulfillment process may then run as a durable workflow that reserves inventory, books shipment, waits for carrier confirmation, and compensates failures.

Combining the patterns prevents two common mistakes: turning the event bus into an implicit state machine, and turning one workflow into a central owner of every downstream reaction. See our durable execution architecture guide for Temporal and Step Functions patterns.

Event-driven AI automation

AI systems can consume events to classify documents, detect anomalies, summarize cases, or propose actions. Their probabilistic output should not silently become a new business fact.

A safer pattern is:

1. Consume a stable business event.
2. Load authorized context using the event's identifiers.
3. Run the model as an observable activity.
4. Validate the structured output against deterministic rules.
5. Request human approval for high-impact actions.
6. Execute tools with idempotency keys.
7. Publish a new explicit result event.

Record model, prompt version, input references, policy decision, cost, and result. An event replay should not unknowingly repeat an external action or produce a different decision without a versioned reason.

The control and evaluation practices in our AgentOps guide are important when agents become event consumers or producers.

Common failure patterns

Publishing database rows as events

This couples consumers to the producer's storage model and exposes fields without a business contract. Publish meaningful domain facts.

One shared queue for independent consumers

Messages are distributed among consumers rather than copied to each. Give each independent consumption path its own queue or subscription.

Assuming the broker owns data consistency

The broker can deliver a message reliably while the producer's database and event stream disagree. Use an outbox and reconciliation.

Treating a DLQ as an archive

Unowned dead-letter queues accumulate business failures. Define alerts, ownership, tooling, and resolution objectives.

Requiring global order

Global order limits concurrency and increases coupling. Define the smallest business scope in which order matters.

No schema lifecycle

Events outlive deployments. Test compatibility, version breaking changes, and track consumers before retirement.

Choosing Kafka for status

Kafka is valuable when its log and processing model solve real requirements. It is not automatically better architecture than a managed bus and queues for moderate business-event traffic.

A practical implementation plan

Phase 1: Map one business flow

Identify producer, authoritative state, consumers, latency expectations, order requirements, data sensitivity, and failure impact. Separate facts from commands.

Phase 2: Define the event contract

Create the envelope, schema, ownership, compatibility rules, and sample payloads. Review it with actual consumer teams.

Phase 3: Implement reliable publication

Add a transactional outbox or equivalent atomic mechanism. Instrument publication delay and test process crashes around every state transition.

Phase 4: Add one isolated consumer

Use a consumer-owned queue or subscription, idempotency storage, bounded retries, dead-letter handling, and business-level monitoring.

Phase 5: Test failure and replay

Deliver duplicates and out-of-order events. Disable dependencies. Fill the queue. Inject invalid schemas. Replay a bounded time range and verify the result through reconciliation.

Phase 6: Establish governance

Publish ownership, naming, schema, security, retention, cost allocation, and deprecation standards. Automate them through the engineering platform where possible.

Build for failure before scaling fan-out

Event-driven architecture creates leverage when business facts can move reliably between independently owned systems. Its value comes from loose coupling and recoverability, not from the number of brokers in the diagram.

Start with a clear event contract, atomic publication, idempotent consumers, bounded retries, observable queues, and tested replay. Choose EventBridge, SQS, SNS, Kinesis, or Kafka according to the delivery model rather than familiarity.

BoundLayer designs and implements event-driven architectures, AWS integrations, multi-account event platforms, transactional outbox and CDC pipelines, workflow orchestration, observability, and production recovery tooling. We can audit an existing messaging estate or build a reliable event backbone around a specific business process.