Event-Driven Architecture

TL;DR

• SNS is the fanout primitive: one publisher, many subscribers, with no coupling between them.
• SQS adds durability that SNS lacks: buffering, visibility timeout, retries, and dead-letter queues.
• SNS→SQS (fanout to multiple queues) decouples producer from consumer availability; SNS→Lambda directly has no buffer and can lose messages or overwhelm consumers.
• EventBridge adds sophisticated routing: content-based filtering, schema registry, and integration with dozens of AWS services and SaaS partners.
• EventBridge Pipes solves point-to-point integration between source and target with built-in filtering and optional transformation.
• Amazon MQ is for protocol interoperability (AMQP, MQTT, STOMP) with legacy message brokers; not the default for AWS-native designs.
• Event sourcing stores the full sequence of events as the source of truth, enabling complete audit trails and replay capability.

Core Concepts

Direct Service Calls vs. Messaging Patterns

The fundamental choice: should Service A directly call Service B, or should Service A publish an event that B consumes?

Direct service-to-service calls are simple for 2 services but become fragile as the number of consumers grows. If Service A calls Service B directly, and Service B calls Service C, then Service C’s downtime breaks the entire chain. Adding a fourth consumer requires code changes to Service A. The coupling is direct and unavoidable.

Messaging patterns decouple publisher from consumers. Service A publishes an event without knowing who listens. Service B and Service C independently consume that event. Adding Service D doesn’t require touching Service A’s code. The coupling is minimal and late-binding.

SNS: The Fanout Primitive

SNS (Simple Notification Service) implements the pub/sub pattern: a publisher sends a message to a topic, and all subscribers receive a copy.

Advantage: One publish reaches many subscribers without the publisher knowing about them.

Limitation: SNS alone provides no buffering. If a subscriber is temporarily down when a message is published, the message is lost (with configurable retries, but not a queue). SNS is also not ideal for high-throughput consumption because subscribers can be overwhelmed if messages arrive faster than they can process.

SQS: Durable Buffering

SQS (Simple Queue Service) provides a managed queue: messages are buffered until a consumer explicitly receives and deletes them.

Key features:

• Visibility timeout: After a consumer receives a message, the message becomes invisible to other consumers for a configurable duration (default 30 seconds). If the consumer fails to delete it, the message reappears for another consumer to try.
• Dead-letter queues (DLQs): After a message fails to process (defined by a max-receive count), it moves to a separate DLQ for investigation.
• Durability: Messages persist in the queue until deleted, even if the consumer is down.

SNS→SQS Pattern

The most durable fanout: SNS publishes to a topic, and each consumer has its own SQS queue subscribed to that topic. Messages are published once to SNS and delivered to multiple queues.

Benefit: Each consumer processes at its own pace. If Consumer B is down, its queue holds messages until it’s back up. Consumer A is not affected.

Anti-pattern: SNS→Lambda directly (no SQS buffer). If Lambda is throttled or failing, messages can be lost or the Lambda invocation throttled, which back-pressures the publisher.

EventBridge: Sophisticated Event Routing

EventBridge is a more powerful event-routing service: it ingests events from multiple sources, matches them against rules, and routes matching events to targets.

Key features:

• Content-based filtering: Define rules like “if the event contains status: "critical"”, not just topic-based filtering.
• Schema registry: Catalog event schemas and use them to generate code.
• Integrations: Dozens of AWS services and SaaS partners act as sources (EC2 events, CodeBuild state changes, SaaS webhooks).
• Targets: Lambda, SQS, SNS, Kinesis, and 70+ other services.

EventBridge is the right choice when routing logic is complex: not all orders trigger billing (only paid plans), so a content-based rule filters them.

EventBridge Pipes: Point-to-Point Integration

Pipes simplifies a common pattern: take events from a source (Kinesis stream, DynamoDB stream, SQS queue), optionally filter and enrich them, and write to a target.

Before Pipes: Custom Lambda function polling the source, transforming events, and writing to the target.

With Pipes: Declarative configuration. The pipe handles polling, optional filtering, optional transformation (via Lambda or built-in functions), and delivery to the target. No Lambda function to maintain.

Amazon MQ: Legacy Protocol Interoperability

Amazon MQ is a managed message broker supporting AMQP, MQTT, and STOMP protocols. It exists for teams migrating from ActiveMQ or RabbitMQ, or integrating with third-party systems expecting these protocols.

When to use: A team is migrating from an existing RabbitMQ deployment and needs the same protocol compatibility during transition.

When not to use: A greenfield AWS-native application. SNS/SQS are purpose-built for AWS integration and cost less.

Event Sourcing: The Event Log as Source of Truth

Instead of storing current state (e.g., “account balance: $1000”), event sourcing stores the sequence of events that led to that state: [“transfer_in($500)”, “withdrawal($200)”, …]. Current state is derived by replaying events.

Benefit: Complete, replayable audit trail. “Show me every transaction on this account” is trivial; “how did we get to the current balance” is answered by the event log itself.

Trade-off: Querying current state is awkward. To answer “which accounts are overdrawn”, you must replay all events for all accounts, which is slow. Event sourcing is usually paired with CQRS (Command Query Responsibility Segregation): maintain a separate read projection (e.g., a table with “account_id, current_balance”) updated as events are appended. Queries hit the projection; writes go through the event log.

Use cases: Financial systems, compliance-heavy domains, anything where “how did we get here” matters as much as “what’s the current value.”

Common Pitfalls

Interview Questions

• Design the messaging architecture for an order-placed event that needs to trigger billing, inventory updates, notifications, and fraud checks independently, with room to add more consumers later. — Tests whether SNS→SQS (each consumer has its own queue) is the instinct, and whether the candidate considers future extensibility.

• When would you choose Amazon MQ over native SNS/SQS/EventBridge? — Tests understanding of Amazon MQ as a migration/interoperability bridge, not a default choice for greenfield designs.

• You’re building an event-driven system where if any consumer fails, the entire flow must retry. What messaging pattern fits? — Tests whether the candidate understands SQS visibility timeout and retries as the mechanism to achieve guaranteed delivery.

• Explain event sourcing and its trade-offs to someone who’s only built with standard CRUD updates. — Tests whether they understand the benefit (audit trail, replay) and the real cost (query complexity, need for CQRS projections) clearly.