The Saga design pattern is a way to manage data consistency across microservices in distributed transaction scenarios. A saga is a sequence of transactions that updates each service and publishes a message or event to trigger the next transaction step. If a step fails, the saga executes compensating transactions that counteract the preceding transactions.
Context and problem
A transaction is a single unit of logic or work, sometimes made up of multiple operations. Within a transaction, an event is a state change that occurs to an entity, and a command encapsulates all information needed to perform an action or trigger a later event.
Transactions must be atomic, consistent, isolated, and durable (ACID). Transactions within a single service are ACID, but cross-service data consistency requires a cross-service transaction management strategy.
In multiservices architectures:
- Atomicity is an indivisible and irreducible set of operations that must all occur or none occur.
- Consistency means the transaction brings the data only from one valid state to another valid state.
- Isolation guarantees that concurrent transactions produce the same data state that sequentially executed transactions would have produced.
- Durability ensures that committed transactions remain committed even in case of system failure or power outage.
A database-per-microservice model provides many benefits for microservices architectures. Encapsulating domain data lets each service use its best data store type and schema, scale its own data store as necessary, and be insulated from other services' failures. However, ensuring data consistency across service-specific databases poses challenges.
Distributed transactions like the two-phase commit (2PC) protocol require all participants in a transaction to commit or roll back before the transaction can proceed. However some participant implementations, such as NoSQL databases and message brokering, don't support this model.
Another distributed transaction limitation is interprocess communication (IPC) synchronicity and availability. Operating system-provided IPC allows separate processes to share data. For distributed transactions to commit, all participating services must be available, potentially reducing overall system availability. Architectural implementations with IPC or transaction limitations are candidates for the Saga pattern.
Solution
The Saga pattern provides transaction management using a sequence of local transactions. A local transaction is the atomic work effort performed by a saga participant. Each local transaction updates the database and publishes a message or event to trigger the next local transaction in the saga. If a local transaction fails, the saga executes a series of compensating transactions that undo the changes that were made by the preceding local transactions.
In Saga patterns:
- Compensable transactions are transactions that can potentially be reversed by processing another transaction with the opposite effect.
- A pivot transaction is the go/no-go point in a saga. If the pivot transaction commits, the saga runs until completion. A pivot transaction can be a transaction that is neither compensable nor retryable, or it can be the last compensable transaction or the first retryable transaction in the saga.
- Retryable transactions are transactions that follow the pivot transaction and are guaranteed to succeed.
There are two common saga implementation approaches, choreography and orchestration. Each approach has its own set of challenges and technologies to coordinate the workflow.
Choreography
Choreography is a way to coordinate sagas where participants exchange events without a centralized point of control. With choreography, each local transaction publishes domain events that trigger local transactions in other services.
Benefits
- Good for simple workflows that require few participants and don't need a coordination logic.
- Doesn't require additional service implementation and maintenance.
- Doesn't introduce a single point of failure, since the responsibilities are distributed across the saga participants.
Drawbacks
- Workflow can become confusing when adding new steps, as it's difficult to track which saga participants listen to which commands.
- There's a risk of cyclic dependency between saga participants because they have to consume each other's commands.
- Integration testing is difficult because all services must be running to simulate a transaction.
Orchestration
Orchestration is a way to coordinate sagas where a centralized controller tells the saga participants what local transactions to execute. The saga orchestrator handles all the transactions and tells the participants which operation to perform based on events. The orchestrator executes saga requests, stores and interprets the states of each task, and handles failure recovery with compensating transactions.
Benefits
- Good for complex workflows involving many participants or new participants added over time.
- Suitable when there is control over every participant in the process, and control over the flow of activities.
- Doesn't introduce cyclical dependencies, because the orchestrator unilaterally depends on the saga participants.
- Saga participants don't need to know about commands for other participants. Clear separation of concerns simplifies business logic.
Drawbacks
- Additional design complexity requires an implementation of a coordination logic.
- There's an additional point of failure, because the orchestrator manages the complete workflow.
Issues and considerations
Consider the following points when implementing the Saga pattern:
- The Saga pattern may initially be challenging, as it requires a new way of thinking on how to coordinate a transaction and maintain data consistency for a business process spanning multiple microservices.
- The Saga pattern is particularly hard to debug, and the complexity grows as participants increase.
- Data can't be rolled back, because saga participants commit changes to their local databases.
- The implementation must be capable of handling a set of potential transient failures, and provide idempotence for reducing side-effects and ensuring data consistency. Idempotence means that the same operation can be repeated multiple times without changing the initial result. For more information, see the guidance on ensuring idempotence when processing messages and updating state together.
- It's best to implement observability to monitor and track the saga workflow.
- The lack of participant data isolation imposes durability challenges. The saga implementation must include countermeasures to reduce anomalies.
The following anomalies can happen without proper measures:
- Lost updates, when one saga writes without reading changes made by another saga.
- Dirty reads, when a transaction or a saga reads updates made by a saga that has not yet completed those updates.
- Fuzzy/nonrepeatable reads, when different saga steps read different data because a data update occurs between the reads.
Suggested countermeasures to reduce or prevent anomalies include:
- Semantic lock, an application-level lock where a saga's compensable transaction uses a semaphore to indicate an update is in progress.
- Commutative updates that can be executed in any order and produce the same result.
- Pessimistic view: It's possible for one saga to read dirty data, while another saga is running a compensable transaction to roll back the operation. Pessimistic view reorders the saga so the underlying data updates in a retryable transaction, which eliminates the possibility of a dirty read.
- Reread value verifies that data is unchanged, and then updates the record. If the record has changed, the steps abort and the saga may restart.
- A version file records the operations on a record as they arrive, and then executes them in the correct order.
- By value uses each request's business risk to dynamically select the concurrency mechanism. Low-risk requests favor sagas, while high-risk requests favor distributed transactions.
When to use this pattern
Use the Saga pattern when you need to:
- Ensure data consistency in a distributed system without tight coupling.
- Roll back or compensate if one of the operations in the sequence fails.
The Saga pattern is less suitable for:
- Tightly coupled transactions.
- Compensating transactions that occur in earlier participants.
- Cyclic dependencies.
Example
Orchestration-based Saga on Serverless is a saga implementation reference using the orchestration approach that simulates a money transfer scenario with successful and failed workflows.
Next steps
- Distributed data
- Richardson, Chris. 2018: Microservices Patterns. Manning Publications.
The following patterns might also be useful when implementing this pattern:
- Choreography has each component of the system participate in the decision-making process about the workflow of a business transaction, instead of relying on a central point of control.
- Compensating transactions undo work performed by a series of steps, and eventually define a consistent operation if one or more steps fail. Cloud-hosted applications that implement complex business processes and workflows often follow this eventual consistency model.
- Retry lets an application handle transient failures when it tries to connect to a service or network resource, by transparently retrying the failed operation. Retry can improve the stability of the application.
- Circuit breaker handles faults that take a variable amount of time to recover from, when connecting to a remote service or resource. Circuit breaker can improve the stability and resiliency of an application.
- Health endpoint monitoring implements functional checks in an application that external tools can access through exposed endpoints at regular intervals. Health endpoint monitoring can help verify that applications and services are performing correctly.
FAQs
What are the drawbacks of Saga pattern? ›
Disadvantages of the Saga pattern
The Saga pattern is difficult to debug, especially when many microservices are involved. Also, the event messages could become difficult to maintain if the system gets complex. Another disadvantage of the Saga pattern is it does not have read isolation.
The saga pattern is a failure management pattern that helps establish consistency in distributed applications, and coordinates transactions between multiple microservices to maintain data consistency.
What is the full form of Saga pattern? ›The term saga refers to Long Lived Transactions (LLT). Let's assume it as abbreviation (which is not preferable in some way). So, in terms of microservices, you can think of Segregated Access of Global Atomicity.
What are the two types of Saga pattern? ›There are two common saga implementation approaches, choreography and orchestration. Each approach has its own set of challenges and technologies to coordinate the workflow.
What are the pros and cons of Saga pattern? ›The main benefit of the Saga Pattern is that it helps maintain data consistency across multiple services without tight coupling. This is an extremely important aspect for a microservices architecture. However, the main disadvantage of the Saga Pattern is the apparent complexity from a programming point of view.
Why Saga is better than 2PC? ›2PC works as a single commit and aims to perform ACID transactions on distributed systems. It is used wherever strong consistency is important. On the other hand, SAGA works sequentially, not as a single commit. Each operation gets committed before the subsequent one, and this makes the data eventually consistent.
How to use Saga pattern in microservices? ›To use the SAGA pattern in a distributed transaction, we can use a compensating transaction to undo the changes made by each microservice in the event of an error. This way, we can ensure that the overall transaction is either completed successfully, or is rolled back to its initial state.
What is the difference between Saga and Cqrs? ›The SAGA pattern is a failure management pattern that brings consistency in distributed applications and coordinates transactions between various microservices to maintain consistency. Command and Query Responsibility Segregation or CQRS is a pattern that separates read and update operations for a data store.
What are examples of saga? ›When your friend tells you every detail of how she tripped over a rock, broke her ankle, and then got into a car accident on the way to the hospital, she is sharing a long, involved story known as a saga. The word saga has its origins in the Middle Ages.
How is Saga pattern implemented? ›What is the saga pattern? A saga is a sequence of local transactions. Each local transaction updates the database and triggers the next local transaction. If a local transaction fails, the saga executes a series of compensating transactions that undo the changes that were made by the preceding local transactions.
How does Saga work? ›
Saga is an enchantment type introduced in Dominaria. Each Saga tells the story of a key event from the past as it unfolds during each of your turns. Each separate step in the story is called a chapter, and is marked by a roman numeral (I, II, III, etc.). Saga cards are historic.
What are the 3 types of pattern? ›- Shape Pattern.
- Letter Pattern.
- Number Pattern.
- Behavioral,
- Creational, and.
- Structural.
Saga pattern can help us maintain the data consistency among the microservices architecture efficiently. Event Sourcing ensures that all changes to business entity's state are stored as a sequence of events.
What is the difference between saga orchestration and Saga choreography? ›Choreography - where microservices work independently but coordinate with each other using cues or events. The method of control of the saga or workflow is determined by a predefined set of cues or events. Orchestration - where microservices are controlled by an orchestrator or conductor.
Why should I use saga? ›The saga essentially acts as a separate thread to your application, listening for specific actions from your main application to perform complex asynchronous tasks and updating your application's state once it is completed.
Which is best saga or thunk? ›The benefit of Redux-Saga in comparison to Redux-Thunk is that you can more easily test your asynchronous data flow. Redux-Thunk, however, is great for small projects and for developers who just entered into the React ecosystem. The thunks' logic is all contained inside of the function.
Is saga a middleware? ›Redux-Saga is one of the popular middleware libraries for Redux. It makes working with asynchronous operations and side effects a lot easier. Unlike other libraries like Redux Thunk, Redux-Saga uses ES6 generators.
How do you handle errors in Saga? ›We can catch errors inside the Saga using the familiar try/catch syntax. In this case, we're passing the throw method a fake error. This will cause the Generator to break the current flow and execute the catch block. Of course, you're not forced to handle your API errors inside try / catch blocks.
What is 2PC vs Saga pattern? ›2PC: Save Customer for every received Invoice request, while both are managed by 2 different systems. Sagas: Book a flight itinerary consisting of several connecting flights, while each individual flight is operated by different airlines.
How do I call API using saga? ›
- Dispatch a REQUEST action to signal the start of the request.
- Calling a function from api. js to make the actual HTTP request.
- Get the data from the request.
- Dispatch a SUCESS action with this data to signal a sucess of the request.
- Dispatch a FAILURE action if anything went wrong during this process.
Example: Orchestration-based saga
The saga orchestrator creates an Order in the PENDING state. It then sends a Reserve Credit command to the Customer Service. The Customer Service attempts to reserve credit. It then sends back a reply message indicating the outcome.
The two are compatible patterns that address different problems, Sagas handle workflow processes where as event sourcing addresses how state is stored. Sagas provide a mechanism for handling a multi-step process and rolling back should steps fail (like a workflow).
What is a saga vs process? ›This is what makes Saga different from Process Manager.
The Saga itself has no state. A Process Manager can be modelled as a state machine. It makes decisions based not only on incoming events but also the current state of the process.
And they were aptly named: saga traces back to an Old Norse root that means "tale." The English word first referred only to those original Icelandic stories, but saga later broadened to cover other narratives reminiscent of those, and the word was eventually further generalized to cover any long, complicated scenario.
What are the features of saga? ›Features of SAGA include analytics, data import/export, notifications, graphical planning, configurable workflows and more. It allows professionals to manage raster and vector data formats and view workspace objects in a tree-like structure.
How do you write a good saga? ›- Step 1: Map out the plot. The first thing you want to do is solidify the ideas you have for your series' plot. ...
- Step 2: Think about the structure. You've now mapped out the plot of your entire story as best you can. ...
- Step 3: Get to know your characters. ...
- Step 4: Work on your setting. ...
- Step 5: Start writing!
In redux-saga , Sagas are implemented using Generator functions. To express the Saga logic, we yield plain JavaScript Objects from the Generator. We call those Objects Effects. An Effect is an object that contains some information to be interpreted by the middleware.
How many are in a saga? ›The term saga is used to classify very long stories or novels that may also cover long periods of time. It may take any number of books to complete a saga, or it could be one long book that involves a very long story.
What is yield in saga? ›Sagas are implemented as Generator functions that yield objects to the redux-saga middleware. The yielded objects are a kind of instruction to be interpreted by the middleware. When a Promise is yielded to the middleware, the middleware will suspend the Saga until the Promise completes.
What are the drawbacks of self registration pattern? ›
- Couples the service to the Service Registry.
- You must implement service registration logic in each programming language/framework that you use to write your services, e.g. NodeJS/JavaScript, Java/Scala, etc.
Another drawback of the publish/subscribe pattern is that it is difficult to gauge the health of subscribers. The publisher does not have perfect knowledge of the status of the systems listening to the messages. For instance, publish/subscribe is commonly used for logging systems.
Which is the drawback of multiple services per host pattern? ›The drawbacks of this approach include: Risk of conflicting resource requirements. Risk of conflicting dependency versions. Difficult to limit the resources consumed by a service instance.
What is the difference between saga orchestration and saga choreography? ›Choreography - where microservices work independently but coordinate with each other using cues or events. The method of control of the saga or workflow is determined by a predefined set of cues or events. Orchestration - where microservices are controlled by an orchestrator or conductor.
What is heartbeat between microservices? ›The heartbeat Microservice is designed to support multiple instances of HB Microservice to run simultaneously. The first instance of the HB Microservice would assume the role of active instance, and instances that started running later would become inactive instances.
Which pattern requires more network hops? ›More network hops are required than when using the Client Side Discovery Pattern.
What is the difference between polling and publish subscribe? ›Polling, in this context, means using a RESTful webservice to GET updates for each user. In contrast, PubSub (Publish/Subscribe) is an architectural approach that uses an asynchronous message passing protocol where publishers are decoupled from any subscribers.
What is the difference between multicast and publish subscribe? ›Pub/Sub has publishers delivering a separate copy of each message to each subscriber which means that the opportunity to guarantee delivery exists. Fan Out has a single queue pushing to all listening clients. Multicast just spams out data and if someone is listening then fine, if not, it doesn't matter.
What is the difference between publish subscribe and push pull? ›The difference is that a PUB socket sends the same message to all subscribers, whereas PUSH does a round-robin amongst all its connected PULL sockets. In your example, if you send just a single message from the root, then all the subscribers will receive it (barring slow subscribers, etc.) but only 1 worker.
Can we deploy multiple microservices on same server? ›Multiple instances of microservice per server
It can run one or more instances of the microservice on a single server. Multiple instances of the microservice can run in the same process or in a group of different processes.
Which design pattern is used in microservices? ›
The Client-side Discovery and Server-side Discovery patterns are used to route requests for a client to an available service instance in a microservice architecture.
Which Saga pattern is best? ›Orchestration Saga Pattern
The orchestrator microservices execute saga transaction and manage them in centralized way and if one of the step is failed, then executes rollback steps with compensating transactions. Orchestration way is good for complex workflows which includes lots of steps.
There are two approaches to implement the Saga pattern: choreography and orchestration.
What is the difference between event sourcing and saga? ›Saga pattern can help us maintain the data consistency among the microservices architecture efficiently. Event Sourcing ensures that all changes to business entity's state are stored as a sequence of events.