What is Event Sourcing and How Does It Work?

Due to ever-increasing data volumes, classic SQL database systems reach their limits when writing in online platforms, IoT applications, and many other modern software examples. This is partly due to transactions following the CRUD principle. (CRUD = Create, Read, Update, Delete)

The following list shows some problems and limitations that can arise with CRUD:

• Editing entries in a database table can lead to conflicts and performance problems when many users perform write operations.
• Distribution across multiple instances can lead to inconsistencies, which limits scalability.
• The history of changes can no longer be traced.
• Rolling back changes that have already been made is not possible.
• Transactions across multiple databases, e.g., with different microservices, cannot be consistent. (2-Generals Problem)

However, the listed problems can be solved with a different approach to data storage – Event Sourcing. To explain the concept of “Event Sourcing,” let’s first look at the properties of an event:

1. Events occurred in the past and should therefore be formulated accordingly. For example: account was charged.
2. Events are immutable because they happened in the past. However, they can cancel or modify the effects of a previous event.
3. Events have exactly one creator to which they can be attributed.
4. Events can contain additional information about themselves.

Using the example of account management, we can compare the concept of Event Sourcing with storing data in a table of a conventional RDBMS (Relational Database Management System). Storing information in an RDBMS can happen as in the following figure. The command Deduct 50 from account with ID 152 is converted into an SQL command. The command addresses the accounts table. The value in the balance column in the row with ID: 152 is then reduced from 200 by 50 to 150. This means the history of changes to the account can no longer be traced.

With Event Sourcing, as shown in the next figure, the current state of the application is represented through the previously defined events. First, the command AdjustBalance with value -50 is converted into the event BalanceAdjusted. The numeric value adjust: -50 defines the change in account balance. Furthermore, a sequence number eventSequenceId: 4 must be present to restore the exact order of events. This may only be assigned once for the respective account. This allows all changes to the account balance to be traced and restored at any point in the past.

The Event Sourcing example described above has been implemented with Scala and akka-typed and made available on github: https://github.com/innFactory/akka-event-sourcing-example

The example contains an actor system. In this actor system, a RootActor creates an AccountSupervisor actor and a CommandBot actor. The AccountSupervisor in turn creates the actors of the individual accounts that represent the individual accounts. The CommandBot sends a random number of commands to the accounts. The commands are processed into events and saved in the event history of the respective actor. The state of the actor, or in other words the current account balance, changes according to the events.

In a real application, the events are not saved in the actors themselves but in a database as a journal. NoSQL databases like Cassandra, DynamoDB, or Google Datastore are particularly suitable for this. This allows the actor’s state to be restored even after an error or crash. Even if there are dependencies on other systems, the system will reach a consistent state (= eventually consistent). The recovery process can be accelerated by storing snapshots of “states” in addition to the events in the journal. This means that when recovering, the actor receives an initial state via a snapshot and only processes the events that occurred after this snapshot.

Since Event Sourcing only writes new data in the form of events and does not modify or delete old data, the events can be distributed across a large number of servers. However, one disadvantage is the read side. This problem can be solved in Event Sourcing using Command-Query-Responsibility-Segregation and separate Read-Views. Event Sourcing is often read in connection with CQRS. However, it is also possible to use only Event Sourcing. For Java and Scala applications, the framework akka-persistence exists for this type of architecture.

In the second part, read about how to accelerate the read side of our example through CQRS.