Asynchronous Communication Patterns: Not all events are the same


When integrating systems we often end-up writing asynchronous messaging interfaces for mostly system-to-system communications. This conversation technique is great because it does not require the sender and receiver to stay connected to each other in a session at the same instance in time, is non-blocking and you can make it reliable through message persistence, incremental backoff based delivery retries etc.

It is interesting to note that asynchronous messaging between systems belongs to a wider class of solutions and knowing what to send when can help build better interfaces

First thing to note is that asynchronous communication is not just a thing between system but is also present in customer-to-system interactions (form submissions, emails) as they all seek to be eventually consistent (vs consistent in real-time with synchronous communication, i.e. a telephone call)

Knowing and recognising event communication patterns will take integration solution designers beyond sync and async; it will help them appreciate the nuances in these event communication styles and their advantages/disadvantages

4 Key Patterns …

The 4 key patterns for asynchronous communication are

  1. State change event
  2. Command Event
  3. Event log with data changes
  4. Event with full data snapshot

1. Command or State change events

Business or Domain events which have the new state of a Business entity, e.g “Order Confirmed”, “Claim Submitted”, “Student Enrolled” etc

These are short sharp messages emitted on a state change, for example ClaimSubmitted, OrderCancelled etc or if there is a specific command to be performed. State change events may originate from a Domain Service and require the subscriber to query back on an endpoint to get more information about the change. Command events can originate from clients or domain services and be issued to services which provide a remote asynchronous interface for commands

In real life, this could be you telling your mortgage broker “Hi, we have an extra income source now”, the broker can either call back and ask for more information or they can process your change event to update their model (financial home loan model, for example)

Status change events are emitted with timestamp to order them but the provider may not keep a record of them

State Transitions

The state transitions happen on “saved”, “submitted” etc above and the events from master system to downstream subscribers can trigger reactions (or nothing). These reactions can be part of a Choreography solution

1. An alphabet is added an event is generated “alphabet added”, consumer can come back to query the current state

2. Event with Command

There are times when we find events do not have a business entity state change rather they contain an explicit command

This used be common in enterprise integration solutions with 2-way async communication over queues. One party would issue a request or command with an event over a queue and wait for the service provider to respond back to their queue (p2p) or to a topic

Command events are seen today for point solutions, use cases like Command Bufferring to avoid flooding downstream master with lots of messages

3. Event log with Data Changes

Streaming Data Logs like Kafka

These are time ordered messages similar to state change messages but they contain information about what changed vs something changedThese messages can come with the pointer or index of this change on a message stream so that the subscriber can query the stream to play the rest of the messages from that-point-on

This is similar to as writing play-by-play account of a sports event to our friend on Whatsapp, our friend can scroll back to any point in time and view the state of the game and continue to scroll down to replay the game from that point on

2. An alphabet is added, it is sent with a timestamp and pointer

4. Full data snapshot event

This is a typical data sync where the full snapshot of a model is pushed into a consumer system. It may start out an as event from the provider but the integration services may enrich and update the consumer system to keep the copy-of-data consistent

There are two implementations of this pattern – one where the in-transit message is small and the end adapter enriches the message before submitting it to the consumer and the second is where the entire message is sent on the wire (email)

3. An alphabet is added, the full string is sent

This is similar to an agency sending out emails with the full contract details once a contract is signed

Conclusion

Offline System-to-System and User-to-System communication is different from real-time synchronous communication via web-services and APIs. We write these asynchronous interfaces to de-couple systems but there are hidden patterns which can make our implementations simple or complex based on the requirements

There are 4 key messaging patterns depending on what information is being sent and how the consumer must react to the messages – State Change, Command Events, Change log message and full data-sync message

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s