Asynchronous API Standardization. Asyncapi for Faster and Better Digital Communication

The rise of Event Driven Architecture (EDA)

Modern enterprises are software organizations that realize their business processes and interactions with their partners and customers using hundreds of APIs.

That’s the natural technological consequence of the total automation of business activities. APIs talk to each other to perform business functions thousands of times faster than humans could do.

Therefore designing and implementing this communication effectively is crucial for the success of digital transformation.

→ Explore Composable architecture as a key for composable enterprises in the digital future

API to API – synchronous vs. asynchronous

The key to scalability and flexibility of the communication between APIs is asynchronous API.

What does it mean?

In case of synchronous communication, one party waits for the response of the other.

For instance, an API calls the bank to ask if the customer has enough money on their account for the transaction and waits for the response (milliseconds, seconds, or up to minutes).

What is the problem? It blocks the flow of the application, thus, the rise in popularity of asynchronous communication.

In an asynchronous  example, an API calls the bank to ask the same thing, but instead of waiting for the answer it immediately receives a token, like an ID of the question, without waiting for the actual result. Then this ID can be used to find out the actual account balance information and can be used in a so called callback,  when an external API calls back our API to inform about the result.

Human example of asynchronous

It’s kind of obvious for the developers, but for the non-developers out there I will provide some human examples.

Option one (synchronous):

You call the bank and ask to open a certificate of deposit, but the person who can do it is busy, so you have to wait for them about 10 minutes without hanging up. You listen to ugly automatic melodies in the meanwhile and it’s really hard to focus on something else during this time.

Option two (asynchronous):

The bank will call you back in 10 minutes and you can do other things in the meantime.

This analogy is the same in the API, which defines the conversations between different applications inside the organization and outside.

Another popular example is video chat vs. email. In a video chat we naturally expect someone to wait for us to speak what’s on our mind or to show up on the screen (live) and respond immediately (synchronously). In the case of email, we send an email and focus on other activities, and when the response arrives we are notified.

→ Read more about Human digital twins. What are they and Why are they?

Event Driven Architecture (EDA) in one minute

This architectural paradigm is focused on events, as the name suggests. Everything that happens in the business processes and transactions may be triggered by events or may trigger events in which other systems (internal and external) may be interested in.

Event driven architecture means another layer of decoupling the systems from each other, enabling enormous flexibility and agility. It’s another dimension, not replacing the existing ideas but taking it another step further.

→ Explore why Generic API or Back-End for Front-End? You can have both. 

In these times of total domination by microservices architecture, the components and APIs are getting smaller and smaller, representing a relatively small functionality which can be exposed to other systems and external partners.

The messaging architecture is critical to ensure fast, reliable and proper flow of the messages, so as to be able to subscribe to interesting topics and events, and react to them in a timely fashion.

Event driven example

For example, an HR system registers a new employee that was just hired.

In EDA, it would generate the NewEmployeeHired event with the information about the new employee (position name, department, boss, date when they start their job, etc.).

The HR system doesn’t know, and never has to know, which other systems might be interested in this event. It’s doing its “pure HR stuff” and letting other systems focus on their business processes.

And yes, there are other systems who are interested in this event.

For instance, the IT security management system creates an inactive account for the user, assigns them the right access groups, and creates a one-time-password.

Asset management systems may assign a laptop for them or generate a request to the purchasing department system to buy a new laptop with a required configuration, which is associated with the position and department of the new hire.

This was an example of how a message informs others about something. The other type of event is commands, when one API tells another API to do something.

So as you can see, we have a lot of flexibility, and there might be zero or hundreds of other APIs interested in our event. And, the sender does not even have to know about their existence, nor functionality.

Usually, publishers and subscribers are not connected directly to each other but via a broker, that has multiple channels (topics, etc. depending on the actual implementation) which allow the communication.

What does AsyncAPI deliver?

Lowered cost

Asynchronous APIs have been popular for almost twenty years in the architecture of digital solutions for businesses.

What number of connections are we talking about? It’s good to understand that the number of interactions between the components is scaled with the square number of the systems.

The actual formula is n*(n-1)/2 and enterprises have 800-1000 APIs on average.

In the case of just 800, we can have 319,600 possible interactions. This is helped a lot by the message brokers, but only in the transport layer, and many other problems still remain.

With a lack of API call standardization, it’s much worse. There’s an additional cost of remembering and implementing multiple types of interactions; API 123 uses the pattern A, while API 2345234 uses the pattern B, etc. It adds an additional complexity and increases cost.

Lack of standardization always means additional costs for figuring out how to talk to the other API and learning how to talk to the other API in a different way than the previous API. Some of this cost is directly visible, but the majority of this cost is hidden in uncontrolled complexity, surprises, bugs, and strange production issues.

API designers and developers should focus more on the business side of things and not on figuring out how to talk to this or that API, or why it’s another or even another different way, which then requires another then another approach. It’s a waste of time and resources, plus it demotivates and frustrates the team and can even end relationships with customers.

Message formats

All the exchanged messages should have well defined structure. The old way, in the old times, was to use the SOAP family of protocols to define end points and messages.

The message format is like a language all the participants of the conversation are expecting and understanding. The way the format is defined is unified by the standard. Multiple schemas are supported (JSON Schema, Avro) as well.

Making message receivers happy

Event driven architecture promises the flexibility and freedom of not knowing what API is on the receiving end of the communication.

However, in reality there’s another team responsible for using your APIs for their purposes, and without proper definitions and implementation they will have a lot of trouble working with your APIs.

Multiple protocols – protocol agnostic

There are many different protocols for exchanging data between different APIs: WebSocket, Kafka, AMQP, MQTT, etc… Understanding the differences between WebSockets vs HTTP is crucial for developers choosing the most appropriate protocol for their event-driven applications.

The AsyncAPI standard is expected to be applicable no matter what protocol is in use.

No protocol is enforced or recommended.

The AsyncAPI standard 2.0 introduced the concept of binding the definitions with the protocols, in order to preserve the independence of the endpoint and keep a message definition from the various protocols.

Tools for developers and API designers

Ready to use tooling that helps with practical implementation is essential for the success of any standard, and it’s the case with AsyncAPI. There are tools (code generators, documentation generators, and designers) for the most popular platforms that make it easier for developers to implement the standard in their APIs.

AsyncAPI documents

These are YAML or JSON files which contain standardized definitions of channels, subscribers, messages and payloads. Payload definition may include various data types, regexp, minimum values, or maximum values.

A server’s section defines endpoints and protocol, by telling the receiver how to connect.

The other section describes security: how to authenticate with different security themes and standards.

Example of a file:(https://www.asyncapi.com/docs/tutorials/streetlights/)

Future of AsyncAPI

What about an OpenAPI?

OpenAPI (Swagger) and GraphQL were already here. So why AsyncAPI?

Open API is focused on REST APIs, while AsyncAPI is more about asynchronous APIs in more complex integration scenarios. AsyncAPI creators like to think about their AsyncAPI specification as one of the founding blocks for Event Driven Architecture (EDA), for which OpenAPI was not enough.

There are similarities between them, but also key differences like the multi-protocol focus of AsyncAPI and the native support for operations such as publish and subscribe.

(from https://www.asyncapi.com/docs/getting-started/coming-from-openapi/ )

Support

AsyncAPI is supported by more and more companies who are selecting it as their preferred way to define and manage asynchronous APIs in their organizations.

AsyncAPI works great with the most popular open source messaging platform, which is Kafka, and this is one of the reasons for its increasing popularity.

Being a transport agnostic, it is also used in technology migrations from one messaging platform to another, as it helps to preserve the core contract definitions.

Will it become a worldwide standard that is used and accepted by the vast majority of organizations? Only time will tell.

→ There is more on the Trends & Technology 2021.

If you want to accelerate your API-enabled organization and modernize your API management, design and implementation, then we at Avenga are more than glad to help.