Kafka + WebSockets + Angular: event-driven microservices all the way to the frontend

In the the initial post of the Event-driven microservices with Kafka series (see here or here), I talked about the advantages of using event-driven communication and Kafka to implement stateful microservices instead of the standard stateless RESTful ones.
I also presented the architecture and the source code of a related proof of concept application.

In this post, I would like to show how to extend the asynchronous event-driven communication all the way from Kafka to the Web frontend passing through the Java backend.
Hence, in the first post of this series, we got rid of HTTP as the communication protocol among microservices in the backend, and now we are also replacing it (with WebSockets) as the communication protocol between the frontend and the backend.

Ok, but why would you do that?

Because it provides a better experience to the end user!.
Using WebSockets you can build legit real-time user interfaces, the updates are pushed immediately from the server to the client, no need for the user to click any button nor to refresh the browser.

Wait, but we can do the same using good old HTTP, can't we?

Well... HTTP was not designed to provide real-time communication, but since people really wanted it, a number of workaround techniques and technologies have been create on top of it, such as:

• Long polling
• Short polling
• Server-Sent Events

Those are little more than hacks, IMHO.
WebSockets, on the other hand, provide full-duplex bidirectional communication, which means that information can flow from the client to the server and also in the opposite direction simultaneously, it cannot get more real-time than that (that's what I meant when I said 'legit real-time applications' before).

HTTP vs WebSocket (source)

The example

I added a new service to the same PoC application which I used in the previous post and also an Angular web frontend.

Ok, but what does this new service do?

This service allows the frontend user to receive information about the current balance of any account of her/his choice as well as live updates when the balance changes.

Also, information about any transactions related to that currently selected account (i.e., any amount of money credited or debited) past or live, is displayed to the users.

Architecture

The new Transfers WebSockets service main components are:

• Two Kafka consumers (one for each topic) to retrieve messages from the Kafka cluster
• Two Kafka Streams local stores to retrieve the latest data associated with a given key (id)
• A custom local store implemented using a simple Map to store the list of transactions for a given account.
• Two dispatchers which keep track of what WebSockets sessions (i.e., frontend clients) are registered to receive live balance updates or live transfers data related to which account and then dispatch the updates accordingly.

And these are the main components of the Angular frontend:

• A WebSockets service which handles the connection lifecycle and dispatches the received messages to the corresponding RxJS Subject. Please note that different types of JSON messages (requests, responses and updates) flow through the only WebSockets connection.
• A form which passes the user input (i.e., the id of the chosen account) to the WebSockets service so it can be sent to the remote server.
• An Angular component to display the account balance which gets asynchronously notified of balance changes (updates) through an RxJS Observer which is subscribed to the corresponding RxJS Subject in the previously mentioned WebSockets service, and then re-renders the value accordingly.
• Another Angular component which similarly as the previous one, gets asynchronously notified of any new transaction related to the selected account and adds it to the list of displayed transaction data.

This is a diagram depicting the PoC application components and several frontend instances:

See it in action

Check out this Youtube video for a demo, where a test process which continuously generates Kafka messages representing random money transfers is used to feed the application.
And on the Web UI the user can see how the transfers pops up as they happen and the balance of the related account changes accordingly, all in real time.

Final Thoughts

HTTP has had a huge success, it made possible the WWW as we know it today.
It has been so successful that it has spread to the backend (RESTful services and open APIs) where it has been the king for more than a decade already.
However, technology evolves, paradigms come and go and for some use cases other tools and designs may be the right choice.
I do not know how software applications will look like in the future, all I can do is guess.
Event-driven microservices look very powerful and implementing them with Kafka and adding WebSockets to the mix seems promising.
In fact, some companies are able to see beyond REST APIs and are starting to offer WebSockets APIs to their clients for faster streaming access to data (example).
And there are other alternative ways to implement event-driven microservices, for example, recently I learnt about Uber Cadence but that's for a future post 😃