Me and a couple developers are going to be building a POC of a system in the next couple weeks, and I'm already trying to figure out the best way to build the actual system as soon as we're done with the POC and we've decided to move forward with it.
Rough explanation of the system: a root piece the system takes in data from a data source by listening to HTTP requests which the source sends whenever new data is available. The frequency could be hundreds of requests per second. The piece of our system then parses the data to find events that we're interested in. The "events we're interested in" will be defined through our front-end applications, which we have two of.
Both of the front-end applications are interested in same data events, but use them for a different purpose. Essentially the root piece of the system just parses and simplifies the data, and serves the simplified data to the front-end applications. It is essential that the data flow is very close to real-time (like 50ms delay between the root-piece and the front-end application is alright). The front-end applications display the data flow, manage the events the applications are interested in AND they are also used to configure the endpoints of the events.
The endpoints could be for example a device like a MIDI device or another HTTP server.
The root server of the system in the beginning only listens to one kind of data source, but in the future we might need to support some other data source, where the data might flow through websockets, UDP or the same way via HTTP POST requests, but the data model will be different. We most likely don't need to listen to multiple data sources at once.
These systems will run locally on a user's device, be it a Linux/PC/Mac. The systems would have some kind of a root system running in Cloud, where users can download updates to their locally running system and handle their licenses etc.
My initial thought of the architecture would be roughly this:
- A Kotlin backend (maybe with Ktor or Spring Boot), which listens to the data source, parses it and sends events parsed from the data to any of the front-end applications
- A NX Angular monorepo, which features those two front-end applications as separate apps within the monorepo, and separate NestJS back-ends for both which are used as a "proxy" and data storage for everything between the front-ends and the Kotlin backend
- On top of this, we'd have library/libraries in the same monorepo, to share components, datamodels etc.
- Angular front-end and a Kotlin backend (also Ktor or Spring Boot) in the Cloud to handle the updates etc. to users' systems
This is my first time actually taking some time to properly think about the system architecture, I've only a couple years of experience as a full-stack developer so I'd love to hear some thoughts from more experienced developers/system architects/whoever!