From Data Modeling to Effective Message Exchange
Event-driven systems efficiently handle streams of events or data by processing them as they occur. An event, defined as any observable occurrence at a specific point in time forms the core of these systems1. Due to the decoupled nature of event-driven architectures, effective communication of the events content and intent becomes crucial, relying on both proper semantics and syntax.
Effective communication, whether written or spoken, requires careful attention to syntax and semantics2. Messages with proper syntax are not only easier to read or listen to but also pave the way for clear semantics, enabling an accurate understanding of the intended meaning and fostering successful communication between sources and receivers.
Photo by Mathyas Kurmann on Unsplash
Data Modeling
From ancient times, mapping has served as a means for people to communicate across different languages. In the realm of software engineering, this mapping is evident in various tools and techniques such as Model Driven Engineering ( MDE) 3, Unified Modeling Language (UML) 4, and Domain-Driven Engineering (DDD) 5.
In Geographical Mapping, a map symbolically represents selected characteristics of a place, reflecting the real world 6. Within Domain-Driven Design, the Domain Model organizes knowledge and data around the problem context, capturing the concepts of the problem domain 5. When creating a message structure, it is crucial to provide a comprehensive description to prevent ambiguity and duplication. The message should exhibit a clear knowledge structure ( syntax) and strive to convey the intended meaning (semantic). However, similar to mapping, including too many details in the message can increase its size, diminishing readability while augmenting elements that aid in understanding different locations on the map.
Message Exchange
Exchanging messages between people usually is not an easy feat. Let’s forget about security and Byzantine problems for a minute. Assume two persons speak English, and they want to send a letter via post. What do you need?
- Address and person to whom you are sending the message
- Address and person who is sending the message and expects an answer
- What else do you need? The letter content, easy right?
Unfortunately, no. You need clear language and grammar at a level both persons understand. How about if you are sending specific content like a measure, temperature, time, and location? You need to specify the metric system, time zone, and coordinate system.
Now, let’s say we have a good enough message system for 100 persons, sending the same message. After a while, we are sending new data, such as distance. Now you have to make sure everyone is clear with all the rules specified before and what has changed.
Usually, between people, it’s a manageable process. But if you include machines in the process of exchanging messages, and it’s not 100 persons but millions of devices, this becomes an unmanaged problem, given the variability of the world and the models used for communication.
Tackling the Issues
Extensive research has been done to address this challenge, offering solutions ranging from Object-Oriented typed languages to the frameworks of Service-Oriented Architectures and Microservices. In today’s landscape, these solutions commonly use versatile exchange formats, including Extensible Markup Language (XML), JavaScript Object Notation (JSON), and YAML.
These data formats are pivotal, acting as a bridge between human understanding and machine processing 7. While XML excels in enforcing precise rules8, its verbosity prompts consideration of more streamlined alternatives such as JSON and YAML.
A scenario where everyone on a team speaks the same language, a language shared by both business and tech experts. This is precisely what DDD advocates for—a shared vocabulary or “ubiquitous language.” Moreover, DDD recommends the use of bounded contexts, breaking down the project into manageable segments. This minimizes confusion and prevents teams from being inundated with unnecessary intricacies.
The significance of this approach becomes apparent when working collaboratively. A shared understanding and organized structure make the project more manageable, akin to having a well-defined map for the entire journey. DDD facilitates consensus on how components should function and their desired appearance—a fundamental aspect of ensuring the success of any project.
To navigate this process effectively, it’s imperative to maintain synchronization across diverse teams collaborating on the project. Establishing agreement on both semantics and syntax becomes the foundation of a successful implementation.
Conclusion
In the world of event-driven systems, effective communication plays a crucial role. Events, the main actors in these systems, require a blend of clear language and understanding to function well. We’ve explored data modeling, drawing comparisons to ancient map-making tools used in today’s software landscape.
Whether it’s sending letters between people or managing messages among millions of devices, we’ve seen that clear communication faces challenges. Despite advanced tech solutions, the everyday challenge for engineers is navigating the twists and turns of communication in our digital era. Thus, the journey goes on, highlighting that simple and effective communication is the key to success for every engineer.