State-of-the-Art Distributed Systems with CoralMQ

In this article we introduce the big picture of CoralMQ, a full-fledged, ultra-low-latency, high-reliability, software-based middleware for the development of distributed systems based on asynchronous messages. We discuss CoralMQ’s main parts and how it uses a sophisticated and low-latency protocol to distribute messages across nodes through reliable UDP broadcast/multicast.

Fundamentals

  • What is a distributed system?
    It is an integrated system that spans multiple machines, clients or nodes which execute their tasks in parallel in a non-disruptive way. A distributed system should be robust enough to continue to operate when one or more nodes fail, stop working, lag or are taken down for maintenance. Each machine can run multiple nodes, not just one. Each node communicates with each other by sending messages to the queue (i.e. broker). Nodes can produce and consume messages.
  • What is a Messaging Queue or MQ?
    It is a message-oriented middleware that enables the development of a distributed system by using asynchronous messages for inter-node communication. A traditional MQ approach for message distribution is the publish-subscribe pattern. CoralMQ uses a different all-messages-to-all-nodes approach through a reliable UDP protocol as we explain below.
  • What are distributed systems built on top of a MQ good at?
    They are great at implementing applications with a lot of moving parts (i.e. nodes) which need real-time monitoring and real-time reaction capabilities. To summarize they provide: parallelism (nodes can truly run in parallel), tight integration (all nodes see the same messages in the same order), decoupling (nodes can evolve independently), failover/redundancy (when a node fails, another one can be running and building state to take over immediately), scalability/load balancing (just add more nodes), elasticity (nodes can lag during activity peaks without affecting the system as a whole) and resiliency (nodes can fail / stop working without taking the whole system down).
  • What is the advantage of CoralMQ?
    CoralMQ is a state-of-the-art implementation of a messaging queue for ultra-low-latency distributed systems that require a high degree of reliability. It has all the Coral Blocks’ advantages: high performance, low latency, low variance, zero garbage and simplicity. As you will see in subsequent articles, CoralMQ is extremely easy to develop, configure, deploy, monitor and maintain.
  • What are some examples of systems that can be built on top of CoralMQ?
    Electronic exchanges (ECNs), trading platforms, banking systems, automated online advertising systems, defense/military systems, credit card systems or any distributed system with a large number of transactions that requires all the advantages listed above plus ultra low latency and high reliability.

Quick Facts

  • All nodes read all messages in the exact same order, dropping messages they are not interested in
  • All messages are persisted so late-joining nodes can catch up and build the exact same state as other nodes
  • Message broadcasting is done through a reliable UDP protocol: no message is ever lost
  • No single point of failure, from the software down to the hardware infrastructure
  • Each session is automatically archived with all its messages so it can be replayed later for testing, simulation, analysis and auditing
  • High-level, straightforward API makes it easy to write nodes that publish/consume messages
  • As low-latency as it can be through UDP broadcast/multicast
  • Zero garbage created per message – no gc overhead
  • Remote administration, backup, logging, TCP bridge and more

Node Example


package com.coralblocks.coralmq.node;

import com.coralblocks.coralmq.message.Message;
import com.coralblocks.coralmq.mq.Node;
import com.coralblocks.coralreactor.nio.NioReactor;
import com.coralblocks.coralreactor.util.Configuration;

public class SampleNode extends Node {
	
	private final static int PERIOD = 3000; // 3 seconds...
	
	private long time;

	public SampleNode(NioReactor nio, String name, Configuration config) {
	    super(nio, name, config);
    }
	
	@Override
	protected void handleActivated() {
		// this method is called when the node becomes active
		sendCommand();
	}
	
	@Override
	protected void handleEventTimeout(long now, int period) {
		// this method is triggered by the event timeout you are setting in the handleMessage method
		sendCommand();
	}
	
	private void sendCommand() {
		time = System.nanoTime();
		sendCommand("TIME");
	}

	@Override
    protected void handleMessage(boolean isMine, Message msg) {
		
		if (!isMine || isRewinding()) return; // not interested, drop it
		
		System.out.println("Took " + (System.nanoTime() - time) + " nanoseconds to see my message in the bus");
		
		setEventTimeout(PERIOD); // set a trigger to send the command again after 3 seconds (goes like that forever)
    }
}

For the remaining of this article and/or more information about CoralMQ please contact Coral Blocks!