CoralSequencer’s structured data serialization framework

CoralSequencer uses its own binary and garbage-free serialization framework to read and write its internal messages. For your application messages, you are free to use any serialization library or binary data model you choose. The fact that CoralSequencer is message agnostic gives you total flexibility in that decision. But you can also consider using CoralSequencer’s native serialization framework described in this article.

To define the schema of a message you simply inherit from AbstractProto and define the message data fields. Below a self-explanatory example:

import com.coralblocks.coralsequencer.protocol.AbstractProto;
import com.coralblocks.coralsequencer.protocol.field.*;

public class OrderNew extends AbstractProto {
	
	private static final int SYMBOL_LENGTH = 8;

	public static final char 	TYPE 	= 'O';
	public static final char 	SUBTYPE = 'N';
	
	public final TypeField typeField = new TypeField(this, TYPE);
    public final SubtypeField subtypeField = new SubtypeField(this, SUBTYPE);
	
	public	final	CharsField		symbol = new CharsField(this, SYMBOL_LENGTH);
	public	final	CharField		side = new CharField(this);
	public	final	LongField		size = new LongField(this);
	public 	final	LongField		price = new LongField(this);
	public	final	LongField		myTimestamp = new LongField(this);
	public	final	LongField		splitTimestamp = new LongField(this);
	public	final	BooleanField	isLastChild = new BooleanField(this);
}

To send out the OrderNew message above, you can simply re-use the same OrderNew instance, over and over again, creating zero garbage. Below an example of how you would send an OrderNew message from a CoralSequencer node:

			if (topBook.isSignal.get()) {
				
				long splitTimestamp = useEpoch ? timestamper.nanoEpoch() : timestamper.nanoTime();

				for(int i = 0; i < ordersToSend; i++) {

					boolean isBid = (i % 2 == 0);
					
					orderNew.symbol.set(topBook.symbol.get());
					
					if (isBid) {
						orderNew.side.set('B');
						orderNew.size.set(topBook.bidSize.get());
						orderNew.price.set(topBook.bidPrice.get());
					} else {
						orderNew.side.set('S');
						orderNew.size.set(topBook.askSize.get());
						orderNew.price.set(topBook.askPrice.get());
					}
					
					orderNew.myTimestamp.set(useEpoch ? timestamper.nanoEpoch() : timestamper.nanoTime());
					orderNew.splitTimestamp.set(splitTimestamp);
					
					orderNew.isLastChild.set(i == ordersToSend - 1);
					
					if (batching) {
						writeCommand(orderNew);
					} else {
						sendCommand(orderNew);
					}
				}
				
				if (batching) flush();
			}

As you can see, you simply populate the fields with data and call the sendCommand(Proto) method of a CoralSequencer node.

Now to receive a CoralSequencer Proto message, you first need to define a parser for your Proto messages. Luckily that’s super easy as you can see below:

import com.coralblocks.coralsequencer.protocol.AbstractMessageProtoParser;
import com.coralblocks.coralsequencer.protocol.Proto;

public class ProtoParser extends AbstractMessageProtoParser {

	@Override
    protected Proto[] createProtoMessages() {
	    return new Proto[] { new OrderNew(), new TopBook(), new OrderCancel() };
    }
}

Then you can use the proto parser above inside your Node’s handleMessage method to parse a Proto message out of a CoralSequencer message:

    private final ProtoParser protoParser = new ProtoParser();

	@Override
    protected void handleMessage(Message msg) {
	    
		if (isRewinding()) return; // do nothing during rewind...
		
		char type = protoParser.getType(msg);
		char subtype = protoParser.getSubtype(msg);
		
		if (type == OrderNew.TYPE && subtype == OrderNew.SUBTYPE) {
			
			OrderNew orderNew = (OrderNew) protoParser.parse(msg);
			
			if (orderNew == null) {
				Error.log(name, "Can't parse OrderNew:", msg.toCharSequence());
				return;
			}
			
			long now = useEpoch ? timestamper.nanoEpoch() : timestamper.nanoTime();
			
			long latency = now - orderNew.myTimestamp.get();
			
			ordersBench.measure(latency);

			if (orderNew.isLastChild.get()) {
				latency = now - orderNew.splitTimestamp.get();
				splitBench.measure(latency);
			}
		}
    }

Cool! That’s great! So what is the downside of using CoralSequencer’s serialization framework? To keep things simple, super fast and garbage-free, it does not give you any help for schema evolution, versioning and backwards compatibility. You are able to add (i.e. append) new fields to an existing message without having to update all nodes, but if you attempt to remove a field or change the order of the fields appearing in a message, then all nodes of your distributed system will have to be updated with the new schema class code in order to use/support the new message format. There is also support for IDL, optional fields and repeating groups.

Coral Blocks