CoralReactor vs Vert.x Performance Comparison

In this article we write two identical http servers using CoralReactor and Vert.x and compare their throughput and latency for different number of simultaneous connections.

Test Details

  • The HTTP server will accept a GET request to /plaintext and respond with a simple plaintext response: 
    GET /plaintext HTTP/1.1
User-Agent: CoralReactor
Host: www.coralblocks.com
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive

    HTTP/1.1 200 OK
Content-type: text/plain
Server: CoralReactor
Date: Tue, 2 Feb 2016 14:25:28 America/Chicago
Content-Length: 13

Hello, world!
  • The connections are keep-alive so multiple requests are made through the same connection.
  • Both CoralReactor and Vert.x HTTP server will handle all request from all connections on the same thread pinned to an isolated CPU core.
  • The client begins by opening X http connections to the server. Each connection will make Y requests to the server. A http connection only proceeds to send the next request when it receives the response for the previous request from the server. After a connection receives Y responses it stops sending requests.
  • The clients handles all HTTP connection inside the same reactor thread pinned to an isolated CPU core different than the one being used by the server.
  • The client measures the latency of each individual request up until the response is received from the server (ping-pong). In other words, the start time is right before the request is sent and the stop time is right after the response is received.
  • The client measures the total time it takes to send and receive the response for all messages. That way, the throughput can be calculated in messages per second. (Note: This is not really a true throughput number because we wait for the response from the previous message to send the next one)
  • The client first warms up with Z requests before it starts taking measurements.
  • Both client and server were run on the same machine, connected through loopback/localhost (127.0.0.1).
  • Both CoralReactor and Vert.x http servers were run using the -verbose:gc to check for garbage creation.

CoralReactor Http Server Code

You can click here to check the CoralReactor code for this simple plaintext http server.

Vert.x Http Server Code

The Vert.x code used is exactly the same as the one used for the TechEmpower benchmarks. You can check the source on github by clicking here. The only difference is that we made the Vert.x server single threaded by doing:

Vertx.vertx(new VertxOptions().setEventLoopPoolSize(1));

We were then able to pin the vert.x-eventloop-thread-0 to an isolated CPU core.

Http Client Code

You can click here to check the client code used to perform the benchmark. As stated above, all connections and requests were handled by the same reactor thread pinned to an isolated CPU core.

Results

Note: Contact us if you want to run the benchmarks on your own environment.

Number of connections: 1
Number of requests per connection: 10 million
Number of requests to warmup: 1 million

CoralReactor:

Requests per second: 184,584

Iterations: 9,000,000 | Avg Time: 5.37 micros | Min Time: 5.004 micros | Max Time: 67.513 micros | 75% = [avg: 5.253 micros, max: 5.356 micros] | 90% = [avg: 5.303 micros, max: 5.83 micros] | 99% = [avg: 5.355 micros, max: 5.969 micros] | 99.9% = [avg: 5.365 micros, max: 8.165 micros] | 99.99% = [avg: 5.368 micros, max: 10.601 micros] | 99.999% = [avg: 5.369 micros, max: 19.984 micros]

Zero garbage created by the server

Vert.x:

Requests per second: 91,717

Iterations: 9,000,000 | Avg Time: 10.857 micros | Min Time: 8.747 micros | Max Time: 9.758 millis | 75% = [avg: 10.745 micros, max: 10.851 micros] | 90% = [avg: 10.768 micros, max: 10.94 micros] | 99% = [avg: 10.82 micros, max: 12.116 micros] | 99.9% = [avg: 10.839 micros, max: 15.402 micros] | 99.99% = [avg: 10.845 micros, max: 19.014 micros] | 99.999% = [avg: 10.846 micros, max: 120.687 micros]

Server produces a lot of garbage (GC kicks in 46 times)



Number of connections: 10
Number of requests per connection: 1 million
Number of requests to warmup: 1 million

CoralReactor:

Requests per second: 278,086

Iterations: 9,000,000 | Avg Time: 34.934 micros | Min Time: 6.024 micros | Max Time: 791.866 micros | 75% = [avg: 31.959 micros, max: 35.144 micros] | 90% = [avg: 32.832 micros, max: 44.813 micros] | 99% = [avg: 34.603 micros, max: 66.406 micros] | 99.9% = [avg: 34.894 micros, max: 67.513 micros] | 99.99% = [avg: 34.925 micros, max: 81.037 micros] | 99.999% = [avg: 34.932 micros, max: 145.331 micros]

Zero garbage created by the server

Vert.x:

Requests per second: 134,771

Iterations: 9,000,000 | Avg Time: 69.822 micros | Min Time: 9.207 micros | Max Time: 23.137 millis | 75% = [avg: 61.641 micros, max: 66.104 micros] | 90% = [avg: 62.979 micros, max: 127.968 micros] | 99% = [avg: 69.072 micros, max: 131.862 micros] | 99.9% = [avg: 69.661 micros, max: 142.079 micros] | 99.99% = [avg: 69.73 micros, max: 184.135 micros] | 99.999% = [avg: 69.774 micros, max: 1.814 millis]

Server produces a lot of garbage (GC kicks in 46 times)



Number of connections: 100
Number of requests per connection: 100,000
Number of requests to warmup: 1 million

CoralReactor:

Requests per second: 271,462

Iterations: 9,000,000 | Avg Time: 365.63 micros | Min Time: 35.111 micros | Max Time: 1.351 millis | 75% = [avg: 302.943 micros, max: 526.011 micros] | 90% = [avg: 342.4 micros, max: 554.915 micros] | 99% = [avg: 362.965 micros, max: 595.942 micros] | 99.9% = [avg: 365.226 micros, max: 730.979 micros] | 99.99% = [avg: 365.584 micros, max: 789.276 micros] | 99.999% = [avg: 365.623 micros, max: 911.946 micros]

Zero garbage created by the server

Vert.x:

Requests per second: 145,614

Iterations: 9,000,000 | Avg Time: 660.654 micros | Min Time: 26.134 micros | Max Time: 24.131 millis | 75% = [avg: 623.152 micros, max: 679.096 micros] | 90% = [avg: 633.264 micros, max: 689.056 micros] | 99% = [avg: 652.877 micros, max: 1.327 millis] | 99.9% = [avg: 659.032 micros, max: 1.356 millis] | 99.99% = [avg: 659.922 micros, max: 3.156 millis] | 99.999% = [avg: 660.426 micros, max: 23.336 millis]

Server produces a lot of garbage (GC kicks in 47 times)



Number of connections: 1,000
Number of requests per connection: 10,000
Number of requests to warmup: 1 million

CoralReactor:

Requests per second: 209,370

Iterations: 9,000,000 | Avg Time: 4.24 millis | Min Time: 153.437 micros | Max Time: 161.154 millis | 75% = [avg: 3.382 millis, max: 6.224 millis] | 90% = [avg: 3.874 millis, max: 6.436 millis] | 99% = [avg: 4.18 millis, max: 9.477 millis] | 99.9% = [avg: 4.229 millis, max: 9.784 millis] | 99.99% = [avg: 4.234 millis, max: 10.352 millis] | 99.999% = [avg: 4.239 millis, max: 159.802 millis]

Zero garbage created by the server

Vert.x:

Requests per second: 115,511

Iterations: 9,000,000 | Avg Time: 8.465 millis | Min Time: 169.897 micros | Max Time: 33.606 millis | 75% = [avg: 8.148 millis, max: 8.526 millis] | 90% = [avg: 8.217 millis, max: 8.646 millis] | 99% = [avg: 8.377 millis, max: 16.817 millis] | 99.9% = [avg: 8.454 millis, max: 17.358 millis] | 99.99% = [avg: 8.464 millis, max: 24.52 millis] | 99.999% = [avg: 8.465 millis, max: 28.741 millis]

Server produces a lot of garbage (GC kicks in 47 times)



Number of connections: 10,000
Number of requests per connection: 1,000
Number of requests to warmup: 1 million

CoralReactor:

Requests per second: 150,386

Iterations: 9,000,000 | Avg Time: 60.671 millis | Min Time: 551.754 micros | Max Time: 509.849 millis | 75% = [avg: 52.618 millis, max: 60.087 millis] | 90% = [avg: 54.554 millis, max: 79.888 millis] | 99% = [avg: 59.577 millis, max: 118.592 millis] | 99.9% = [avg: 60.277 millis, max: 448.415 millis] | 99.99% = [avg: 60.628 millis, max: 452.036 millis] | 99.999% = [avg: 60.667 millis, max: 509.795 millis]

Zero garbage created by the server

Vert.x:

Requests per second: 111,005

Iterations: 9,000,000 | Avg Time: 89.484 millis | Min Time: 8.177 millis | Max Time: 512.795 millis | 75% = [avg: 83.511 millis, max: 93.813 millis] | 90% = [avg: 85.537 millis, max: 98.795 millis] | 99% = [avg: 88.308 millis, max: 160.282 millis] | 99.9% = [avg: 89.118 millis, max: 427.392 millis] | 99.99% = [avg: 89.443 millis, max: 489.883 millis] | 99.999% = [avg: 89.479 millis, max: 512.751 millis]

Server produces a lot of garbage (GC kicks in 47 times)

Conclusions

For a ping-pong HTTP test, CoralReactor is approximately twice as fast as Vert.x without producing any garbage.