Skip to end of metadata
Go to start of metadata

Performance Tests (Streaming, Scheduling, Network Connections)

Using the Simple Workload with recordcount=100 and operationcount=100. Increments of start times are reduced to [1;10] milliseconds to stress the system.

Checked that operations are executed in the scheduled order if threadcount is set to 1. Option threadcount=1 can be used if operation order needs to be preserved.

Tests

  • Test 1: Streaming. How fast can the driver iterate through the generated operations?
  • Test 2: Streaming and Scheduling. How fast can the driver iterate through the generated operations, send them to the thread pool, and run them? (Running an operation = just returning null)
  • Test 3: Streaming, Scheduling, and Network Connections. How fast can the driver iterate through the generated operations, send them to the thread pool, and run them? (Running an operation = send an asynchronous HTTP GET request to a website, ignore answer)

Test script

Follow the instructions on github to automatically run a test and visualize its runtime.

Results

Every configuration (test number + thread count) was run 100 times. The runtimes of each configuration are shown in one boxplot diagram. Diagrams of the same test number but with different thread counts are combined into one graphic to allow runtime comparison.

Laptop: 2 CPUs, Server: 16 CPUs

Test 1

Test 2

Test 3

Interpretation

Driver should show the same performance on every system, driver should be hardware independent. Queries are therefore not sent 'as fast as possible' but according to their scheduled start time. 

However, in these tests, we schedule one query per nanosecond. Therefore speedup on bigger machines/with more threads indicates that the machines are not fast enough to send all queries on time. Using more hardware and/or threads reduces the start time delays.

  • No labels