July 28, 2011

How SMALL can you get?

As part of a project of my own, I have to fit my server (UnboundID Directory Server to be precise) on a small machine. How small? Well that's exactly the question I'm trying to get an answer to so I can figure out what to buy for that particular project. Options are many, from fitPC to trimslice and atom-based netbooks, but the question is how small can I make the server and still be able to have meaningful use of it. The experiment needs to be able to store data from 14 sensors every 16ms. This is a total throughput of inbound data of 875 writes per second. More importantly than the sheer throughput -which will be coming from an external microcontroller in bursts- the server needs to have small response time in order to be able to process all the writes before the next burst in order to keep up with the constant flow of sensor data.

Fiddling with it a bit today I managed to fit the server in a 48MB footprint on a test laptop -which is one of the candidates as it makes my life easier on the battery side of things, provided it will be able to sustain the cold temperatures, more on that in a later post- and I am happy to report that I could comfortably achieve over 1,600 writes per second. Note that while I was doing these test, not only was I running DS in a very small heap, but the laptop is also running the UnboundID Synchronization Server in an equally small JVM and order to investigate the performance, I'm also running netbeans, jvisualvm and some other tools (Apache Derby, FirebirdSQL and JEdit to be precise). So you can see we're not nearly in a realistic environment but it does mean that there is a lot MORE stress at the moment on the machine, which is good.
So 1,600 writes per second on this extremely small and contrived setup is good, right? Not quite, the response times are hovering at 1.2ms, which is too much for the application I need DS for. So I throttled it down a bit to see what I could get without pushing the server to the limit, here's what I got :
Column 1: recent throughput (writes per second)
Column 2: recent response time (in millisecond)
Column 3: recent error rate
Column 4: average throughput (writes per second)
Column 5: average response time (in millisecond)

So there you go, ~0.8ms write response time on an old laptop that looks like a good candidate.

I pushed even further the experiment by attaching the sensor board and looking at the REAL traffic going in to the DS and the response times further fell to around ~0.7 which gives me a 0.3ms window of headroom.

Great, I might be able to do some extra crunching "in flight".

1 comment:

  1. ah yes, there's a searchrate job going on the whole time to simulate the application conusming the data.