MongoDb optimisation for scale

This blog has moved

I redesigned the schema for one of our MongoDb clusters today. In doing so, I managed to reduce request times by 1.5 orders of magnitude!

We're now sustaining request times of roughly 30ms. The first mongo schema was about 400ms on average, with occasional bursts to over a second. 30ms is also much better than the original mysql based solution.

In the original mongo schema, I was just stuffing all data for the last 30 days into two big collections. This caused problems as the resulting indexes were huge. As it turned out, the active data set only actually needed the most recent hour's worth of data.

So I moved the active set into a TTL collection set to expire documents older than an hour. This means the indexes are now absolutely tiny.

We still need data over an hour old, but exclusively for read-only use. I moved this into datestamped collections: each day's data is now in its own collection. As we retrieve data in day chunks, this makes retrieval and deletion of old data extremely simple and performant.

PHP MongoDb replicaset gotcha

This blog has moved

Ran into a gotcha with PHP+MongoDb the other day.

Now, PHP ain't my favourite language, but its what Piwik is written in so I'm stuck with it for that. I've partially ported Piwik to use mongodb (vs mysql) as a storage system for scalability reasons.

The problem is that if you're connecting to a mongodb replicaset, you must ensure the hostnames you use are the same as the ones used in the replicaset configuration: if you refer to them as "node1, node2" in the replicaset, but connect to them as ",", it doesn't work well.

The PHP Mongo docs do actually warn about this:

"The host names that you specify here must match the names as given in the replica set configuration because the driver uses the name as specified in the replica set configuration as identifier in its internal mapping. Specifying (for example) the IP address instead of the configured host name makes that the driver keeps a connection for both the host name and its accompanying IP address."

I found it was worse than this: if you get it wrong, it will not cache connections at all; every db request will create a new connection to the database, which will cause unnecessary load. Its easy to miss where the load is coming from too; but as usual, good old tcpdump showed the problem almost instantly.

One simple reconfiguration of the db hostnames later and all was well.

Authenticated coinage

This blog has moved

Interesting, so the new £1 coins have a high speed authentication system built in.

BBC article here.

Discussion on technical details here.

So far, it sounds like it's just an anti fraud measure; it doesn't sound like each coin has a unique id, which would be worrying.

Return to the joggler

This blog has moved Now that Intel have a slightly less crap driver for the Poulsbo/GMA500 chipset, with the more recent EMGD driver, I've converted one of my O2 Jogglers to be a touchscreen media player in the kitchen. I'm using Arch Linux, and I've just released some AUR packages to support it. If you install:It'll have a kernel and xorg support customised for the Joggler. I'm running bog standard KDE on it, and it works pretty well with hardware OpenGL support. I've not managed to get video acceleration working though, but smplayer/mplayer plays all standard definition files fine.

The linux-joggler package has my joggler-specific patches forward ported to 3.2, so if you need them for a different distro, you can grab them from the above link (download the tarball and extract).

Here's a pic of the setup :) 

GAN / UMA / WIFI Calling / Signal Booster on the Galaxy S II

This blog has moved

I've been happily using my Galaxy S II for the last few months, and of course I've replaced the original firmware with a custom one.

However, recently a colleague acquired one as well. His had the stock Orange firmware, and has a special app called "Signal Boost". Searching around, I found that this is actually an implementation of GAN / UMA. I'd heard of this technology, but wasn't aware it was available with Android. As it turns out, it has been implemented by Kineto. The net effect is the phone will tunnel through your local WIFI and use that instead of the GSM network. This is quite handy if you're in a crap signal area. Of course, your mobile nework has to support it, and luckily Orange do in the UK.

I'm using cyanogenmod 7.1 on my phone, and as this is a proprietary impementation of GAN, it is not included. However, they have included the kernel-side device, gan-eth required by the proprietary user space code, which makes things a lot easier.

Basically, we need to extract a few files from the Orange ROM and add them into the cyanogen ROM on the phone, and that's about it: I have it working on my phone now.


The following instructions are using Linux for the extraction, and cyanogenmod on the phone, but I assume they'll be similar for other ROMs.

Oh, and obviously doing this is entirely at your own risk, will void your warranty and eat your gerbils etc etc.

You will need:

  • 7zip

  • Orange UK rom image downloaded from this page. Grab the BVKG3 one for Orange UK.

  • The simg2img utility from here (you'll need to compile it). If you run Arch Linux, there's an AUR build for it called "ext4_utils".

  • ADB, and developer mode enabled on the phone.

The following will extract the files on the Linux machine:
7z x GT-I9100_ORA_I9100BVKG3_I9100BVKG2_I9100ORAKG3.exe
tar xf GT-I9100_ORA_I9100BVKG3_I9100BVKG2_I9100ORAKG3.TAR
simg2img cache.img cache.raw
mkdir c
mount -o loop cache.raw c
cp c/recovery/ .
umount c
simg2img factoryfs.img factoryfs.raw
mount -o loop factoryfs.raw c
cp c/lib/ .
umount c

Now, copy and paste the following into a text file called "default.prop":

After that, you'll have all the files needed. Plug in your phone to USB to push the files across with adb as follows:
adb shell remount -o rw,remount /system
adb push system/lib/ /system/lib
adb push system/lib/ /system/lib
adb push system/lib/ /system/lib
adb push system/app/WiFi-Calling.apk /system/app
adb push /system/lib
adb push default.prop /system

Finally, reboot the phone, go to the Signal boost app, and enable it. After a few seconds (assuming your wifi is strong enough) the red house icon on your status bar should go green, and the GSM signal bars should drop to 0. If you use "adb logcat", you can see lots of output as it negotiates the connection.


I originally tried this using the that came with cyanogen, but logcat showed rild crashed as soon as I tried to enable wifi calling. The one from the orange rom solved this. I'm assuming it either needs some sort of co-operation from the hardware RIL, or unloading the RIL on the fly hadn't ever really been tested before.

It sounds like its switching the RIL implementation on the fly, which is quite cool.

I wish this was a standard feature of android; it might encourage more networks to support it.