Fairy-Wing Wrapup: Nokogiri Performance Wednesday, May 18, 2011

TL;DR

• Nokogiri’s DOM parser was extremely way faster than either the SAX or Reader parsers, in this particular real-world example.
• ActiveSupport Hash#from_xml, I am dissapoint.
• On JRuby, Nokogiri 1.5.0 is extremely way faster than Nokogiri 1.4.4, in this particular real-world example.

(Shout-out to @jonathanpberger for the Artist’s Pre-enactment of Paul Dix wearing the fairy wings.)

Previously, on the Fairy Wing Throwdown …

So, you might remember that a few months back, @pauldix bet me that JSON parsing is an order of magnitude faster than XML parsing. (If you’re not in the loop, you can read the dramatization of the bet).

TL;DR, Paul lost that bet, and so will be wearing my daughter’s dress-up fairy wings during his RailsConf 2011 talk on Redis on Thursday. Awesome!

You can view the winning benchmark here.

I want to go to there.

The bet revolved around a real-world use case (Paul and I both work at Benchmark Solutions, a stealth financial market data startup in NYC).

You can view the data structure at the Offical Fairy-Wing Throwdown Repo™, https://github.com/flavorjones/fairy-wing-throwdown, but the summary is that it’s 54K when serialized as JSON, and is comprised (mostly) of an array of key-value stores (i.e., hashes).

Because I wanted to not just win, but to destroy Paul, I implemented the same parsing task using Nokogiri’s DOM parser, SAX parser, and Reader parser, expecting that code complexity and performance would correlate, somehow. In my mind, the graph looked like this:

But I was shocked and dismayed to see the real results:

What the WHAT?

Yes, that’s right. My payback for increasing the complexity of the code was a reduction in performance. The DOM parser was extremely way faster than either the Reader or SAX parsers.

Let me say that again: the DOM parser implementation was compellingly faster (1.3x) than the SAX parser implementation.

Why would that be? Good question, which I’ll deep-dive into in my next post. But suffice to say, the SAX parser is bottlenecked on making lots of callbacks from C into Ruby-land.

ActiveSupport, I am dissapoint.

Another big wow for me was how slow ActiveSupport’s Hash#from_xml method is. The benchmark shows that it’s about 40 times slower than the partial implementation using Nokogiri’s DOM parser.

Somebody should work on that! It wouldn’t be tough to hack an alternative implementation of Hash#from_xml on top of Nokogiri. If anybody’s looking for an interesting project, there it is.

You can be my @yokolet

Here’s a chart of how the DOM parser implementation works on various platforms:

Holy cow! The pure-Java implementation on Nokogiri 1.5.0.beta.4 is 4 times faster than the FFI-to-C implementation on Nokogiri 1.4.4 (28s vs 117s). That’s crazytown!

Thanks to everyone who’s committed to the pure-Java code, notably @headius, @yokolet, @pmahoney and @serabe.

Chart Notes

The “expected performance” line chart is in imaginary units.

The “actual performance” line chart renders performance in number of records processed per second, so bigger is better. The Saikuro and Flog scores were normalized on their values for #transform_via_dom.

The “DOM parser on various platforms” bar chart renders total benchmark runtime, so smaller is better.