Fairy-Wing Wrapup: Nokogiri Performance Wednesday, May 18, 2011

TL;DR

  • Nokogiri’s DOM parser was extremely way faster than either the SAX or Reader parsers, in this particular real-world example.
  • ActiveSupport Hash#from_xml, I am dissapoint.
  • On JRuby, Nokogiri 1.5.0 is extremely way faster than Nokogiri 1.4.4, in this particular real-world example.

Artists Pre-enactment

(Shout-out to @jonathanpberger for the Artist’s Pre-enactment of Paul Dix wearing the fairy wings.)

Previously, on the Fairy Wing Throwdown …

So, you might remember that a few months back, @pauldix bet me that JSON parsing is an order of magnitude faster than XML parsing. (If you’re not in the loop, you can read the dramatization of the bet).

TL;DR, Paul lost that bet, and so will be wearing my daughter’s dress-up fairy wings during his RailsConf 2011 talk on Redis on Thursday. Awesome!

You can view the winning benchmark here.

I want to go to there.

The bet revolved around a real-world use case (Paul and I both work at Benchmark Solutions, a stealth financial market data startup in NYC).

You can view the data structure at the Offical Fairy-Wing Throwdown Repo™, https://github.com/flavorjones/fairy-wing-throwdown, but the summary is that it’s 54K when serialized as JSON, and is comprised (mostly) of an array of key-value stores (i.e., hashes).

Because I wanted to not just win, but to destroy Paul, I implemented the same parsing task using Nokogiri’s DOM parser, SAX parser, and Reader parser, expecting that code complexity and performance would correlate, somehow. In my mind, the graph looked like this:

Expected complexity and performance

But I was shocked and dismayed to see the real results:

Reality bites

What the WHAT?

Yes, that’s right. My payback for increasing the complexity of the code was a reduction in performance. The DOM parser was extremely way faster than either the Reader or SAX parsers.

Let me say that again: the DOM parser implementation was compellingly faster (1.3x) than the SAX parser implementation.

Why would that be? Good question, which I’ll deep-dive into in my next post. But suffice to say, the SAX parser is bottlenecked on making lots of callbacks from C into Ruby-land.

ActiveSupport, I am dissapoint.

Another big wow for me was how slow ActiveSupport’s Hash#from_xml method is. The benchmark shows that it’s about 40 times slower than the partial implementation using Nokogiri’s DOM parser.

Somebody should work on that! It wouldn’t be tough to hack an alternative implementation of Hash#from_xml on top of Nokogiri. If anybody’s looking for an interesting project, there it is.

You can be my @yokolet

Here’s a chart of how the DOM parser implementation works on various platforms:

DOM parser on various platforms

Holy cow! The pure-Java implementation on Nokogiri 1.5.0.beta.4 is 4 times faster than the FFI-to-C implementation on Nokogiri 1.4.4 (28s vs 117s). That’s crazytown!

Thanks to everyone who’s committed to the pure-Java code, notably @headius, @yokolet, @pmahoney and @serabe.

Chart Notes

The “expected performance” line chart is in imaginary units.

The “actual performance” line chart renders performance in number of records processed per second, so bigger is better. The Saikuro and Flog scores were normalized on their values for #transform_via_dom.

The “DOM parser on various platforms” bar chart renders total benchmark runtime, so smaller is better.

6 comments:

anne.ominous said...

I'm not too sure that using Nokigiri's parser in ActiveSupport would be such a good idea. Sure, performance is important, but there are other considerations also. I have a project that includes the Nokigiri gem, which takes about 10 times as long to load on startup than it does without Nokogiri.

Mike said...

I wasn't suggesting adding Nokogiri to ActiveSupport. I was suggesting that someone build a gem that implemented #from_xml using Nokogiri, and that could be used in place of those ActiveSupport methods.

pete higgins said...

ActiveSupport's XMLMini module is what is used by from_xml, and you can change what background it uses with something like:

ActiveSupport::XMLMini.backend = :nokogiri

... or :nokogirisax, :libxml, :libxmlsax, and a few others. It defaults to REXML, unfortunately.

Check it out:
Hash.from_xml

ActiveSupport::XMLMini

pete higgins said...

s/background/backend/

lth said...

did you try yajl-ruby for the JSON parsing?

https://github.com/brianmario/yajl-ruby

The numbers are tight, but paul might have unnecessarily donned those wings.

stepheneb said...

Nokogiri is already available as a backend for ActiveSupport's XmlMini Hash.from_xml builder:

https://github.com/rails/rails/blob/master/activesupport/lib/active_support/xml_mini/nokogiri.rb

When you ran your tests under JRuby with ActiveSupport did you try using the JDOM backend for XmlMini?

XmlMini.backend = 'JDOM'

See: https://github.com/rails/rails/blob/master/activesupport/lib/active_support/xml_mini/jdom.rb

I haven't updated these benchmarks to test the pure Java Nokogiri backend -- see:

https://github.com/stepheneb/rails_hash_from_xml

and

https://github.com/stepheneb/rails_hash_from_xml/blob/master/bench_hash_from_xml.rb