Tuesday, September 22, 2009

Client Side Google Protocol Buffers & GWT

I have implemented a, crude, but working 'complier' to turn Google Protocol Buffers into GWT ready
java. It uses the base from the google code project here ... http://code.google.com/p/protobuf-js/, which I translated to Java so it could be included and complied by GWT (I added the ability to parse floats as well).

To test I accessed ~3500 protocol buffers and parsed them on different browsers (each containing a String and ~7-10 floating point numbers). I found varying results...

Chrome: ~2500ms
Safari: ~2800ms
FF: ~18,000ms
IE: ~72,000ms (stellar)

... this result had to use event chaining as to not choke the browser (FF & IE).

With such variation in parse time, this was not a reasonable solution. Parsing the protobuffers server side and sending them in JSON turned out to be MUCH faster. With all browsers having a reasonable JSON parsing time (under a second).

For the optimal solution I just modified my 'compiler' to spit out GWT overlay classes for the JSON instead of the proto classes. I then send the JSON wrapped up in a GWT-RPC. All works great now, very snappy performance.


Rusco said...

Opera ?

SkateThere said...

Could you please explain a little more of your solution? Google already releases Java ProtoBuf code. Since you're only sending JSON to the client, the ProtoBuf code doesn't need to be altered to be GWT-compilable, right? So all that's needed is to use the provided ProtoBuf parser to decode the protobuf, and then emit JSON. Am I missing something?

Justin Merz said...

So I would read this first. http://code.google.com/p/google-web-toolkit/issues/detail?id=2649&q=protocol ... Tell's you why GWT can't compile the standard ProtoBuf code. As for what I am now doing; basically I have java code that runs the (ProtoBuf) java code (generated from the .proto file) and creates GWT overlay types. Then I have a server library that reads serialized protocol buffers and spits out JSON (that is then wrapped by the overlay types on the client). I have a test gadget here. You can see the queries are snappy. The protobufs are being stored in an RBNB server that is queried by a GWT servlet on tomcat, the protobufs are de-serialized, turned into json, then sent to the client.

SkateThere said...

Ahh, OK. So the interesting part is the .proto parser, which creates your overlay types. No actual new parser for a protobuf object (at least in your final code), since Google's reference implementation works fine for emitting server-side JSON. Thanks.

Mark Schmit said...

Any chance you're planning on releasing your compiler for others to use? I'd love to be able to use Protobufs in my GWT frontend, but haven't had time to make such a thing myself.

Justin Merz said...

Sure, but please, before you judge the code, realize this was a test. Once I did not get the performance I needed, I stopped working on this code base, so it is a bit ugly. Here are the two files you need Main Class and include functions for js protobuffer parsing.