Jul 7, 2008

Protocol Buffers: Now in Open Source Flavor!

Google has open sourced protocol buffers, our data interchange format that we use for, well, just about everything we store. Protocol buffers are fast. And cool. And fast. Did I mention fast? I thought so.


Karl Fogel said...


As writing Cloud Code becomes more common, and programmers spend more of their time dealing with the problem of moving data between computation sites (as opposed to tuning single-threaded algorithms), having something like Protocol Buffers around to be a standard could make everyone's lives easier...

Karl Fogel said...

But, hmmm, why doesn't the protobuf project at GoogleCode have a version control repository available? I don't care if it's Subversion, Hg, Git, or whatever, but right now there's nothing. It's hard to see how this can be run as a real open source project without a public repository... (Or maybe that was never the plan?)

Ah, this thread seems to explain why. I'm a little surprised by Kenton Varda's assertion that " It would be infeasible for us to move primary development out of our own internal repository because so much code at Google depends on it", though. Hard? Yes. Infeasible? Probably not. Surely there are other open source projects that are equally intertwingled with Google's own code, and yet they manage somehow.

This sounds like a situation tailor-made for a DVCS.

Reading the thread, they don't seem to realize that they're making incoming patches much less likely by hiding the development stream from the public. Who outside Google would want to write a patch for protobufs, knowing that the internal code could already be arbitrarily different from whatever's currently visible on the download page? I know I'd probably just choose something else to do with my time, rather than write a patch that has a high risk of being obsolete before it's done.


Fitz said...

I agree with you Karl. We're working on it. *sigh*

Antony said...

Karl is exactly right.

For years, there has been precious few patches from the community to the MySQL server simply because practically no one in the community wanted to pay the cold hard cash for BitKeeper licences. Even though the repository was "open", since BitKeeper was commercial only, it may as well be closed as practically no one used it.

Open source needs open repositories.