As you might have guessed from my last couple blog posts, I’ve been experimenting with a few languages and libraries for a new project. I’ve finally gotten things far enough along to the point where I’d like to start getting other developers and potential users involved. I’m introducing SlackDB, an open source project that combines the functionality of relational databases with the ideas behind eventually consistent, shared-nothing data stores to provide a new database to support new and existing web applications in the cloud (enough buzzwords in there?). This is an idea I wrote about a while ago and recently I started putting a lot of night and weekend time into it.
It is still very early on in the development process, but the ideas behind it are starting to solidify and a fair amount of code is already written. It’s not all that useful yet, but much of the framework is there now. I’m starting from scratch and chose to use Erlang for the reasons mentioned in my last post (I wrote a prototype in Python and scrapped it due to performance reasons). It already speaks the MySQL protocol so you can interact with it using any of the APIs or tools you would normally use for MySQL. Other protocols and query languages may be supported in the future, but I decided to focus on the most popular one for now to help gain adoption.
I’m looking for database internals and/or Erlang programmers to help out (or anyone else who wants to learn). I want this to be a true community-driven project from design to documentation, there is just too much to do alone. Perhaps you’re looking for a side project or a reason to learn a language like Erlang. Here is your chance. This project may sound a bit ambitious, but we don’t need to write a general purpose database to support all uses cases. We’ll take an iterative approach and choose what’s important. I’ve outlined a number of things already in the design principles page on the wiki. I also have a number of blueprints up on the Launchpad project page.
SQL meets NoSQL
As much as I dislike the ambiguity of ‘NoSQL’, SlackDB is blurring the lines between the two by using techniques normally found in ‘NoSQL’ projects like eventual consistency and deterministic replication. People still have problems to solve and applications to run that require relational data models. A number of applications don’t even use most of the features provided by an ACID compliant relational database. For example, WordPress, one of the most popular web applications in the world, doesn’t even use real transactions (it auto-commits on every command). It is a prime application to use a relaxed consistency back-end with a true multi-master setup, allowing for higher availability and geographic redundancy.
Isn’t this what project X is trying to do?
Not that I’ve seen yet. I haven’t found any relational databases that throw away immediate consistency, isolation, and in turn provide a deterministic, idempotent replication model. Also, I don’t see any relational databases trying to solve the multi-tenant cloud problem in a way that embraces common properties of the cloud. Resources need to be elastic and services should tolerate network partitions and not show degraded performance on either side.
Is this a company? Are you going to sell it? Who will use it?
While I would like to be paid to work on this, I never plan to sell the software or release some parts closed (no open-core). It’s BSD licensed, so anyone could try to fork and sell it, but I hope you don’t. My primary motivation is that I feel the open source cloud service space is still lacking a big component and I want to help fill the gap. I would like to see projects such as WordPress and Drupal be able to run out of the box with multi-master SlackDB configurations. Down the road I would really like to see public and private cloud providers adopt this as part of their multi-tenant platform-as-a-service (PaaS) offering.
I’ll be at OpenSQL Camp in Boston in a couple weeks and am planning on talking more about SlackDB there. If you are interested try to make it, otherwise feel free to drop me a note at eday@oddments.org.
PlanetMySQL Voting: Vote UP / Vote DOWN