Hello! Today you may read here my interview to Christopher Meiklejohn, one of the speakers at the upcoming Erlang Factory in San Francisco. Christopher is working on Riak at Basho Technologies.
Erlang, Vector Clocks and Riak!
Paolo – Hello Chris! It’s great to have another Basho Erlanger here! Can you introduce yourself please?
Christopher – It’s great to be here! My name is Christopher Meiklejohn, and I’m currently a software engineer at Basho Technologies, Inc., where I work on our distributed datastore, Riak. In addition to that, I’m also a graduate student at Brown University, in Providence, RI, where I study distributed computing.
Paolo – Before joining Basho you were working in a different company (i.e., Swipely) where you dealt with Ruby code. Did you already know Erlang when you started at Basho? How would you describe the switch between these two languages?
Christopher – During the time I was at Swipely they had Riak deployed in production, which was what initially got me interested in Basho, Riak, in particular, Erlang. When I joined Basho, I knew very little Erlang and spent my first few weeks at the company learning it.
That said, I love Erlang as a language and as platform to build application on. I wouldn’t necessarily say that the change from Ruby to Erlang was anything that was unexpected, specifically because I already had functional programming experience using languages like Scheme and Haskell.
Paolo – Rubyists tend to be addicted to TDD. Were you able to maintain such a good practice also when coding Erlang?
Christopher – Well, I’ll start with a disclaimer. I was primarily responsible for the introduction of behavior driven development at Swipely for feature development, in addition to promoting pair programming within the development team.
That said, testing and verification of software is a very interesting topic to me.
While I believe that all software should be properly tested, I’ve never been particularly dogmatic about when in the cycle of development testing is performed: whether it’s done during development to guide the design of the software or whether it’s done afterwards to validate the authored components. I do, however, have one major exception to this rule: when attempting to reproduce a customer issue and validate a fix for the issue.
This is purely a pragmatic decision that’s come from working on large scale distributed systems: testing and verification of distributed systems is extremely hard given the number of cooperating components involved in completing a task.
At Basho, we take two major approaches to testing Riak: integration testing using a open source test harness we’ve developed that allow us to validate high level operations of the system, and QuickCheck for randomized testing of smaller pieces of functionality.
Paolo – At the upcoming Erlang Factory in San Francisco you will give the following talk: “Verified Vector Clocks: An Experience Report”. Can you introduce in a few words the arguments you will treat during the talk?
Christopher – My talk is going to look at an alternative way of asserting correct operation of software components, commonly known as formal verification.
The talk will specifically focus on modeling vector clocks for use in the Riak datastore using an interactive theorem prover called Coq. This allows us to assert certain mathematical properties about our implementation, and perform extraction of the component into Erlang codewhich we can directly use in our system.
Paolo – Who should be interested in following your talk and why?
Christopher – Given the topics involved, I’m planning on keeping the talk pretty high level and will touch a variety of topics: the theorem prover Coq, which implements a dependently-typed functional programming language, the basics of using Core Erlang, a de-sugared subset of the Erlang programming language, and how we put all of the pieces together.
Paolo – Lamport’s vector clocks are well known by people working in fields connected to distributed systems. Can you explain briefly what they are and in what fields they can be used?
Christopher – Vector clocks provide a model for reasoning about events in a distributed system. It’s a bit involved for this interview to get into the specifics about how they work and when they should be used, so I’ll refer to you two excellent articles written by Bryan Fink and Justin Sheehy of Basho.
“Why Vector Clocks Are Easy”
“Why Vector Clocks Are Hard”
Paolo – About the application vvclocks, are you planning to keep the development on? If so how can people contribute?
Christopher – At this point, the project mainly serves as a playground for exploring how we might begin to approach building verifiable software components in Erlang. What has been done so far is available on GitHub, it’s actively being worked on by myself as my time allows, and if you’re interested in helping to explore this further, feel free to reach out to me via e-mail or on Twitter.
Hello! Probably most of you already know the Erlang Factory Light will be held in San Francisco between 3 and 15 March . Among the 50 talks there will be the one held by Ferndando Benavides. Fernando is an Erlang developer from Argentina, currently working at Inaka as director of engineering. Let’s talk with him about Erlang, Inaka and his talk!
Let’s get started!
Paolo – Hello Fernando, thanks for making yourself available for the interview. Would you like to introduce yourself?
Fernando – Hi! Well… I’m an software developer from Argentina. I started programming when I was 10 using QBasic and “improving” gorillas.bas :P. From that point on I studied computing and programming my whole life until I graduated as a computer scientist in the Universidad de Buenos Aires, a couple of years ago. I started working when I was 18, coding in Visual Basic, then moved to .Net after several years. But thanks to my career path in the university I discovered functional programming (Haskell) and fell in love with it, so when I had a chance to actually work doing functional programming (this time in Erlang) I had no doubt I wanted to do so. That was 5 or 6 years ago. And I’ve been doing Erlang development ever since. About an year ago, I’ve been assigned as the Director of Engineering at Inaka. That basically means I’m now in charge of the architecture and design of all Inaka’s systems, whether they’re written in Erlang or not.
Paolo – You are currently working as Director of Engineering at Inaka: would you like to give some insight about Inaka?
Fernando – Well… you should check out our website: inaka.net 🙂
Inaka was found on September 2008 by Chad DePue, and I’m glad to say I was there. We were just 5 people working in a small apartment in Buenos Aires and we had just one project. Now we are a much much bigger team, with people working from different countries, in a big number of projects. But the idea behind our work was always the same: Our clients have beautiful startup ideas… and we do our best to turn them into real systems or applications that they can quickly launch and grow. Most of our products are iOS/Android applications with Erlang and/or Ruby servers on their backend. And most of the products we build have some important large-scale and/or high-concurrency requirements.
Paolo – How does Erlang fit in Inaka’s ecosystem? Why did you start using it?
Fernando – Erlang is our #1 language choice (at least since I’m Director of Engineering :P) for server developing.
We started using it since our very first system. That was a second screen system that provided additional content to display on iOS devices when a particular show was on air on TV. It also let users interact, by writing comments and “kinda” chatting. When designing the system, Chad detected the high concurrency and real-time demands that it had: during the hour the show was on air, lots of users would be connected and interacting and they devices had to be showing the right content in the right moment, not later. Erlang was the right choice for a language to write the servers with. BTW, making that system scale was not an easy task, even when written in Erlang, and that was what my last talk in an Erlang Factory was about 😉
Paolo – At Inaka you have been using Erlang for many years. How would answer to this question? Do you confirm what Bob Ippolito wrote in his answer?
Fernando – Being a Haskell fan myself, I can tell you I nodded in approval on almost every paragraph he wrote, including the last one ;). So, yes… I confirm what he wrote.
As him, I’m also not aware of another platform that has the right combination of features, performance and maturity to make it possible to build the kind of systems that we build at Inaka. Maybe instead of binary matching syntax, I would mention that one of the main parts of most of our servers is a RESTful API that usually has a SSE component that keeps clients up to date with server pushes. How easy, clean and nice it is to write the code to implement that and make it scale in Erlang is hardly comparable to any other language that I know.
Paolo – In your talk at Erlang Factory Lite you will talk about: “Lucene Server: From Erlang to Java and Back Again”. Do you mind giving us some information about the talk?
Fernando – It will be a similar talk as the one I gave on the last ErlangDC. I’ll talk about a system that I’ve built and it’s now open-source, thanks to TigerText: Lucene Server. It’s an Erlang application that lets its users index, delete and query documents (i.e. proplists, in Erlang terms) using Lucene (which is written in Java). The key point here is that no Java knowledge is required for lucene_server users. All Java compilation, running, handling, stopping, etc. is managed internally by the lucene_server application. I’ll tell you how it’s works and how it’s implemented, and I’ll show you some lessons I learned along the way 😉
Paolo – Who should attend your talk and why?
Fernando – My talk is mostly aimed at Erlang developers that once faced JInterface, got scared and then decided to implement their solution in a different way… I was there, too 😉
Also, those devs that need a great indexing solution like Lucene as part of their systems. They will learn about an open-source application that may be exactly what they need.
Paolo – About Erlang and Java, I really liked your blog post on Inaka’s blog. Can you talk briefly about it?
Fernando – I wrote that post before lucene_server was open sourced and, in the same way that I couldn’t talk about the second screen system I described before but nevertheless I wrote a blog post about its story, I wrote this post (or rather this still unfinished series of posts) about lucene_server’s history.
You worked both with JVM and Erlang VM which are widely acknowledged among developers. Talking about performances and tuning, can you give some pros and cons for both?
I never feel comfortable talking about performances and tuning of VMs. I’m not sure I know enough about Erlang VM to talk about that and I’m positive I know nothing about JVM in this respect. Also, comparing the performance of two virtual machines for two completely different languages is tricky to say the least. It’s not like you can run the same code in both of them and see which one behaves better 😉
Paolo – Last question: lately I have seen a lot of tweets complaining about the Erlang syntax. What do you think about this?
Paolo – BTW, I am really puzzled about your name: is it Brujo or Fernando?
Fernando – Well… According to my passport, I’m Fernando Benavides Rodriguez. According to anyone else, I’m just “Brujo” :). In other words, “Brujo” is my nickname, it means something close to “sorcerer” or “shaman” in spanish. The nickname was taken from my favorite character in a series of books I love (La Saga de Los Confines, by Liliana Bodoc) and it stuck with me forever. So, just call me Brujo 🙂