An interview with Michał Ślaski about TDD and testing in Erlang
Hello! After a very veeeeeeeeery long time I had the chance to write again in this blog and for me this is awesome: this year I promise myself to keep this blog up and running (or restart doing it using a supervisor 😛 ).
Getting back to serious stuff, today you may read my interview to Michał Ślaski from Erlang Solutions Kraków. Because of his experience as a developer and trainer I strongly recommend this interview not only to Erlang developers, but also to all the readers interested in TDD, testing and automation.
Exploring TDD and testing in a functional world
Paolo – Hello Michał, thank you for being here with us! Would you like to introduce yourself to our readers?
Michał – Hello, thanks for having me. My name is Michał Ślaski and I’m a senior architect at Erlang Solutions, branch manager of the Kraków office, founder of Lambda Academy and co-chair of the Lambda Days conference.
Paolo – Can you tell us how you started working with Erlang and how you discovered Test Driven Development?
Michał – In 2004 my friend and I came up with a proposal for our Master’s Degree project, which was a massively multiplayer on-line game. At the time we were working as C++/DirectX developers at one of the game studios in Kraków. We went to our supervisor, described the proposal, and he suggested we use Erlang on the server side. My friend implemented the front-end in Java3D and I implemented the back-end in Erlang + Mnesia. One year later I graduated and soon after got an interview at Erlang Solutions in London. I met with Francesco Cesarini and presented my Master’s Thesis about prototyping MMOGs with Erlang. It is a mystery to me why I got the job at Erlang Solutions, because the thesis was written in Polish…Francesco helped me publish a presentation at the Erlang User Conference in Stockholm and you can find it here: http://www.erlang.se/euc/05/Slaski.pdf
Paolo – You give courses about Erlang and TDD. Can you please explain to our readers new to TDD what it is and why we should apply this practice when writing our Erlang code?
Michał – Let me quote our training material here as it answers your question well. Testing is as old as programming itself. It is useful to document behaviours and demonstrate that a program has the required properties. Testing was traditionally done after writing code. Test-Driven Development dictates that tests should be written before the code. Writing tests before the code is not used for testing, but for design. Tests represent the specification to be implemented. Testing before writing code asks the developer to think about interfaces and how to use the code. It ensures that the application is written for testability, and ensures that parts of the application are modular and reusable. Tests are a side effect of test-driven development.
Paolo – Many confuse TDD with the so-called “test first”, but TDD is not only about tests, is also about design. Do you think that in Erlang as well we can apply TDD to explore the system and to build software with a better design?
Michał – Yes, Erlang being a functional language is a good fit for TDD. When we design Erlang systems we need to think not only about unit testing, but also about integration testing, system testing and acceptance testing. If you can have test cases on different levels it helps ensure both the design and the implementation are leaning towards a solution which will be acceptable by the end user or customer. Additionally, test suites provide information about your intentions to other developers who will work on your code later.
Paolo – There are many tools one can use to test Erlang code. Can you give to our readers a brief overview over the most common testing frameworks available and explain for what kind of tests they should be used?
Michał – We typically implement test cases as Erlang functions. For unit testing we use the common_test library, which is part of Erlang/OTP. Some developers would suggest using eunit instead, but we are interested in not only unit testing, but also integration testing. Integration testing means that software modules or units are combined and tested as a group. It allows to test for requirements in performance, reliability and functionality. I believe common_test is better suited for integration testing. Having all unit tests and integration tests implemented with common_test framework makes it easier to execute and maintain.
We teach how to use common_test from unit tests to large scale testing. This covers unit test, testing with state, test groups and suites, how to configure tests, write specifications and do distributed testing. During the training we also cover the principles behind property-based testing. We use QuickCheck Mini to see how to write property-based tests: generators, properties, symbolic representation of tests, etc. This also briefly covers shrinking strategies to find errors. Property-based testing is not applicable to all testing scenarios, but it exceptionally good in protocol testing, finding edge cases and I think TDD with property-based testing can help build even better designs.
Paolo – In OOP, TDD usually goes hand in hand with mocking. I had the chance to use Meck for some of my projects and I really liked it, still many Erlang programmers don’t like mocks. What do you think about mocking in Erlang?
Michał – Mocks are good if you can’t setup the whole system, but would like to execute some unit tests anyway. While I believe it makes sense to write mocks in meck in many situations, I also like the approach of the team at Erlang Solutions, which is responsible for the MongooseIM project. For MongooseIM testing we use common_test and we setup the whole system up front. The same tests suites are run in CI against serveral different server configurations, ensuring MongooseIM behaves according to spec, regardless of the backends used for storage, etc.
The reason why some Erlang programmers don’t use mocks that much is because it is so easy to setup an Erlang/OTP system that it sometimes pays off to test against a real one, not a partially-mocked one.
I believe meck works great when testing library code, especially in tandem with QuickCheck. Here, the boundaries of your library are well-defined, and you can verify (for example) that a particular internal component always receives messages corresponding to the data that’s coming in from outside. Or, alternatively, you can mock some bit of internal state and verify that the whole system satisfies a set of properties when it’s in that particular state.
Paolo – In your opinion, what are the main resources (e.g. online documentation or books) an Erlang developer interested in TDD should take a look at to grasp these concepts in a better way?
Michał – Each Erlang/OTP library has a user’s guide and for me it is often a starting point when I need to learn something new. For example you can find the Common Test User’s Guide at http://www.erlang.org/doc/apps/common_test/users_guide.html
For property-based testing there are quite a few articles on Erlang Central, see here http://erlangcentral.org/wiki/index.php?title=Category:QuickCheck
The best way to grasp these concepts is to practice them. One can contribute a test case to an open source project or try TDD with his/her next hobby project and see what happens. When you try TDD for the first time there will be a lot of questions arising and then you are sort of forced to find an answer to your questions – that is the best way to understand the concept. Ideally you would have someone experienced sitting next to you and answering your questions, but the social networks at Erlang Central or IRC channels are also good fellows.
Paolo – There aren’t many books or blog posts about best practices or real world examples of TDD and testing in Erlang. (one of my favourite is Gianfranco Alongi’s blog). Why do you think there are so few information about the topic available to the masses?
Michał – I’m not sure how to answer this question. Instead, let’s improve the situation 🙂