An interview with Stavros Aronis about #erlang and Dialyzer
What’s up Erlang addicts? Here is another interview you may find of interest.
In the post I prepared for you today you will learn something about Dialyzer from Stavros Aronis. Stavros is a PhD student at Uppsala University and will be one of the speakers at the upcoming EUC 2013. The talk will be: “Parallel Erlang – Speed beyond Concurrency”. Hope you will enjoy it!
Dialyzer is your friend!
Paolo – Hi Stavros. Thanks for accepting my interview. Would you like to introduce yourself to our readers?
Paolo – Your experience with Erlang started in Greece. Can you tell us something about your first projects with Erlang?
Stavros – I didn’t know anything about the language until the final years of my studies at the National Technical University of Athens (NTUA). I was looking for an interesting topic for my diploma thesis and I turned for suggestions to Kostis Sagonas, who was then the head teacher of the Programming Languages courses there. I don’t think that I need to introduce Kostis here, as he is evidently popular in the Erlang community! At that time, Kostis had quite a few projects for diploma thesis students (I was working together with the students who developed the 0.1 versions of PropEr and Concuerror) and I decided to work on Dialyzer. My very first task was to implement support for the callback attributes and add them to the behaviors in the OTP distribution. After that I worked for a while on extending Dialyzer’s detection of race conditions to work on code that uses behaviors, but then I changed my focus and worked on enhancing Dialyzer’s type inference algorithm so that it could detect errors that were not possible to catch before.
Paolo – Currently you are a PhD student at Uppsala University, a place widely known by Erlangers. Can you give us some insight about the researches you do there?
Stavros – My current research is on Concuerror, a tool for exploring all the possible ways that the processes of an Erlang program can be interleaved during scheduling. In the simplest terms, you give a test to Concuerror and it returns to you a scheduling scenario that makes one of your processes throw an uncaught exception or leads your processes to a state where they are all waiting for some message and no progress can be made. If Concuerror is unable to do either, then there is no possible scheduling of your test that can lead to these generally undesired states. Kostis’ presentation in the Erlang User Conference 2013 will be on Concuerror.
There are two more PhD students working on Erlang in our group in Uppsala University, David Klaftenegger and Kjell Winblad. They will also be in the EUC’13, presenting their research on improving the concurrent performance of ETS tables.
Paolo – At Erlang User Conference 2013 you will give a talk about the parallel use of Erlang and the tool Dialyzer. Can you provide a brief description of your talk? Why should we parallelize Dialyzer?
Stavros – My talk will be about my experience from parallelizing Dialyzer, work that was included in OTP R15B02. It was a task that I already wanted to work on in Greece, as for the evaluation of my diploma thesis I wanted to run Dialyzer on the entire Erlang/OTP codebase to see whether I would catch any new errors. The extension I was developing made Dialyzer quite slower (this is by the way the reason it has not yet been included in OTP), so having to run it two times to compare results was already time consuming. I remember my frustration back then, as I was watching only one of the processors of my dual core laptop do all the work while the other was idling! With Erlang having such a wonderful support for concurrency, it was obvious that parallelizing Dialyzer should not be a very challenging task. The real story of course was a little different, with some interesting twists which I want to share with the other participants of the conference.
Paolo – Who should follow your talk and why? Is the talk only for experienced Erlang developers?
Stavros – Not at all! The talk has no real requirements, other than elementary understanding of Erlang’s concurrency primitives (spawn, send, receive and inserts/lookups on public ets tables). I want to show how easy it is using these primitives to parallelize an algorithm in a very natural way and what unique caveats you may run into.
Paolo – When should we use Dialyzer?
Stavros – At the very least, every time you want to commit changes on any Erlang project! Dialyzer is (famously) a totally automatic, “push button” tool, that you configure easily, just once, with the code that you depend on and trust to be correct, and then you are good to go. It is not a coincidence that celebrated members of the Erlang community, like Loic Hoguin, require that any contributions to their projects produce no Dialyzer errors. Dialyzer will catch many of the errors that you would otherwise have to write tests for (e.g. an erroneous call to a function in a rarely reached path in your code) and will also catch many that you could not detect otherwise (e.g. dead code, wrong specs).
Paolo – How much time/effort can we now expect when running Dialyzer?
Stavros – As I said, Dialyzer is fully automatic so it shouldn’t really require any effort to use. You just give your modules as input and you get warnings that are ‘always right’. Dialyzer will never produce a warning if there is no real issue. Fixing the problem is the only part where you really spend effort.
Regarding the time it takes, Dialyzer’s analysis is very much dependent on the structure of your code. On our most difficult real test, analyzing the entire OTP distribution, using the parallel version we managed to bring execution time from 1 hour 20 minutes down to 6 minutes (13x faster) on a 32 core machine. Scalability is very good on desktop machines as well: the aforementioned test scales linearly up to 4 cores.
Paolo – Last question: what do you see in the future of Erlang and Dialyzer? Is it possible to improve what we have now and if so how?
Stavros – The recent research in our group has shown that there is still plenty of room for improvements in the performance of the Erlang VM on big multicore machines. Operations of ETS tables are already being optimized and we plan to target the schedulers next. Uppsala University is a partner of the RELEASE (http://www.release-project.eu/) project, that aims to improve the scalability of the Erlang VM on thousands of cores. Ericsson is also a partner of RELEASE, so the OTP team is also contributing significantly with improvements in other parts of the VM.
Dialyzer has currently no major updates scheduled. However, I will at some point find some time to update and include my diploma thesis contributions in the OTP, so that Dialyzer will be able to catch even more errors, before you have to really think about them!