Reliability with Erlang
October 31st, 2007 | Published in column, erlang | 9 Comments | Bookmark on Pinboard.in
You’ve probably heard a lot by now about Erlang’s concurrency capabilities, and in fact that’s what I covered in my Sept./Oct. Internet Computing “Toward Integration” column, Concurrency with Erlang. But concurrency is only part of the story — Erlang also provides outstanding support for building highly-reliable software. My latest column, Reliability with Erlang, first describes some of the problems that highly-reliable systems face, and then explains some of Erlang’s core primitives that provide a solid foundation for reliable systems. It’s available in HTML and PDF formats.
BTW, I can’t recommend enough that you pick up a copy of Joe Armstrong’s Programming Erlang. It’s a truly excellent book that has deeply and positively affected the way I think about designing, building, and implementing software systems. As I mentioned in my columns, my only disappointment with Erlang is that I didn’t discover it 10 years ago when it was first open-sourced, as it could have saved me a ton of time and trouble in my middleware development efforts over the years.
November 1st, 2007 at 12:56 am (#)
Good article Steve, you infected me with this Erlang-mania. But i think Erlang is lacking kind of infrastructure. It lacks for IDE, lacks for community, lacks for testing capabilities. And if we try today to use it in real applications we must ‘Build a castle’ again. Today we need rapid solutions, and i’m not sure that Erlang can allow normal facilities to manufacture this solutions. Now erlang gaining populariry and maybe somewhen will occupy some ‘erlang niche’, but not today.
November 1st, 2007 at 7:39 am (#)
IDE based on Netbeans: http://sourceforge.net/projects/erlybird
EUnit: http://support.process-one.net/doc/display/CONTRIBS/EUnit
November 1st, 2007 at 11:57 am (#)
@Steve: If you enjoyed Joe’s book, you should definitely read his PhD thesis “Making reliable distributed systems in the presence of software errors”. Joe describes his “fail-fast” approach to handling errors. In short: you don’t even try to recover from errors, instead you let the process fail immediately and try again. This methodology is easy to follow in Erlang/OTP where pattern matching can be used to force the process to fail immediately and the OTP supervisors allow you to restart computations automatically. Joe’s thesis really opened my eyes and showed me an elegant way to handle errors.
http://www.sics.se/~joe/thesis/armstrong_thesis_2003.pdf
— Lauri
November 1st, 2007 at 2:47 pm (#)
RabbitMQ is an AMQP broker implementation in Erlang. So people are talking Erlang seriously.I haven’t played around with it, but it would be nice to see how well they scale ..etc.
One of my goals for this year is to learn Erlang.
November 1st, 2007 at 4:29 pm (#)
Do you have any thoughts on Erlang’s test frameworks? Something that put me off learning Lisp for a very long time was the lack of LispUnit, but I eventually persevered with Peter Seibel’s excellent book. I’m going through Joe’s book now, and all I’ve found thus far is “write a makefile that exercises the code with test code”. It feels a bit far off JUnit, ANT and Ivy, and that worries me. Should it?
November 1st, 2007 at 7:11 pm (#)
I prefer ErUnit over EUnit. It was simply easier for me to start using.
http://bestfriendchris.com/blog/2007/04/23/erunit-unit-testing-for-erlang/
Not sure about feature comparison. I don’t demand much of a unit test framework.
November 2nd, 2007 at 6:12 pm (#)
I’ve built strategized fast-fail into asynchronous FSMs in trading platforms as a default transition. When you have 40+ states and 100s of state-changing transitions in a state machine it gets hard to test every single possible race condition due to the size of the state space involved.
Either way Steve/Lauri you’ve both convinced me to pick up the book! Also, it seems FSM’s are very well supported in erlang:
http://portal.acm.org/ft_gateway.cfm?id=1292529&type=pdf&coll=GUIDE&dl=&CFID=15151515&CFTOKEN=6184618
November 6th, 2007 at 7:42 am (#)
Steve,
Nice articles, thanks. I’m trying to learn Erlang, and having fun.
One entertaining exercise was trying to speed up pow.
Obviously, doing pow linearly shouldn’t be faster than doing a “binary exponentiation” pow computation.
http://en.wikipedia.org/wiki/Exponentiation_by_squaring
What was driving me crazy is that although this algorithm is O(lg n) and the other pow is O(n), I had a heck of a time getting a powers of 2 implementation that beat the linear one in erlang.
I had to “unroll” the last few levels of my function to explicitly match exponents 1, 2, and 3 before this approach started to win.
This final implementation wins big, though, and also has a big advantage in terms of the largest power that can be computed. Trying to compute pow(5000,50000) on my mac runs out of memory, but geometric_pow(5000,50000) computes it in
2.7 seconds.
Here’s the implementation:
% Licensed under Artistic License, see
% http://www.perl.com/pub/a/language/misc/Artistic.html
% Mike Giroux, rmgiroux@gmail.com
-module(geometric_pow).
-export([geometric_pow/2]).
geometric_pow(_, 0, Acc) -> Acc;
% It’s interesting that if 1 and 2 aren’t “unrolled”,
% the linear implementation beats geometric_pow.
geometric_pow(Square, 1, Acc) -> Square*Acc;
geometric_pow(Square, 2, Acc) -> Square*Square*Acc;
geometric_pow(Square, 3, Acc) -> Square*Square*Square*Acc;
geometric_pow(Square, 4, Acc) -> Square*Square*Square*Square*Acc;
geometric_pow(Square, Exp, Acc) ->
geometric_pow(Square*Square, Exp div 2, ((Exp rem 2)*(Square-1)*Acc + Acc)).
geometric_pow(Base, Exponent) ->
geometric_pow(Base,Exponent,1).
November 7th, 2007 at 1:29 pm (#)
PS: Yes, I realize that without the unrolling, Square*Square will be computed once too often by the “generic” geometric_pow/3. But the slowness happened even with versions of geometric_pow/3 that didn’t have that problem. Very strange…