Archive

Posts Tagged ‘XMPP’

An interview with Michal Ptaszek (@michalptaszek)

June 3, 2011 Leave a comment

This is one of the last interviews I made to the speakers of  Erlang Factory London 2011. One of my favourite erlang developers is Michal Ptaszek. Michal works at Erlang Solutions Ltd in Krakow. He is pretty famous among ejabberd developers and XMPP lovers.

Ask and Aswer

Paolo – Michal, can you introduce yourself to our readers?
Michal – Hello, I am Michal Ptaszek – Erlang passionate and evangelist, someone who became seduced by the beauty of logic of maths and algorithms in his early childhood. Currently I work for Erlang Solutions, trying to cope with the complexity of massive scale Internet Messaging systems. From personal perspective, I have done my masters in Krakow University of Science and Technology in Grid systems field (Erlang obviously!), currently I live in Krakow, Poland, love jazz and modern literature (especially latin). Some day would like to find some time to learn how to play sax.

Paolo – How did you “discover” Erlang?
Michal – It has started in the early 2008, when Joe Armstrong came to Poland to give a talk on Erlang concepts for Student’s IT Festival. Of course almost no one has ever heard of the language (or. even if, never touched it), and as Joe is such a great speaker, he planted some seeds of curiosity in my mind.

Paolo – How long did it take to you to master Erlang?
Michal – I wouldn’t say I’ve mastered Erlang. Comparing to such people as Ulf Wiger or Robert Virding, I’m still in my infancy 🙂 The great thing in Erlang is that you do not have tons of code everywhere: language is simple, standard libraries are manageable, even VM: if you clench your fists you can read and understand the entire code in few weeks.
In my opinion people are in never-ending continuos process of developing themselves: when I look at my code written a year/two ago I feel ashamed of myself and would love to throw it away and rewrite it in some nicer way. I guess at the time when I discover there is nothing to improve I will be forced to change the profession (or at least the programming language) 😉

Paolo – How did you find your first Erlang related job?
Michal – Few months later after Joe’s visit to Poland, Erlang Solutions (Erlang Training and Consulting back then) decided to open an office in Krakow: I did not hesitate a lot and applied for the internships. The office grew from 3 to 12 people since then and is extending all the time: I guess it was a really good choice.

Paolo – What are the things you love most of Erlang?
Michal – Just to name two most important things:
– built-in parallelism and distribution – I can not even imagine how do people distribute their applications and protect their critical sections using semaphores and mutexes. It’s so 1990…
– fault tolerance – one of the least appreciated things in the language: ability to isolate the errors, propagate them in the controlled way and recover automatically. In the systems that run on top of tens/hundreds of machines crash of one of them is not a question of ‘if’ but ‘when’.

Paolo – During Erlang Factory, you will talk about Erlang and XMPP: can you give to our readers a brief description of your course?
Michal – Yes, together with Michal Slaski we are going to teach a one day course on Erlang and XMPP (ejabberd in particular). During those few hours we would like to show how the things work under ejabberd’s hood, why it is not a best idea to keep the default configuration, what are the
low hanging fruits for ejabberd, BEAM and OS enhancements, how can we monitor existing server, how to cluster several daemons together in one command and finally: what are the ways to extend the functionality programmatically.

Paolo – Who should follow your course and why?
Michal – Definitely people who are interested in Internet Messaging, even not necessarily in XMPP. Architecture of message/presence router, as well as the easiness of extending current functionality, federating with other protocols or finally scalability might be stimulative to someone who
plans to set up a startup or replace his current messaging server with something better.

Paolo – Why should an XMPP programmer use Erlang as development language?
Michal – Mostly because of the concurrency and scalability. Erlang actor process model fits perfectly in the concept of independent user sessions and communicating with each other using asynchronous message passing.

Paolo – In your opinion, what is the most important thing to keep in mind when developing with Erlang and XMPP?
Michal – Global state/shared data is a root of all evil. The less things you share – the better you can scale. XMPP systems have tendencies to grow rapidly: it’s basically the linear relation to the number of simultaneously connected users. Shared global state will not pop up as a bottleneck until it is too late: when users are storming our gates and we realize that things should have been done in a different way long time ago.
And there is also one more rule, that people often forget about: KISS. IM systems are really simple, let’s do not overengineer them

Paolo – Can you recommend to our readers some books, sites, or blogs that will help them in the world of Erlang and XMPP?
Michal – Definitely take a look at Jack Moffitt’s both book (“Professional XMPP Programming”) and webiste (http://metajack.im/). Another worth-reading blog is Stefan Strigler’s http://blog.jwchat.org/. To be honest: there is still no book on Erlang and XMPP in particular. Maybe it is a good time to change it? 🙂

Advertisements

Testing your XMPP external component using Tsung and ejabberd part 2

August 1, 2010 5 comments

In the previous post we discussed ow to enable anonymous authentication in ejabberd in order to use Tsung for our testing purposes.

Tsung (a.k.a. Tsunami) is a shell testing tool fairly easy to use when dealing with XMPP, it is mainly used to simulate thousands of users for testing the scalability and performances of IP based client/server applications.

For XMPP test purposes it allows:

  • Authentication (plain-text, digest and sip-digest)
  • presence and register messages
  • Chat messages to online or offline users
  • MUC: join room, send message in room, change nickname
  • Roster set and get requests
  • Global users' synchronization  set on specific actions
  • raw XML messages
  • PubSub
  • Multiple vhost instances support

First thing first: install Tsung! Download the source from http://tsung.erlang-projects.org and just follow the common procedure configure, make, make install (you may need sudo).

You may check whether the installation was successful by typing:

paolo@blog $ tsung -h
Usage: tsung  start|stop|debug|status
Options:
    -f   set configuration file (default is ~/.tsung/tsung.xml)
    -l   set log file (default is ~/.tsung/log/tsung.log)
    -i   set controller id (default is empty)
    -r   set remote connector (default is ssh)
    -F   use long names (FQDN) for erlang nodes
    -v   print version information and exit
    -h   display this help and exit

When you start Tsung (we will see how) it executes a set of operation based on an xml file (tsung.xml) that you easily configure by using a text editor. This file is inside the directory  ~/.tsung but sometimes it is not present so I suggest you to copy one of the examples you can find in the directory “examples” of the installation package, copy it there and rename it.

The Great Wave Of Kanagawa

Let’s take a look at tsung.xml.

Scenarios start with clients (Tsung cluster) and server definitions, so for a non-distributed load, you can use a basic setup like:

<clients>
    <client host="client-ip" use_controller_vm="true"/>
</clients>

<servers>
    <server host="server-ip" port="80" type="tcp"></server>
</servers>

You can put more than one client or one server in you tsung.xml, using either a string or an ip-addres to identificate them. In case of multiple servers, a round robin algorithm will be used to chose a server.

Let’s define now a load progression by specifing a set of  arrival phases to the server:

<load>
  <arrivalphase phase="1" duration="1" unit="hour">
    <users interarrival="6" unit="second"></users>
  </arrivalphase>

  <arrivalphase phase="2" duration="40" unit="minute">
    <users interarrival="1" unit="second"></users>
  </arrivalphase>

  <arrivalphase phase="3" duration="5" unit="minute">
    <users interarrival="0.1" unit="second"></users>
  </arrivalphase>
</load>

Above we specified 3 phases:

  1. duration 1 hour, 1 user each 6 seconds will start a session
  2. duration 40 minutes, 1 user every 1 second
  3. duration 5 minutes, 1 user every 0.1 seconds

The phases will happen one after the other and the test will finish when all users have ended their session. Notice that duration of the test can be much longer than the duration of arrival phases, therefore if you want to stop the test after a threshold duration (even if phases are not finished or if some sessions are still active), you can do by setting a duration attribute to load:

<load duration="4" unit="hour">
 .......
</load>

You can then edit the options as follows:

<options>
  <option type="ts_jabber" name="global_number" value="5"></option>
  <option type="ts_jabber" name="userid_max" value="1000"></option>
  <option type="ts_jabber" name="domain" value="yourdomain"></option>
  <option type="ts_jabber" name="username" value="yourusername"></option>
  <option type="ts_jabber" name="passwd" value="yourpwd"></option>
</options>

In the next post (the final part of this tutorial on Tsung) we will see how to deal with sessions and how to run a test and evaluate its results.

Categories: English, Erlang Tags: , ,

Testing your XMPP external component using Tsung and ejabberd part 1

July 21, 2010 Leave a comment

In many previous posts, I gave you some ideas on how to build an external component for XMPP servers; now it is time to test your work!

Tsung is a stress testing tool written in Erlang that you can use to test HTTP, WebDAV, LDAP, MySQL, PostgreSQL, SOAP and XMPP servers; it can simulate hundreds of simultaneous users on a single system in a single server or in a clustered environment therefore Tsung is the right tool to test our XMPP services.

First of all: when you use Tsung, you simulate few or many users….the problem is that you need to authenticate these users against the server, so you have two solutions: the first one is to create all the users…the second one (the smartest) is to enable anonymous authentication in your XMPP server.

Anonymous authentication is a standard login using classical login and password mechanisms, but where password is accepted or preconfigured for all anonymous users. This login is compliant with sasl authentication, password and digest non-sasl authentication. For more information see this.

On ejabberd, to provide anonymous authenticatio to users you have to tune your configuration file as follows:

1) set up two virtual hosts on the server, one public (any user can login) and one private (login only with valid credentials):

{hosts, ["private.yourdomain", "public.yourdomain"]}.

2) set the global authentication method that you will use. In our case we will  authenticate against ejabberd internal database, but alternative solutions can be MySQL, PostgreSQL, LDAP and so on:

{auth_method, internal}.

3) At the end override the host_config options to provide anonymous login to public host’s clients:

{host_config, "public.yourdomain", [{auth_method, anonymous},
                                     {anonymous_protocol, login_anon}]}.

Just start/restart your ejabberd from this point you can connect to the virtual host “public.yourdomain” with any pair (JID, pwd) but you will be able to authenticate against  “private.yourdomain” only with a valid pair.

Ok, now you have anonymous authentication enabled in you XMPP server!

This week I’m really busy 😦 , so I will teach you how to use Tsung and how to configure it for our purposes in the next post.

Sending and receiving stanzas using EXMPP

May 10, 2010 4 comments

A couple of posts ago, I introduced EXMPP, the Erlang library developed by Process One for XMPP.

In that post we used EXMPP to connect an external component to eJabberd server by using the XEP-0114.

This time I will show you how to process streams directed to our component, analyze them and reply to the sender.
First of all I used a gen_server behaviour and put all the code regarding the handshake inside the init([]) method:

init([]) ->
    exmpp:start(),
    Session = exmpp_component:start_link(),
    exmpp_component:auth(Session, ?COMPONENT, ?SECRET),
    _StreamId = exmpp_component:connect(Session, ?SERVER_HOST, ?SERVER_PORT),
    ok = exmpp_component:handshake(Session),
    {ok, #state{session = Session}}

You may notice that I used a set of Erlang macros to identify the variables needed for the connection,
those macros were previously defined in the code as:

-define(COMPONENT, get_conf_value(component)).
-define(SECRET, get_conf_value(secret)).
-define(SERVER_HOST, get_conf_value(server_host)).
-define(SERVER_PORT, get_conf_value(server_port)).

Where get_conf_value(Value) is simply a function I implemented to parse a configuration file.

EXMPP provides to developers a set of records (a record may be thought as a complex data structure and I will write about
them in the future); among these records we need in our code the one representing a received_packet.
It is defined as following:

-record(received_packet,
                   {
                   packet_type, % message, iq, presence
                   type_attr,   % depend on packet. Example: set, get, subscribe, etc
                   from,        % JID
                   id,          % Packet ID
                   queryns,     % IQ only: Namespace of the query
                   raw_packet   % raw exmpp record
}).

After the handshake we can send any kind of stanzas defined in XMPP realm towards the server we are connected to as long as
we have a reference to the session we established before.

A really cool feature of EXMPP is that two different processes spawend by our gen_server can send a stanza at the same time over the same session and the stanzas will not be mixed during their “trip” to the XMPP server.

About incoming stanzas, you must know that all of them will be processed by the gen_server as normal Erlang messages.

I told you before that all the stanzas can be handled by processes spawned by the gen_server, so it seems sensible to spawn
a new process any time a packet is received; this new process will elaborate the stanza and if required will send a reply stanza, then it
terminates.

To do that we add the gen_server something like this:

handle_info(#received_packet{} = Packet, #state{session = Session} = State) ->
            spawn(fun() ->
                          process_received_packet(Session, Packet)
                  end),
{noreply, State};

In practice whenever we receive a Packet we spawn (without linking!) a new process which takes as input the current session and the packet itself.
This process is represented by the following function:

process_received_packet(Session, Packet) ->
       Packet_Type = Packet#received_packet.packet_type,
       case Packet_Type of
          iq ->
               process_received_iq(Session, Packet);
          message ->
               process_received_message(Session, Packet);
          presence ->
               process_received_presence(Session, Packet);
       end.

The previous function is pretty simple: basically it checks the type of the packet and then based on it a different function is called.

Let’s implement process_received_message(Session, Packet) so that whenever a message stanza is received our component sends back the same message:

process_received_message(Session,
                      #received_packet{packet_type=message, raw_packet=Packet}) ->
    %% retrieve recipient jid
    From = exmpp_xml:get_attribute(Packet, from, <<"unknown">>),
    %% retrieve sender jid
    To = exmpp_xml:get_attribute(Packet, to, <<"unknown">>),
    %% set recipient jid
    Pkt1 = exmpp_xml:set_attribute(Packet, from, To),
    %% set sender jid
    Pkt2 = exmpp_xml:set_attribute(Pkt1, to, From),
    %% remove old id
    NewPacket = exmpp_xml:remove_attribute(Pkt2, id),
    %% send new packet
    exmpp_session:send_packet(Session, NewPacket).

Here is the result (captured by using PSI):

Message reply

Let’s  implement process_received_iq(Session, Packet) that sends back a simple message if an iq stanza is received:

process_received_iq(Session,
            #received_packet{packet_type=iq, type_attr=Type, raw_packet=IQ}) ->
    %% get namespace of IQ stanza
    NS = exmpp_xml:get_ns_as_atom(exmpp_iq:get_payload(IQ)),
    %% set text value of IQ stanza
    Reply = exmpp_xml:element(NS, 'response', [],
                                      [{xmlcdata,<<"your iq has been received">>}]);
    %% build result packet
    Result = exmpp_iq:result(IQ, exmpp_xml:element(NS, 'query', [], [Reply]));
    %% sends new packet
    exmpp_component:send_packet(Session, Result).

Ok! For now it is enough…try to read EXMPP specifications and see how you can change slightly this code!

Categories: English, Erlang Tags: , ,

Connecting an XMPP external component via Erlang and EXMPP

February 23, 2010 10 comments

Here I am again. Sorry for the time it took me to write a new blog entry, but I was preparing the speech for my bachelor thesis..so as you could understand that took all of my free time.
My thesis explained how social networking could be integrated in an XMPP environment, and I have to say that during the time it took me to design and write the code I became an XMPP nuts. XMPP is a protocol aimed to real-time messaging and presence, but it allows developers to use many extensions to implement different kind of services.
For my thesis I used as XMPP server the software Tigase and since it is developed in Java, I wrote all of my code in the same language, but I was aware of the fact that a XMPP server developed in Erlang and called eJabberd exists so I decided to port all of my code in Erlang.
In this post I will teach you how enable an external component in eJabberd and how to code it using Erlang and EXMPP a library provided by Process One. First of all we have to enable the external component in the eJabberd configuration file; by default you can find it in /etc/ejabberd, so:
bellerofonte@pegaso:~$ cd /etc/ejabberd/
belleforonte@pegaso:/etc/ejabberd$ ls
ejabberd.cfg  ejabberdctl.cfg  inetrc

At this point let’s edit ejabberd.cfg (you may be asked to do it with the sudo option).

bellerofonte@pegaso:~$ sudo nano ejabberd.cfg

Now you have to find the part of the configuration file that starts with:

%% ejabberd_service: Interact with external components (transports...)

And add the following lines to add two different external components:

  {8888, ejabberd_service, [
                          {access, all},
                          {shaper_rule, fast},
                          {ip, {127, 0, 0, 1}},
                          {hosts, ["weather.localhost", "weather.trentino.com"],
                           [{password, "weather"}]
                          }
                         ]},

  {8888, ejabberd_service, [
                          {access, all},
                          {shaper_rule, fast},
                          {ip, {127, 0, 0, 1}},
                          {hosts, ["news.localhost", "news.trentino.com"],
                           [{password, "news"}]
                          }
                         ]},

If you don’t know about XMPP and external component I can just tell you that the names I specified in the list after hosts represent the names that will be used to identify this component in the XMPP network, while the value of password is just the password used by the component to perform the handshake with the server. For more info you can read this.

At this point save the configuration file and restart eJabberd.

Now let’s use EXMPP to connect from the Erlang shell to eJabberd as if we were one of the two components.

bellerofonte@pegaso:/etc/ejabberd$ erl
Erlang R13B03 (erts-5.7.4) 

Eshell V5.7.4  (abort with ^G)
1> exmpp:start().
ok
2> Session = exmpp_component:start_link().
<0.59.0>
3> exmpp_component:auth(Session, "weather", "weather").
ok
4> _StreamId = exmpp_component:connect(Session, "localhost", 8888).
"1627139564"
5>  ok = exmpp_component:handshake(Session).
ok

In the code above, I first started  the exmpp application, then I created a Session for the XMPP component and made the authorization stream by passing to the function exmpp_component the session, the name used by the component and the password needed  for the handshake.

Then I connected it to the eJabberd server by using the function connect, which needs the session, the name of the server and the port on which it is listening to.

After that just connect the component by using the function handshake.

Now, you can log in to the server with one test account of yours and perform a service discovery, the result should be something like:

PSI: Service Discovery

Ok, I know this post was kinda short, but I promise that next one will be much better, and in it I will teach you how to receive and send XMPP stanzas from our component!

Categories: English, Erlang Tags: , ,