<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Hello,</p>
<p>(cross-posting because the details are useful for both devels and
users)</p>
<p>Yesterday during the IRC meeting about Kamailio, one group of
topics was about Kamailio 5.0, and the new feature of using
embedded interpreters for external programming languages was
highly debated. So I am trying to clarify some of its aspects.<br>
</p>
<br>
The new framework is referenced as kemi, chosen by me as an
abbreviation for Kamaialio EMbedded Interface.<br>
<br>
We already had embedded interpreters for many years, respectively:<br>
- Lua via app_lua module (probably most developed and most used)<br>
- Perl via app_perl module (iirc, the oldest embedded interpreter)<br>
- Python via app_python (I discovered during the last days that it
has quite a lot of features, just not documented -- never used it
before)<br>
- Java via app_java (this one is a bit of a blackhole for me,
never looked at it, don't know what it offers/how it is supposed to
be used)<br>
- .NET group of languages via app_mono (e.g., C#, but also some
variants of JavaScript, Python and maybe even Java)<br>
<br>
To clarify, an embedded interpreter means that Kamailio is linking
to itself the interpreter of that language and kamailio becomes the
interpreter of the language. It is not launching the standard
interpreter for that language, therefore it is very fast at runtime.
Moreover, kamailio is extending the language with new modules,
giving access to Kamailio C functions. In other words, for example
with Python, Kamailio becomes equivalent to the "python" application
plus two extensions (modules) named Router (developed in the past
inside app_python) and KSR (developed for kemi).<br>
<br>
At this moment, the kemi was implemented in app_lua and app_python
modules.<br>
<br>
To understand what is all about, I am going to give an example with
a function from maxfwd module. In default kamailio.cfg next
statement is present:<br>
<br>
if (!mf_process_maxfwd_header("10")) {<br>
sl_send_reply("483","Too Many Hops");<br>
exit;<br>
}<br>
<br>
mf_process_maxfwd_header("10") is the function exported by maxfwd:<br>
<br>
-
<a class="moz-txt-link-freetext" href="https://www.kamailio.org/docs/modules/stable/modules/maxfwd.html#maxfwd.f.mf_process_maxfwd_header">https://www.kamailio.org/docs/modules/stable/modules/maxfwd.html#maxfwd.f.mf_process_maxfwd_header</a><br>
<br>
The parameter can be as a static number (like above) or a number
provided via a variable, like next:<br>
<br>
$var(p) = 10;<br>
mf_process_maxfwd_header("$var(p)")<br>
<br>
The corresponding C code for this config function is:<br>
<br>
static int w_process_maxfwd_header(struct sip_msg* msg, char* str1,
char* str2)<br>
{<br>
int mfval;<br>
if (get_int_fparam(&mfval, msg, (fparam_t*) str1) < 0) {<br>
LM_ERR("could not get param value\n");<br>
return -1;<br>
}<br>
return process_maxfwd_header(msg, mfval);<br>
}<br>
<br>
The function get_int_fparam() is practically returning the int value
of the parmeter, no matter it was a static value or a variable, then
process_maxfwd_header(...) is executed with the int parameter. The
msg is the SIP message structure present mostly everywhere in the C
code related to the classic config interpreter.<br>
<br>
To be used inside the config file, the C function has to be exported
via a cmd_export_t structure added to "struct module_exports
exports", respectively:<br>
<br>
static cmd_export_t cmds[]={<br>
...<br>
{"mf_process_maxfwd_header",
(cmd_function)w_process_maxfwd_header, 1,<br>
fixup_var_int_1, 0, REQUEST_ROUTE},<br>
...<br>
<br>
To get the same function in a kemi interpreter (Lua, Python at this
moment), following C code was added:<br>
<br>
static sr_kemi_t sr_kemi_maxfwd_exports[] = {<br>
{ str_init("maxfwd"), str_init("process_maxfwd"),<br>
SR_KEMIP_INT, process_maxfwd_header,<br>
{ SR_KEMIP_INT, SR_KEMIP_NONE, SR_KEMIP_NONE,<br>
SR_KEMIP_NONE, SR_KEMIP_NONE, SR_KEMIP_NONE }<br>
},<br>
<br>
{ {0, 0}, {0, 0}, 0, NULL, { 0, 0, 0, 0, 0, 0 } }<br>
};<br>
<br>
int mod_register(char *path, int *dlflags, void *p1, void *p2)<br>
{<br>
sr_kemi_modules_add(sr_kemi_maxfwd_exports);<br>
return 0;<br>
}<br>
<br>
The structure sr_kemi_t sr_kemi_maxfwd_exports exports directly the
existing function process_maxfwd_header(), which is also executed by
the function from kamailio.cfg after resolving the parameter to an
int value. In the case of kemi, the Lua or Python will give directly
an integer value, so no need for resolving that with a fixup
wrapper.<br>
<br>
The mod_register() function is something already existing, providing
a way to execute code when a module was loaded (e.g., is used for
many years by modules such as tls, mysql, etc.). In this case it was
added, because maxfwd didn't need anything at load time before.<br>
<br>
Practically, no C code extension was written to maxfwd module from
point of view of SIP routing. Just an existing function was
exported.<br>
<br>
Now, what's actually the role of this kemi framework ...<br>
<br>
The app_lua already exported the kamailio.cfg function
mf_process_maxfwd_header() as sr.process_maxfwd() since Kamailio
v3.1, see its usage in the tutorial:<br>
<br>
- <a class="moz-txt-link-freetext" href="https://kb.asipto.com/kamailio:usage:k32-lua-routing#lua_script">https://kb.asipto.com/kamailio:usage:k32-lua-routing#lua_script</a><br>
<br>
But a Lua specific wrapper had to be written:<br>
<br>
-
<a class="moz-txt-link-freetext" href="https://github.com/kamailio/kamailio/blob/master/modules/app_lua/app_lua_exp.c#L1327">https://github.com/kamailio/kamailio/blob/master/modules/app_lua/app_lua_exp.c#L1327</a><br>
<br>
All the other embedded interpreters would have needed to get a
wrapper as well to export this function. That's obviously costing
time and maintenance resources.<br>
<br>
So kemi is a common framework to export C function to any of the
embedded interpreters, meaning that once a function is exported from
a module, it becomes available to all embedded interpreters, no
longer being necessary to write a specif wrapper per interpreter.
The app_* module needs a bit of coding to use the kemi exported
functions (already done for Lua and Python), but then nothing needs
to be done for new exported functions by modules.<br>
<br>
As a matter of fact, lots of functions can be just exported, as you
could see in the example with maxfwd. In some cases, maybe some
functions need to be split, so the fixup handling is done separately
and then calls a common C function.<br>
<br>
So, what we had so far:<br>
<br>
[native kamailio cfg script] <===> [fixup framework for
variables] <===> [C implementation of function]<br>
<br>
The above stays in place, no change to it. What we got extra:<br>
<br>
[embedded language script] <===> [kemi framework] <===>
[C implementation of function]<br>
<br>
In addition, kemi allows to write the routing blocks all in embedded
language. Core parameters, loading modules and modules' parameters
stay like so far, but there is no need to write request_route{},
failure_route, etc.. blocks. Some functions with a specific names
have to be written in the embedded language script, see more at:<br>
<br>
- <a class="moz-txt-link-freetext" href="https://www.kamailio.org/wiki/devel/config-engines">https://www.kamailio.org/wiki/devel/config-engines</a><br>
<br>
In several days I expect to be possible to have routing blocks of
kamailio-basic.cfg written via kemi interpreters. That will allow
testing the performance impact.<br>
<br>
Right now Lua and Python use different mechanism of exporting
Kamailio C functions, the one from Python should be faster, but
relies on defining some C wrappers inside the module. Anyhow Lua
might be faster as interpreter itself -- and at the end it can just
get the same mechanism as Python.<br>
<br>
app_lua already supports reloading the routing script without
restarting kamailio, one of the features asked from time to time.<br>
<br>
Besides that, of course, a big benefit is the access to a very large
set of extensions already available as Lua/Python libraries. Also
important, many peoples are already familiar with these languages
and there is plenty of good documentation about those languages.<br>
<br>
The classic/native kamailio.cfg format (and the interpreter for it)
stays there -- same common C functions are shared with kemi, so any
addition in the future will be visible to everywhere. The native
interpreter will remain the option for extreme optimizations for
those that deal with enormous amount of SIP traffic (e.g., heavy
load balancers), but I expect that at least Lua will be in pair of
performances with a registrar deployment.<br>
<br>
I hope I could shed some light on various aspects of the new
framework for embedded interpreters. Expect very soon news about
ability to build full routing logic in Lua/Python, hopefully many of
you will join to test the results and compare with native
alternatives.<br>
<br>
Cheers,<br>
Daniel<br>
<br>
<pre class="moz-signature" cols="72">--
Daniel-Constantin Mierla
<a class="moz-txt-link-freetext" href="http://www.asipto.com">http://www.asipto.com</a>
<a class="moz-txt-link-freetext" href="http://twitter.com/#!/miconda">http://twitter.com/#!/miconda</a> - <a class="moz-txt-link-freetext" href="http://www.linkedin.com/in/miconda">http://www.linkedin.com/in/miconda</a>
Kamailio World Conference, Berlin, May 18-20, 2016 - <a class="moz-txt-link-freetext" href="http://www.kamailioworld.com">http://www.kamailioworld.com</a></pre>
</body>
</html>