<div dir="ltr"><div><div><div>Hi Vicente,<br><br></div>Thank you for your quick reply.<br><br></div>I'm ready to retest the patch.<br><br></div>Regards,<br><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Nov 28, 2013 at 11:07 AM, Vicente Hernando <span dir="ltr"><<a href="mailto:vhernando@systemonenoc.com" target="_blank">vhernando@systemonenoc.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Hello,<br>
<br>
I think you have discovered a bug I made using variadic functions.<br>
<br>
Very soon I gonna send a patch to correct it.<br>
<br>
<br>
Thanks,<br>
Vicente.<div><div class="h5"><br>
<br>
On 11/28/2013 10:14 AM, Tuan Viet Nguyen wrote:<br>
</div></div></div><div><div class="h5">
<blockquote type="cite">
<div dir="ltr">Hello Vicente,<br>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">Thank you for your reply, you'll find
my answer below<br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Nov 28, 2013 at 12:03 AM,
Vicente Hernando <span dir="ltr"><<a href="mailto:vhernando@systemonenoc.com" target="_blank">vhernando@systemonenoc.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Hello,<br>
<br>
also full steps to crash kamailio and reproduce the
error would be good.</div>
</div>
</blockquote>
<div><br>
</div>
<div>Here is the architecture<br>
<br>
</div>
<div>A <--> Asterisk <--> Kamailio 1 <--->
kamailio2 <--- ISP---> mobile<br>
<br>
</div>
<div>Kamailio 1 & 2 are connected to a local redis
server<br>
</div>
<div>1/ I restarted the redis server<br>
</div>
<div>2/ From the mobile I made a call to A then cancelled
it. In the script of kamailio1, if a call has missed or
failed, it sends a message to the redis. And in this case,
it crashes<br>
<br>
</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>
<div><br>
<br>
<br>
On 11/27/2013 11:35 PM, Daniel-Constantin Mierla
wrote:<br>
</div>
</div>
<div>
<blockquote type="cite"> Hello,<br>
<br>
can you give the full output for 'bt full' with gdb
on the core file? You gave only partial list of the
frames, not being enough to see the execution trace.<br>
<br>
Cheers,<br>
Daniel<br>
<br>
<div>On 11/27/13 6:52 PM, Tuan Viet Nguyen wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>Hello,<br>
<br>
</div>
I'll try to shut down the redis server
to test the behavior of kamailio and it
has crashed if a call is received and
then cancelled. <br>
<br>
<b>1/The kamailio version is 4.0.4</b><br>
<br>
</div>
<div><b>2/ Kamailio log </b><br>
</div>
<div>/usr/local/sbin/kamailio[25333]:
ERROR: ndb_redis [redis_client.c:364]:
redisc_exec(): Redis error: Server
closed the connection<br>
/usr/local/sbin/kamailio[25361]: :
<core> [pass_fd.c:293]:
receive_fd(): ERROR: receive_fd: EOF on
13<br>
/usr/local/sbin/kamailio[25328]: ALERT:
<core> [main.c:788]:
handle_sigs(): child process 25333
exited by a signal 11<br>
/usr/local/sbin/kamailio[25328]: ALERT:
<core> [main.c:791]:
handle_sigs(): core was generated<br>
<br>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</blockquote>
</div>
I assume you disconnect redis server and don't reconnect
it. It is that correct?<br>
<br>
Then this line is an error but it should recover from
that. I probably should set this as a warning instead an
error.
<div><br>
/usr/local/sbin/kamailio[25333]: ERROR: ndb_redis
[redis_client.c:364]: redisc_exec(): Redis error:
Server closed the connection<br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Yes, it has been restarted<br>
<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div> <br>
</div>
<div>
<blockquote type="cite">
<blockquote type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div> </div>
<u><b>3/ Interesting information in the
core</b></u><br>
#3 0x00007fc79412893d in redisvCommand
(c=0x64657461, format=0x9 <Address 0x9
out of bounds>, ap=0x30,
ap@entry=0x7fff0ff56aa8) at hiredis.c:1304<br>
No locals.<br>
#4 0x00007fc794341713 in redisc_exec
(srv=srv@entry=0x7fff0ff56be0,
res=res@entry=0x7fff0ff56c00,
cmd=cmd@entry=0x7fff0ff56bf0) at
redis_client.c:368<br>
rsrv = 0x7fc794565150<br>
rpl = 0x7fc7946fab70<br>
c = 0 '\000'<br>
ap = {{gp_offset = 48, fp_offset =
48, overflow_arg_area = 0x7fff0ff56bb0,
reg_save_area = 0x7fff0ff56ac0}}<br>
__FUNCTION__ = "redisc_exec"<br>
#5 0x00007fc79433b781 in w_redis_cmd5
(msg=<optimized out>,
ssrv=<optimized out>,
scmd=<optimized out>,
sargv1=<optimized out>,
sargv2=0x7fc7946f7bf0
"p\243_\224\307\177", sres=0x7fc7946f7c50
" \253_\224\307\177") at
ndb_redis_mod.c:250<br>
s = {{s = 0x7fc7945fb300
"kamailio_redis", len = 14}, {s =
0x7fc7945f5f50 "PUBLISH %s %s", len = 13},
{s = 0x7fc7945fab20 "r", len = 1}}<br>
arg1 = {s = 0x7fc7945f5f80
"notification", len = 12}<br>
arg2 = {<br>
s = 0x7fc794551c60 "info
XXX"..., <br>
len = 212}<br>
c1 = 0 '\000'<br>
c2 = 0 '\000'<br>
__FUNCTION__ = "w_redis_cmd5"<br>
<br>
<br>
</div>
</div>
</div>
</div>
</blockquote>
</blockquote>
</div>
In the source code:<br>
<br>
rpl->rplRedis = redisvCommand(rsrv->ctxRedis,
cmd->s, ap );<br>
if(rpl->rplRedis == NULL)<br>
{<br>
/* null reply, reconnect and try again */<br>
if(rsrv->ctxRedis->err)<br>
{<br>
LM_ERR("Redis error: %s\n",
rsrv->ctxRedis->errstr);<br>
}<br>
if(redisc_reconnect_server(rsrv)==0)<br>
{<br>
rpl->rplRedis =
redisvCommand(rsrv->ctxRedis, cmd->s, ap);<br>
}<br>
}<br>
<br>
First redisvCommand executes but returns nothing. Then
it shows a redis error.<br>
<br>
It tries to reconnect and it manages to connect ??
because it shows no more errors.<br>
<br>
And then executes redisvCommand again and crashes.<br>
<br>
If server is down it should not be able to connect and
so not to execute redisvCommand again.<br>
</div>
</blockquote>
<div><br>
</div>
<div>According to the core, we MUST be in this case <b>if(redisc_reconnect_server(rsrv)==0)<br>
</b></div>
<div>But I am wondering how the first redisvCommand can
succeed before the reconnection ? (the connection
kamailio1 <-> redis has already been taken down).
Does all the redis context always there when we first call
redisvCommand? <br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> <br>
<br>
May be I would get more clues with more information.<br>
<br>
Regards,<br>
Vicente.</div>
</blockquote>
<div><br>
</div>
<div>Thank you<br>
</div>
<div>Regards,<br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div><br>
<br>
<blockquote type="cite">
<blockquote type="cite">
<div dir="ltr">
<div>
<div>
<div> </div>
I've found one of post that this issue has
been fixed but it seems that it's always the
case ..<br>
<a href="http://www.mail-archive.com/search?l=sr-users@lists.sip-router.org&q=subject:%22Re%3A+%5BSR-Users%5D+ndb_redis+module+fails+after+a+while%22" target="_blank">http://www.mail-archive.com/search?l=sr-users@lists.sip-router.org&q=subject:%22Re%3A+%5BSR-Users%5D+ndb_redis+module+fails+after+a+while%22</a><br>
<br>
</div>
Do you have any idea? <br>
</div>
Thank you<br>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
sr-dev mailing list
<a href="mailto:sr-dev@lists.sip-router.org" target="_blank">sr-dev@lists.sip-router.org</a>
<a href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev" target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev</a>
</pre>
</blockquote>
<br>
<pre cols="72">--
Daniel-Constantin Mierla - <a href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a></pre>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
sr-dev mailing list
<a href="mailto:sr-dev@lists.sip-router.org" target="_blank">sr-dev@lists.sip-router.org</a>
<a href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev" target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev</a>
</pre>
</blockquote>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br></div></div>