<div dir="ltr">Hi Daniel,<div><br></div><div>That bt you sent is normal. That is the CDP timer process so I don't see any issues there. What you can do is send me bt's on the "CDP worker childs", the "cdp_receiver_peer" for your HSS connection, and the "cdp acceptor".</div>
<div><br></div><div>to get the processes, use sercmd ps. For example, mine looks like this:</div><div><br></div><div><div>8633<span class="" style="white-space:pre"> </span>cdp worker child=0</div><div>8634<span class="" style="white-space:pre"> </span>cdp worker child=1</div>
<div>8635<span class="" style="white-space:pre"> </span>cdp worker child=2</div><div>8636<span class="" style="white-space:pre"> </span>cdp worker child=3</div><div>8637<span class="" style="white-space:pre"> </span>cdp worker child=4</div>
<div>8638<span class="" style="white-space:pre"> </span>cdp worker child=5</div><div>8639<span class="" style="white-space:pre"> </span>cdp worker child=6</div><div>8640<span class="" style="white-space:pre"> </span>cdp worker child=7</div>
<div>8641<span class="" style="white-space:pre"> </span>cdp worker child=8</div><div>8642<span class="" style="white-space:pre"> </span>cdp worker child=9</div><div>8643<span class="" style="white-space:pre"> </span>cdp receiver peer unknown</div>
<div>8644<span class="" style="white-space:pre"> </span>cdp_receiver_peer=<a href="http://ZAUbuntu005.it.za.smilecoms.com">ZAUbuntu005.it.za.smilecoms.com</a></div><div>8645<span class="" style="white-space:pre"> </span>cdp_acceptor</div>
<div>8646<span class="" style="white-space:pre"> </span>cdp_timer</div></div><div><br></div><div>re. logs, you can turn on debugging using sercmd/kamcmd.</div><div><br></div><div>cheers</div><div>Jason</div><div><br></div>
<div>Cheers</div><div>Jason</div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Jan 30, 2014 at 12:11 AM, Daniel Ciprus <span dir="ltr"><<a href="mailto:daniel.ciprus@acision.com" target="_blank">daniel.ciprus@acision.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style bgcolor="#FFFFFF" text="#000000">
Jason, <br>
<br>
this is bt generated from running process .. I think my scscf is in this stage now. Obviously hss has listener up and running and I can telnet to the port. "Lately" means that nobody paid attention to this issue before :-), it was discovered just recently.
<br>
<br>
As I said: nothing's in capture : <br>
<br>
[root@kamailio-4-1-test kamailio]# date ; tcpdump -i any -s0 -w /tmp/crap.pcap -v port 3868 -n ; date<br>
Wed Jan 29 16:45:15 EST 2014<br>
tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes<br>
^C0 packets captured<br>
0 packets received by filter<br>
0 packets dropped by kernel<br>
Wed Jan 29 17:09:08 EST 2014<br>
[root@kamailio-4-1-test kamailio]# <br>
<br>
Once I restart scscf/icscf everything will be OK ... so there is likely no way to get logging out of the running scscf, is it ?<div><div class="h5"><br>
<br>
<div>On 01/29/2014 09:45 AM, Jason Penton wrote:<br>
</div>
<blockquote style="border-left:2px solid #009900!important;border-right:2px solid #009900!important;padding:0px 15px 0px 15px;margin:8px 2px" type="cite">
<div dir="ltr">Hi Daniel,
<div><br>
</div>
<div>This looks like a bootup of kamailio, where the peer process is trying to start? Is your HSS up. Can you telnet to your HSS:3868?</div>
<div><br>
</div>
<div>Also, your first message says you are having trouble with CDP "lately". What does this mean? There have been no changes to CDP code for some time now...</div>
<div><br>
</div>
<div>Cheers</div>
<div>jason</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Wed, Jan 29, 2014 at 4:30 PM, Daniel Ciprus <span dir="ltr">
<<a href="mailto:daniel.ciprus@acision.com" target="_blank">daniel.ciprus@acision.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">(gdb) bt<br>
#0 0x00007f5b564f7900 in __nanosleep_nocancel () from /lib64/libc.so.6<br>
#1 0x00007f5b564f7790 in sleep () from /lib64/libc.so.6<br>
#2 0x00007f5b5095cb71 in timer_loop () at timer.c:120<br>
#3 0x00007f5b5095d5a4 in timer_process (returns=0) at timer.c:207<br>
#4 0x00007f5b5093f891 in diameter_peer_start (blocking=0) at diameter_peer.c:396<br>
#5 0x00007f5b5094102b in cdp_child_init (rank=0) at mod.c:237<br>
#6 0x00000000004f7ec2 in init_mod_child (m=0x7f5b5609a670, rank=0) at sr_module.c:924<br>
#7 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b218, rank=0) at sr_module.c:921<br>
#8 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b5c0, rank=0) at sr_module.c:921<br>
#9 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b970, rank=0) at sr_module.c:921<br>
#10 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609c000, rank=0) at sr_module.c:921<br>
#11 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609c478, rank=0) at sr_module.c:921<br>
#12 0x00000000004f8048 in init_child (rank=0) at sr_module.c:948<br>
#13 0x000000000046d57c in main_loop () at main.c:1694<br>
#14 0x000000000047030b in main (argc=13, argv=0x7ffff3e21128) at main.c:2533<br>
(gdb) bt full<br>
#0 0x00007f5b564f7900 in __nanosleep_nocancel () from /lib64/libc.so.6<br>
No symbol table info available.<br>
#1 0x00007f5b564f7790 in sleep () from /lib64/libc.so.6<br>
No symbol table info available.<br>
#2 0x00007f5b5095cb71 in timer_loop () at timer.c:120<br>
now = 1390934103<br>
i = 0x0<br>
cb = 0<br>
ptr = 0x0<br>
interval = 1<br>
__FUNCTION__ = "timer_loop"<br>
#3 0x00007f5b5095d5a4 in timer_process (returns=0) at timer.c:207<br>
__FUNCTION__ = "timer_process"<br>
#4 0x00007f5b5093f891 in diameter_peer_start (blocking=0) at diameter_peer.c:396<br>
pid = 0<br>
k = -2<br>
p = 0x0<br>
__FUNCTION__ = "diameter_peer_start"<br>
#5 0x00007f5b5094102b in cdp_child_init (rank=0) at mod.c:237<br>
__FUNCTION__ = "cdp_child_init"<br>
#6 0x00000000004f7ec2 in init_mod_child (m=0x7f5b5609a670, rank=0) at sr_module.c:924<br>
__FUNCTION__ = "init_mod_child"<br>
#7 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b218, rank=0) at sr_module.c:921<br>
__FUNCTION__ = "init_mod_child"<br>
#8 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b5c0, rank=0) at sr_module.c:921<br>
__FUNCTION__ = "init_mod_child"<br>
#9 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b970, rank=0) at sr_module.c:921<br>
__FUNCTION__ = "init_mod_child"<br>
#10 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609c000, rank=0) at sr_module.c:921<br>
__FUNCTION__ = "init_mod_child"<br>
#11 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609c478, rank=0) at sr_module.c:921<br>
__FUNCTION__ = "init_mod_child"<br>
#12 0x00000000004f8048 in init_child (rank=0) at sr_module.c:948<br>
No locals.<br>
#13 0x000000000046d57c in main_loop () at main.c:1694<br>
i = 8<br>
pid = 19298<br>
si = 0x0<br>
si_desc = "udp receiver child=7 sock=<a href="http://10.96.173.17:6060" target="_blank">10.96.173.17:6060</a>\000[\177\000\000\001\000\000\000z\000\000\000~}^\000\000\000\000\000`o^\000\000\000\000\000\254\305\006m\000\000\000\000\300LA\000\000\000\000\000
\021\342\363\377\177", '\000' <repeats 18 times>, "`\017\342\363\377\177\000\000.\241K\000\000\000\000"<br>
nrprocs = 8<br>
__FUNCTION__ = "main_loop"<br>
#14 0x000000000047030b in main (argc=13, argv=0x7ffff3e21128) at main.c:2533<br>
cfg_stream = 0x1a7f010<br>
c = -1<br>
r = 0<br>
tmp = 0x7ffff3e22f40 ""<br>
tmp_len = 0<br>
port = 0<br>
proto = 0<br>
options = 0x5e02c0 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:"<br>
ret = -1<br>
seed = 2714878883<br>
rfd = 4<br>
debug_save = 0<br>
debug_flag = 0<br>
dont_fork_cnt = 0<br>
---Type <return> to continue, or q <return> to quit---<br>
n_lst = 0x7f5b5645bb88<br>
p = 0x5ca3e0 "H\211l$\330L\211d$\340H\215-\017\234*"<br>
__FUNCTION__ = "main"<br>
(gdb) <br>
<div>
<div>
<div>On 01/29/2014 08:45 AM, Daniel Ciprus wrote:<br>
</div>
<blockquote style="border-left:2px solid #009900!important;border-right:2px solid #009900!important;padding:0px 15px 0px 15px;margin:8px 2px" type="cite">
Unfortunately the only thing I have is core file and bt. But I can assure you that there was absolutely no traffic between
diameter interfaces. Looks like I will need to reproduce the problem to get right input.
<br>
<br>
thanks<br>
Dan<br>
<br>
<br>
<div>On 01/29/2014 02:39 AM, Jason Penton wrote:<br>
</div>
<blockquote style="border-left:2px solid #009900!important;border-right:2px solid #009900!important;padding:0px 15px 0px 15px;margin:8px 2px" type="cite">
<div dir="ltr">Hi,
<div><br>
</div>
<div>can you send us a pcap as well as a log file? I can't debug much with what you have given so far.</div>
<div><br>
</div>
<div>Cheers</div>
<div>Jason</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Tue, Jan 28, 2014 at 8:42 PM, Daniel Ciprus <span dir="ltr">
<<a href="mailto:daniel.ciprus@acision.com" target="_blank">daniel.ciprus@acision.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br>
<br>
seems like CDP module is giving us hard time in last few days. For some<br>
reason s-cscf and i-cscf is loosing connection to HSS and is not even<br>
trying to close socket nor trying to reconnect back. Here is what I see<br>
on the kamailio node:<br>
<br>
Name : kamailio Relocations: (not relocatable)<br>
Version : 4.2.0 Vendor: <a href="http://kamailio.org" target="_blank">
kamailio.org</a><br>
Release : dev0.0.el6 Build Date: Wed Dec 18<br>
12:46:11 2013<br>
<br>
[root@kamailio-4-1-staging kamailio]# netstat -apn | grep 3868<br>
tcp 0 0 <a href="http://10.96.173.17:34915" target="_blank">
10.96.173.17:34915</a> <a href="http://10.96.173.18:3868" target="_blank">
10.96.173.18:3868</a><br>
CLOSE_WAIT 10087/kamailio<br>
tcp 0 0 <a href="http://10.96.173.17:35361" target="_blank">
10.96.173.17:35361</a> <a href="http://10.96.173.18:3868" target="_blank">
10.96.173.18:3868</a><br>
CLOSE_WAIT 19308/kamailio<br>
[root@kamailio-4-1-staging kamailio]#<br>
<br>
and HSS:<br>
<br>
[root@hss-config-staging phss-sop]# netstat -apn | grep 3868<br>
tcp 0 0 <a href="http://10.96.173.18:3868" target="_blank">
10.96.173.18:3868</a> 0.0.0.0:*<br>
LISTEN 10549/ugc_server<br>
[root@hss-config-staging phss-sop]#<br>
<br>
I have core files (created by gcore) if this helps.<br>
<br>
thanks<br>
Dan<br>
<br>
<br>
<br>
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other
party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you for understanding.<br>
<br>
<br>
<br>
_______________________________________________<br>
sr-dev mailing list<br>
<a href="mailto:sr-dev@lists.sip-router.org" target="_blank">sr-dev@lists.sip-router.org</a><br>
<a href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev" target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev</a><br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<div>-- <br>
<b>Daniel Ciprus</b><br>
Integration engineer<br>
<a href="http://www.acision.com" target="_blank">http://www.acision.com</a><br>
<br>
9954 Mayland Dr<br>
Suite 3100<br>
Richmond, VA 23233<br>
USA<br>
T: <a href="tel:%2B1%20804%20762%205601" value="+18047625601" target="_blank">
+1 804 762 5601</a><br>
E: <a href="mailto:daniel.ciprus@acision.com" target="_blank">
daniel.ciprus@acision.com</a><br>
</div>
<br>
<hr>
<font color="Gray" face="Arial" size="1">This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed
to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you for understanding.<br>
</font><br clear="all">
</blockquote>
<br>
<div>-- <br>
<b>Daniel Ciprus</b><br>
Integration engineer<br>
<a href="http://www.acision.com" target="_blank">http://www.acision.com</a><br>
<br>
9954 Mayland Dr<br>
Suite 3100<br>
Richmond, VA 23233<br>
USA<br>
T: <a href="tel:%2B1%20804%20762%205601" value="+18047625601" target="_blank">
+1 804 762 5601</a><br>
E: <a href="mailto:daniel.ciprus@acision.com" target="_blank">
daniel.ciprus@acision.com</a><br>
</div>
<br>
<hr>
<font color="Gray" face="Arial" size="1">This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed
to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you for understanding.<br>
</font><br clear="all">
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<div>-- <br>
<b>Daniel Ciprus</b><br>
Integration engineer<br>
<a href="http://www.acision.com" target="_blank">http://www.acision.com</a><br>
<br>
9954 Mayland Dr<br>
Suite 3100<br>
Richmond, VA 23233<br>
USA<br>
T: <a href="tel:%2B1%20804%20762%205601" value="+18047625601" target="_blank">+1 804 762 5601</a><br>
E: <a href="mailto:daniel.ciprus@acision.com" target="_blank">daniel.ciprus@acision.com</a><br>
</div>
<br>
<hr>
<font face="Arial" color="Gray" size="1">This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed
to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you for understanding.<br>
</font>
<br clear="all">
</div></div></div>
</blockquote></div><br></div>