<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    Hello,<br>
    <br>
    I pushed the commit to branch 3.3 now -- forgot about it.<br>
    <br>
    The code you pasted below becomes (after the two patches, thus is
    the final version that should be used):<br>
    <br>
        ENTER;                /* everything created after here */<br>
        SAVETMPS;            /* ...is a temporary variable.   */<br>
        PUSHMARK(SP);        /* remember the stack pointer    */<br>
    <br>
        m = sv_newmortal();<br>
        sv_setref_pv(m, "Kamailio::Message", (void *)_msg);<br>
        SvREADONLY_on(SvRV(m));<br>
    <br>
        XPUSHs(m);            /* Our reference to the stack... */<br>
    <br>
        if (mystr)<br>
            XPUSHs(sv_2mortal(newSVpv(mystr, strlen(mystr))));<br>
    <br>
    <br>
    Cheers,<br>
    Daniel<br>
    <br>
    <div class="moz-cite-prefix">On 7/31/13 1:48 PM, David Cunningham
      wrote:<br>
    </div>
    <blockquote
cite="mid:CAHGbP--nTWDQKp5oUgMxDcYUVCu4R1JdYtctS85q8yeZgsdgGA@mail.gmail.com"
      type="cite">Hi Daniel,<br>
      <br>
      In 3.3.x the code is arranged a little differently and I'd like to
      make sure we get it right. Could you please advise?<br>
      <br>
      <br>
              m = sv_newmortal();<br>
              sv_setref_pv(m, "OpenSER::Message", (void *)_msg);<br>
              SvREADONLY_on(SvRV(m));<br>
      <br>
      <br>
              ENTER;                          /* everything created
      after here */<br>
              SAVETMPS;                       /* ...is a temporary
      variable.   */<br>
              PUSHMARK(SP);                   /* remember the stack
      pointer    */<br>
              XPUSHs(m);                      /* Our reference to the
      stack... */<br>
      <br>
              if (mystr)<br>
                      XPUSHs(sv_2mortal(newSVpv(mystr, strlen(mystr))));<br>
                                              /* Our string to the
      stack... */<br>
      <br>
      <br>
      <br>
      <div class="gmail_quote">On 31 July 2013 21:44, David Cunningham <span
          dir="ltr"><<a moz-do-not-send="true"
            href="mailto:dcunningham@voisonics.com" target="_blank">dcunningham@voisonics.com</a>></span>
        wrote:<br>
        <blockquote class="gmail_quote" style="margin:0 0 0
          .8ex;border-left:1px #ccc solid;padding-left:1ex">
          Hi Daniel,<br>
          <br>
          Thank you, we will try that.
          <div class="HOEnZb">
            <div class="h5"><br>
              <br>
              <br>
              <div class="gmail_quote">On 31 July 2013 20:54,
                Daniel-Constantin Mierla <span dir="ltr"><<a
                    moz-do-not-send="true"
                    href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>></span>
                wrote:<br>
                <blockquote class="gmail_quote" style="margin:0 0 0
                  .8ex;border-left:1px #ccc solid;padding-left:1ex">
                  <div text="#000000" bgcolor="#FFFFFF"> Hello,<br>
                    <br>
                    revising the patch I noticed I was moving the
                    initialization of the variable after pushing it to
                    perl environment (from the perl docs, the variable
                    should have been initialized after initializing the
                    environment -- what I tried to do with previous
                    patch).<br>
                    <br>
                    There is a new smaller patch to be added:<br>
                    <br>
                    <a moz-do-not-send="true"
href="http://git.sip-router.org/cgi-bin/gitweb.cgi?p=sip-router;a=commitdiff_plain;h=3935fedf23f3bf2b6675182193cef6af3bbd903a"
                      target="_blank">http://git.sip-router.org/cgi-bin/gitweb.cgi?p=sip-router;a=commitdiff_plain;h=3935fedf23f3bf2b6675182193cef6af3bbd903a</a><br>
                    <br>
                    Practically, the line XPUSHs(m); has to be moved
                    after the line with SvREADONLY_on(SvRV(m));<br>
                    <br>
                    Cheers,<br>
                    Daniel
                    <div>
                      <div><br>
                        <br>
                        <div>On 7/31/13 8:26 AM, David Cunningham wrote:<br>
                        </div>
                        <blockquote type="cite">Hi Daniel,<br>
                          <br>
                          We tried that patch, but Kamailio logged lots
                          of errors like the following. The undefined
                          value in question is $m, which should be the
                          SIP message. Would you have any advice?
                          Thanks.<br>
                          <br>
                          Jul 31 02:13:57 hostname
                          /sbin/kamailio[21087]: ERROR: perl
                          [openserxs.xs:1022]: perl error: Can't call
                          method "pseudoVar" on an undefined value at
                          Foo.pm line 247.#012<br>
                          <br>
                          <br>
                          <div class="gmail_quote">On 25 July 2013
                            17:11, David Cunningham <span dir="ltr"><<a
                                moz-do-not-send="true"
                                href="mailto:dcunningham@voisonics.com"
                                target="_blank">dcunningham@voisonics.com</a>></span>
                            wrote:<br>
                            <blockquote class="gmail_quote"
                              style="margin:0 0 0 .8ex;border-left:1px
                              #ccc solid;padding-left:1ex"> Hi Daniel,<br>
                              <br>
                              I'll suggest that to the customer. Thank
                              you!
                              <div>
                                <div><br>
                                  <br>
                                  <br>
                                  <div class="gmail_quote">On 25 July
                                    2013 15:45, Daniel-Constantin Mierla
                                    <span dir="ltr"><<a
                                        moz-do-not-send="true"
                                        href="mailto:miconda@gmail.com"
                                        target="_blank">miconda@gmail.com</a>></span>
                                    wrote:<br>
                                    <blockquote class="gmail_quote"
                                      style="margin:0 0 0
                                      .8ex;border-left:1px #ccc
                                      solid;padding-left:1ex">
                                      <div text="#000000"
                                        bgcolor="#FFFFFF"> Hello,<br>
                                        <br>
                                        can you try the attached patch?
                                        It's the same patch, just for
                                        two versions, one is for 3.3.x
                                        and the other for devel version<br>
                                        <br>
                                        It initializes the SIP message
                                        variable that is passed to perl
                                        after creating the temporary
                                        environment, so it is actually
                                        destroyed by the perl embedded
                                        interpreter.<br>
                                        <br>
                                        Cheers,<br>
                                        Daniel
                                        <div>
                                          <div><br>
                                            <br>
                                            <div>On 7/25/13 1:29 AM,
                                              David Cunningham wrote:<br>
                                            </div>
                                            <blockquote type="cite">Hi
                                              Daniel,<br>
                                              <br>
                                              The system is running Perl
                                              5.8.8 on Red Hat
                                              Enterprise Linux Server
                                              release 5.4. If I remember
                                              right programs running
                                              under Valgrind can have
                                              issues, so I'm not sure if
                                              the customer will want to
                                              do that. Ideally we'd do
                                              it on a test system, but
                                              I'm not sure if we have
                                              any RHEL available.<br>
                                              I'll see what we can do.
                                              Thanks again.<br>
                                              <br>
                                              <br>
                                              <div class="gmail_quote">On
                                                25 July 2013 04:55,
                                                Daniel-Constantin Mierla
                                                <span dir="ltr"><<a
                                                    moz-do-not-send="true"
href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>></span>
                                                wrote:<br>
                                                <blockquote
                                                  class="gmail_quote"
                                                  style="margin:0 0 0
                                                  .8ex;border-left:1px
                                                  #ccc
                                                  solid;padding-left:1ex">
                                                  <div text="#000000"
                                                    bgcolor="#FFFFFF">
                                                    Hello,<br>
                                                    <br>
                                                    I would say that
                                                    perl_exec() is the
                                                    one with the highest
                                                    chances to be the
                                                    reason for the leak.
                                                    Next is line would
                                                    be db_mysql module,
                                                    if liked with some
                                                    custom mysql client
                                                    library, although
                                                    even in this case
                                                    will be unlikely.<br>
                                                    <br>
                                                    Back to perl, the
                                                    module itself does
                                                    not call any malloc,
                                                    so it might be the
                                                    embedding Perl API
                                                    that is not used
                                                    properly in the
                                                    module.<br>
                                                    <br>
                                                    Can you use some
                                                    testbed, set
                                                    children=1 and run
                                                    kamailio under
                                                    valgrind, then do
                                                    some calls and see
                                                    if it detects the
                                                    source of the leak?<br>
                                                    <br>
                                                    I'm not using the
                                                    perl module, I will
                                                    try to check it
                                                    whenever I get a
                                                    chance in the next
                                                    days. What version
                                                    of perl do you have
                                                    installed?<br>
                                                    <br>
                                                    Cheers,<br>
                                                    Daniel
                                                    <div>
                                                      <div><br>
                                                        <br>
                                                        <div>On 7/24/13
                                                          10:31 AM,
                                                          David
                                                          Cunningham
                                                          wrote:<br>
                                                        </div>
                                                        <blockquote
                                                          type="cite">Hello,<br>
                                                          <br>
                                                          We don't do
                                                          any kamctl
                                                          commands at
                                                          all. We do
                                                          have various
                                                          modules
                                                          loaded, as
                                                          follows. The
                                                          primary
                                                          functions we
                                                          use Kamailio
                                                          for are phone
                                                          registrations
                                                          through
                                                          usrloc, and
                                                          routing calls
                                                          to Asterisk
                                                          through logic
                                                          contained in
                                                          Perl via
                                                          perl_exec().<br>
                                                          Thanks for all
                                                          your advice so
                                                          far!<br>
                                                          <br>
                                                          loadmodule
                                                          "tm.so"<br>
                                                          loadmodule
                                                          "tmx.so"<br>
                                                          loadmodule
                                                          "usrloc.so"<br>
                                                          loadmodule
                                                          "auth.so"<br>
                                                          loadmodule
                                                          "auth_db.so"<br>
                                                          loadmodule
                                                          "ctl.so"<br>
                                                          loadmodule
                                                          "db_mysql.so"<br>
                                                          loadmodule
                                                          "kex.so"<br>
                                                          loadmodule
                                                          "maxfwd.so"<br>
                                                          loadmodule
                                                          "mi_fifo.so"<br>
                                                          loadmodule
                                                          "mi_rpc.so"<br>
                                                          loadmodule
                                                          "nathelper.so"<br>
                                                          loadmodule
                                                          "perl.so"<br>
                                                          loadmodule
                                                          "pv.so"<br>
                                                          loadmodule
                                                          "registrar.so"<br>
                                                          loadmodule
                                                          "rr.so"<br>
                                                          loadmodule
                                                          "sanity.so"<br>
                                                          loadmodule
                                                          "siputils.so"<br>
                                                          loadmodule
                                                          "sl.so"<br>
                                                          loadmodule
                                                          "textops.so"<br>
                                                          loadmodule
                                                          "xlog.so"<br>
                                                          <br>
                                                          <br>
                                                          <div
                                                          class="gmail_quote">On
                                                          24 July 2013
                                                          16:33,
                                                          Daniel-Constantin
                                                          Mierla <span
                                                          dir="ltr"><<a
moz-do-not-send="true" href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>></span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
                                                          style="margin:0
                                                          0 0
                                                          .8ex;border-left:1px
                                                          #ccc
                                                          solid;padding-left:1ex">
                                                          <div
                                                          text="#000000"
bgcolor="#FFFFFF"> Hello,
                                                          <div><br>
                                                          <br>
                                                          <div>On
                                                          7/24/13 4:24
                                                          AM, David
                                                          Cunningham
                                                          wrote:<br>
                                                          </div>
                                                          </div>
                                                          <blockquote
                                                          type="cite">Hello,<br>
                                                          <br>
                                                          <div> Thank
                                                          you very much
                                                          for the email.
                                                          In reply:<br>
                                                          <br>
                                                          1. The system
                                                          ran out of
                                                          memory.
                                                          Linux's
                                                          oom-killer
                                                          killed
                                                          Kamailio.<br>
                                                          </div>
                                                          </blockquote>
                                                          then all the
                                                          instructions I
                                                          gave are
                                                          useless, they
                                                          are for
                                                          debugging
                                                          kamailio's
                                                          internal
                                                          memory
                                                          manager, which
                                                          handles pkg
                                                          and shm
                                                          mallocs.<br>
                                                          <br>
                                                          The chances to
                                                          be from
                                                          kamailio
                                                          itself are
                                                          very low now.
                                                          Do you do lot
                                                          of mi commands
                                                          (e.g., kamctl
                                                          ...)? The mi
                                                          api uses
                                                          system malloc,
                                                          but the rest
                                                          of code should
                                                          use internal
                                                          memory manager
                                                          which does not
                                                          go beyond the
                                                          limits set
                                                          with -m and
                                                          -M, thus not
                                                          causing an OS
                                                          memory
                                                          exhaustion.<br>
                                                          <br>
                                                          Can you list
                                                          what modules
                                                          are you
                                                          loading? At
                                                          some point it
                                                          was a leak in
                                                          libssl, in
                                                          case you use
                                                          tls a lot. But
                                                          could be
                                                          another
                                                          external
                                                          library...<br>
                                                          <br>
                                                          Cheers,<br>
                                                          Daniel
                                                          <div>
                                                          <div><br>
                                                          <br>
                                                          <blockquote
                                                          type="cite"><br>
                                                          2. You're
                                                          right,
                                                          DEBUG_MEMORY
                                                          is a local
                                                          configuration
                                                          setting. If
                                                          defined it
                                                          sets memdbg to
                                                          -2, and memlog
                                                          to -2. The
                                                          debug setting
                                                          is -1. <br>
                                                          <br>
                                                          3. We'll try
                                                          setting
                                                          mem_summary=12,
                                                          thanks.<br>
                                                          <br>
                                                          4. We'll try
                                                          setting
                                                          asynchronous
                                                          syslog,
                                                          thanks.<br>
                                                          <br>
                                                          5.  Our
                                                          configuration
                                                          totals 338
                                                          lines, or
                                                          approx 8.5kb.
                                                          Is that a lot?
                                                          <br>
                                                          <br>
                                                          6. We'll try
                                                          setting
                                                          mem_join=1,
                                                          thanks.<br>
                                                          <br>
                                                          <br>
                                                          <br>
                                                          <div
                                                          class="gmail_quote">On

                                                          23 July 2013
                                                          16:53,
                                                          Daniel-Constantin
                                                          Mierla <span
                                                          dir="ltr"><<a
moz-do-not-send="true" href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>></span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
                                                          style="margin:0
                                                          0 0
                                                          .8ex;border-left:1px
                                                          #ccc
                                                          solid;padding-left:1ex">
                                                          Hello,<br>
                                                          <br>
                                                          first, to
                                                          clarify, is
                                                          the system
                                                          memory or
                                                          kamailio's
                                                          pkg/shm memory
                                                          running out?
                                                          If the
                                                          operating
                                                          system runs
                                                          out of memory,
                                                          then should be
                                                          a leak in a
                                                          library,
                                                          because
                                                          kamailio
                                                          modules uses
                                                          only from a
                                                          pre-allocated
                                                          chunk, not
                                                          going over it.
                                                          <div> <br>
                                                          <br>
                                                          On 7/23/13
                                                          7:33 AM, David
                                                          Cunningham
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
                                                          style="margin:0
                                                          0 0
                                                          .8ex;border-left:1px
                                                          #ccc
                                                          solid;padding-left:1ex">
                                                          Hello,<br>
                                                          <br>
                                                          We're running
                                                          a Kamailio
                                                          3.3.4 system,
                                                          and Kamailio
                                                          is slowly
                                                          using more and
                                                          more memory.
                                                          Over a couple
                                                          of weeks it
                                                          will run out
                                                          of system
                                                          memory.<br>
                                                          <br>
                                                          We tried to
                                                          enable memory
                                                          debugging
                                                          doing the
                                                          following, but
                                                          it resulted in
                                                          Kamailio not
                                                          responding to
                                                          any SIP
                                                          packets. Would
                                                          anyone have
                                                          advice please
                                                          on how to
                                                          debug the
                                                          situation?<br>
                                                          <br>
                                                          1. In
                                                          Makefile.defs
                                                          set MEMDBG to
                                                          1 and
                                                          recompile
                                                          Kamailio.<br>
                                                          2. In
                                                          kamailio.cfg
                                                          add the line:<br>
                                                          #!define
                                                          DEBUG_MEMORY 1<br>
                                                          </blockquote>
                                                          </div>
                                                          do you set
                                                          something
                                                          special in
                                                          config when
                                                          DEBUG_MEMORY
                                                          is 1? It is
                                                          not by default
                                                          there, so I
                                                          assume you
                                                          added some
                                                          rules based on
                                                          this
                                                          pre-processor
                                                          directive.<br>
                                                          <br>
                                                          For memory
                                                          troubleshooting,
                                                          set memlog to
                                                          a value lower
                                                          than debug
                                                          parameter in
                                                          config file
                                                          and try with
                                                          mem_summary=12
                                                          for a more
                                                          compact
                                                          output. See
                                                          more about
                                                          these
                                                          parameters in
                                                          the wiki:<br>
                                                          <br>
                                                          - <a
                                                          moz-do-not-send="true"
href="http://www.kamailio.org/wiki/cookbooks/3.3.x/core#memlog"
                                                          target="_blank">http://www.kamailio.org/wiki/cookbooks/3.3.x/core#memlog</a><br>
                                                          <br>
                                                          Run kamailio
                                                          for a while in
                                                          normal
                                                          conditions,
                                                          then restart
                                                          it to get the
                                                          memory usage
                                                          summaries.
                                                          There should
                                                          be indication
                                                          if there is
                                                          some leak, by
                                                          seeing memory
                                                          chunks
                                                          allocated many
                                                          times from a
                                                          function used
                                                          at runtime.
                                                          You can send
                                                          the memory
                                                          summary for a
                                                          process here,
                                                          we can look at
                                                          it.
                                                          <div> <br>
                                                          <br>
                                                          <blockquote
                                                          class="gmail_quote"
                                                          style="margin:0
                                                          0 0
                                                          .8ex;border-left:1px
                                                          #ccc
                                                          solid;padding-left:1ex">
                                                          <br>
                                                          While this was
                                                          running and
                                                          Kamailio
                                                          didn't respond
                                                          to packets, it
                                                          logged lots of
                                                          lines like
                                                          this:<br>
                                                          </blockquote>
                                                          <br>
                                                          </div>
                                                          Do you have
                                                          syslog to be
                                                          configured in
                                                          asynchronous
                                                          mode? See the
                                                          notes from:<br>
                                                          <br>
                                                          - <a
                                                          moz-do-not-send="true"
href="http://www.kamailio.org/wiki/tutorials/3.2.x/syslog"
                                                          target="_blank">http://www.kamailio.org/wiki/tutorials/3.2.x/syslog</a><br>
                                                          <br>
                                                          The memdbg is
                                                          less than
                                                          debug value,
                                                          that means
                                                          printing few
                                                          log messages
                                                          for each
                                                          memory
                                                          operation. You
                                                          can make
                                                          memdbg higher
                                                          and rely on
                                                          memlog for
                                                          memory
                                                          summaries,
                                                          otherwise will
                                                          be lot of log
                                                          messages
                                                          related to
                                                          memory.
                                                          <div> <br>
                                                          <blockquote
                                                          class="gmail_quote"
                                                          style="margin:0
                                                          0 0
                                                          .8ex;border-left:1px
                                                          #ccc
                                                          solid;padding-left:1ex">
                                                          <br>
                                                          Jul 22
                                                          21:32:22
                                                          hostname
                                                          kamailio: :
                                                          <core>
                                                          [mem/q_malloc.c:369]:
                                                          qm_malloc(0x4000e008,

                                                          128) called
                                                          from
                                                          <core>:
                                                          cfg.lex:
                                                          addstr(1438)<br>
                                                          Jul 22
                                                          21:32:22
                                                          hostname
                                                          kamailio: :
                                                          <core>
                                                          [mem/q_malloc.c:413]:
                                                          qm_malloc(0x4000e008,

                                                          128) returns
                                                          address
                                                          0x40048918
                                                          frag.
                                                          0x40048900
                                                          (size=128) on
                                                          1 -th hit<br>
                                                          Jul 22
                                                          21:32:22
                                                          hostname
                                                          kamailio: :
                                                          <core>
                                                          [mem/q_malloc.c:369]:
                                                          qm_malloc(0x4000e008,

                                                          128) called
                                                          from
                                                          <core>:
                                                          cfg.lex:
                                                          addstr(1438)<br>
                                                          Jul 22
                                                          21:32:22
                                                          hostname
                                                          kamailio: :
                                                          <core>
                                                          [mem/q_malloc.c:413]:
                                                          qm_malloc(0x4000e008,

                                                          128) returns
                                                          address
                                                          0x400489c8
                                                          frag.
                                                          0x400489b0
                                                          (size=128) on
                                                          1 -th hit<br>
                                                          </blockquote>
                                                          </div>
                                                          addstr() is a
                                                          function used
                                                          only for
                                                          parsing
                                                          configuration
                                                          file, as long
                                                          as you can
                                                          still see
                                                          them, the
                                                          configuration
                                                          file parsing
                                                          was not
                                                          finish.
                                                          addstr() is
                                                          not a source
                                                          of leaks
                                                          because it is
                                                          not used at
                                                          runtime.<br>
                                                          <br>
                                                          If you have
                                                          large config
                                                          file, then you
                                                          can get close
                                                          to the limits
                                                          of the private
                                                          memory, which
                                                          is set to 4MB.
                                                          You can
                                                          increase its
                                                          value using -M
                                                          parameter
                                                          (e.g., start
                                                          kamailio with
                                                          -M 8 to set it
                                                          to use 8MB of
                                                          memory).<br>
                                                          <br>
                                                          Over the time,
                                                          the private
                                                          memory can get
                                                          used due to
                                                          fragmentation,
                                                          you can set
                                                          the mem_join
                                                          parameter in
                                                          config file to
                                                          avoid it
                                                          (works when
                                                          compiled with
                                                          MEMDBG=1).<br>
                                                          <br>
                                                          To monitor
                                                          usage of
                                                          internal pkg
                                                          memory, then
                                                          you can use
                                                          sercmd with
                                                          pkg.stats
                                                          command:<br>
                                                          <br>
                                                          <a
                                                          moz-do-not-send="true"
href="http://kamailio.org/docs/modules/3.3.x/modules_k/kex.html#idp16972640"
target="_blank">http://kamailio.org/docs/modules/3.3.x/modules_k/kex.html#idp16972640</a><br>
                                                          <br>
                                                          Shared memory
                                                          stats are
                                                          printed by
                                                          'kamctl fifo
                                                          get_statistics
                                                          shmem:'<br>
                                                          <br>
                                                          When you see
                                                          significant
                                                          increase of
                                                          the memory
                                                          usage, then
                                                          you can
                                                          restart to get
                                                          the summaries.<br>
                                                          <br>
                                                          You should run
                                                          these commands
                                                          after start,
                                                          just to see
                                                          the initial
                                                          usage of
                                                          memory.<br>
                                                          <br>
                                                          Cheers,<br>
                                                          Daniel<span><font
color="#888888"><br>
                                                          <br>
                                                          -- <br>
                                                          Daniel-Constantin

                                                          Mierla - <a
                                                          moz-do-not-send="true"
href="http://www.asipto.com" target="_blank">http://www.asipto.com</a><br>
                                                          <a
                                                          moz-do-not-send="true"
href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a>
                                                          - <a
                                                          moz-do-not-send="true"
href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a><br>
                                                          <br>
                                                          <br>
_______________________________________________<br>
                                                          SIP Express
                                                          Router (SER)
                                                          and Kamailio
                                                          (OpenSER) -
                                                          sr-users
                                                          mailing list<br>
                                                          <a
                                                          moz-do-not-send="true"
href="mailto:sr-users@lists.sip-router.org" target="_blank">sr-users@lists.sip-router.org</a><br>
                                                          <a
                                                          moz-do-not-send="true"
href="http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users"
                                                          target="_blank">http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users</a><br>
                                                          </font></span></blockquote>
                                                          </div>
                                                          <br>
                                                          <br
                                                          clear="all">
                                                          <br>
                                                          -- <br>
                                                          David
                                                          Cunningham,
                                                          Voisonics<br>
                                                          <a
                                                          moz-do-not-send="true"
href="http://voisonics.com/" target="_blank">http://voisonics.com/</a><br>
                                                          USA: <a
                                                          moz-do-not-send="true"
href="tel:%2B1%20213%20221%201092" value="+12132211092" target="_blank">+1


                                                          213 221 1092</a><br>
                                                          UK: <a
                                                          moz-do-not-send="true"
href="tel:%2B44%20%280%29%2020%203298%201642" value="+442032981642"
                                                          target="_blank">+44

                                                          (0) 20 3298
                                                          1642</a><br>
                                                          Australia: <a
moz-do-not-send="true" href="tel:%2B61%20%280%29%202%208063%209019"
                                                          value="+61280639019"
target="_blank">+61 (0) 2 8063 9019</a><br>
                                                          </blockquote>
                                                          <br>
                                                          <pre cols="72">-- 
Daniel-Constantin Mierla - <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a moz-do-not-send="true" href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a>
</pre>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          <br
                                                          clear="all">
                                                          <br>
                                                          -- <br>
                                                          David
                                                          Cunningham,
                                                          Voisonics<br>
                                                          <a
                                                          moz-do-not-send="true"
href="http://voisonics.com/" target="_blank">http://voisonics.com/</a><br>
                                                          USA: <a
                                                          moz-do-not-send="true"
href="tel:%2B1%20213%20221%201092" value="+12132211092" target="_blank">+1
                                                          213 221 1092</a><br>
                                                          UK: <a
                                                          moz-do-not-send="true"
href="tel:%2B44%20%280%29%2020%203298%201642" value="+442032981642"
                                                          target="_blank">+44
                                                          (0) 20 3298
                                                          1642</a><br>
                                                          Australia: <a
moz-do-not-send="true" href="tel:%2B61%20%280%29%202%208063%209019"
                                                          value="+61280639019"
target="_blank">+61 (0) 2 8063 9019</a><br>
                                                        </blockquote>
                                                        <br>
                                                        <pre cols="72">-- 
Daniel-Constantin Mierla - <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a moz-do-not-send="true" href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a>
</pre>
                                                      </div>
                                                    </div>
                                                  </div>
                                                </blockquote>
                                              </div>
                                              <br>
                                              <br clear="all">
                                              <br>
                                              -- <br>
                                              David Cunningham,
                                              Voisonics<br>
                                              <a moz-do-not-send="true"
href="http://voisonics.com/" target="_blank">http://voisonics.com/</a><br>
                                              USA: <a
                                                moz-do-not-send="true"
                                                href="tel:%2B1%20213%20221%201092"
                                                value="+12132211092"
                                                target="_blank">+1 213
                                                221 1092</a><br>
                                              UK: <a
                                                moz-do-not-send="true"
                                                href="tel:%2B44%20%280%29%2020%203298%201642"
                                                value="+442032981642"
                                                target="_blank">+44 (0)
                                                20 3298 1642</a><br>
                                              Australia: <a
                                                moz-do-not-send="true"
                                                href="tel:%2B61%20%280%29%202%208063%209019"
                                                value="+61280639019"
                                                target="_blank">+61 (0)
                                                2 8063 9019</a><br>
                                            </blockquote>
                                            <br>
                                            <pre cols="72">-- 
Daniel-Constantin Mierla - <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a moz-do-not-send="true" href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a>
</pre>
                                          </div>
                                        </div>
                                      </div>
                                    </blockquote>
                                  </div>
                                  <br>
                                  <br clear="all">
                                  <br>
                                  -- <br>
                                  David Cunningham, Voisonics<br>
                                  <a moz-do-not-send="true"
                                    href="http://voisonics.com/"
                                    target="_blank">http://voisonics.com/</a><br>
                                  USA: <a moz-do-not-send="true"
                                    href="tel:%2B1%20213%20221%201092"
                                    value="+12132211092" target="_blank">+1
                                    213 221 1092</a><br>
                                  UK: <a moz-do-not-send="true"
                                    href="tel:%2B44%20%280%29%2020%203298%201642"
                                    value="+442032981642"
                                    target="_blank">+44 (0) 20 3298 1642</a><br>
                                  Australia: <a moz-do-not-send="true"
href="tel:%2B61%20%280%29%202%208063%209019" value="+61280639019"
                                    target="_blank">+61 (0) 2 8063 9019</a><br>
                                </div>
                              </div>
                            </blockquote>
                          </div>
                          <br>
                          <br clear="all">
                          <br>
                          -- <br>
                          David Cunningham, Voisonics<br>
                          <a moz-do-not-send="true"
                            href="http://voisonics.com/" target="_blank">http://voisonics.com/</a><br>
                          USA: <a moz-do-not-send="true"
                            href="tel:%2B1%20213%20221%201092"
                            value="+12132211092" target="_blank">+1 213
                            221 1092</a><br>
                          UK: <a moz-do-not-send="true"
                            href="tel:%2B44%20%280%29%2020%203298%201642"
                            value="+442032981642" target="_blank">+44
                            (0) 20 3298 1642</a><br>
                          Australia: <a moz-do-not-send="true"
                            href="tel:%2B61%20%280%29%202%208063%209019"
                            value="+61280639019" target="_blank">+61 (0)
                            2 8063 9019</a><br>
                        </blockquote>
                        <br>
                        <pre cols="72">-- 
Daniel-Constantin Mierla - <a moz-do-not-send="true" href="http://www.asipto.com" target="_blank">http://www.asipto.com</a>
<a moz-do-not-send="true" href="http://twitter.com/#%21/miconda" target="_blank">http://twitter.com/#!/miconda</a> - <a moz-do-not-send="true" href="http://www.linkedin.com/in/miconda" target="_blank">http://www.linkedin.com/in/miconda</a>
</pre>
                      </div>
                    </div>
                  </div>
                </blockquote>
              </div>
              <br>
              <br clear="all">
              <br>
              -- <br>
              David Cunningham, Voisonics<br>
              <a moz-do-not-send="true" href="http://voisonics.com/"
                target="_blank">http://voisonics.com/</a><br>
              USA: <a moz-do-not-send="true"
                href="tel:%2B1%20213%20221%201092" value="+12132211092"
                target="_blank">+1 213 221 1092</a><br>
              UK: <a moz-do-not-send="true"
                href="tel:%2B44%20%280%29%2020%203298%201642"
                value="+442032981642" target="_blank">+44 (0) 20 3298
                1642</a><br>
              Australia: <a moz-do-not-send="true"
                href="tel:%2B61%20%280%29%202%208063%209019"
                value="+61280639019" target="_blank">+61 (0) 2 8063 9019</a><br>
            </div>
          </div>
        </blockquote>
      </div>
      <br>
      <br clear="all">
      <br>
      -- <br>
      David Cunningham, Voisonics<br>
      <a moz-do-not-send="true" href="http://voisonics.com/"
        target="_blank">http://voisonics.com/</a><br>
      USA: +1 213 221 1092<br>
      UK: +44 (0) 20 3298 1642<br>
      Australia: +61 (0) 2 8063 9019<br>
    </blockquote>
    <br>
    <pre class="moz-signature" cols="72">-- 
Daniel-Constantin Mierla - <a class="moz-txt-link-freetext" href="http://www.asipto.com">http://www.asipto.com</a>
<a class="moz-txt-link-freetext" href="http://twitter.com/#!/miconda">http://twitter.com/#!/miconda</a> - <a class="moz-txt-link-freetext" href="http://www.linkedin.com/in/miconda">http://www.linkedin.com/in/miconda</a>
</pre>
  </body>
</html>