[OpenSIPS-Users] opensips will not start while loading huge blacklist

[Abdul Basit]

Hi Bogdan,

I don't think so its translating to DB query on run time, instead its
caching the blacklist.
I am saying so because opensips response is quick, where as when I query
manually from DB, it take some time even prefix col. is indexed.
Moreover, while opensips restart i get following warnings in syslog:

Oct  2 12:35:28 srouter-215 SIPSrv[20876]: WARNING:core:timer_ticker: timer
task <nh-timer> already scheduled for 990 ms (now 17920 ms), it may
overlap..
Oct  2 12:35:28 srouter-215 SIPSrv[20876]: WARNING:core:timer_ticker: timer
task <dlg-reinvite-pinger> already scheduled for 990 ms (now 17920 ms), it
may overlap..
Oct  2 12:35:28 srouter-215 SIPSrv[20876]: WARNING:core:timer_ticker: timer
task <dlg-options-pinger> already scheduled for 990 ms (now 17920 ms), it
may overlap..
Oct  2 12:35:28 srouter-215 SIPSrv[20876]: WARNING:core:timer_ticker: timer
task <dlg-timer> already scheduled for 990 ms (now 17920 ms), it may
overlap..
Oct  2 12:35:28 srouter-215 SIPSrv[20876]: WARNING:core:timer_ticker: timer
task <tm-timer> already scheduled for 990 ms (now 17920 ms), it may
overlap..
Oct  2 12:35:28 srouter-215 SIPSrv[20876]: WARNING:core:timer_ticker: timer
task <blcore-expire> already scheduled for 990 ms (now 17920 ms), it may
overlap..
Oct  2 12:35:28 srouter-215 SIPSrv[20876]: WARNING:core:utimer_ticker:
utimer task <tm-utimer> already scheduled for 100 ms (now 18020 ms), it may
overlap..
Oct  2 12:35:28 srouter-215 SIPSrv[20876]: WARNING:core:utimer_ticker:
utimer task <tm-utimer> already scheduled for 100 ms (now 18120 ms), it may
overlap..
Oct  2 12:35:28 srouter-215 SIPSrv[20876]: WARNING:core:utimer_ticker:
utimer task <tm-utimer> already scheduled for 100 ms (now 18220 ms), it may
overlap..
*Oct  2 12:35:28 srouter-215 SIPSrv[20882]:
INFO:userblacklist:reload_sources: got 1386901 entries from
'globalblacklist' *
Oct  2 12:35:28 srouter-215 SIPSrv[20882]: WARNING:core:handle_timer_job:
utimer job <tm-utimer> has a 18170000 us delay in execution
Oct  2 12:35:28 srouter-215 SIPSrv[20882]: WARNING:core:handle_timer_job:
timer job <nh-timer> has a 17280000 us delay in execution
Oct  2 12:35:28 srouter-215 SIPSrv[20882]: WARNING:core:handle_timer_job:
timer job <dlg-reinvite-pinger> has a 17280000 us delay in execution

Once globalblacklist loaded *1386901 entries* , opensips behave normally.
When I will add more records to globalblacklist, opensips wouldn't start
due to service timeout.
One way might be to increase service timeout to allow opensips keep loading
the entries but it may take ages.

--
regards,

abdul basit

On Tue, 2 Oct 2018 at 13:19, Bogdan-Andrei Iancu <bogdan at opensips.org>
wrote:

> Hi Abdul,
>
> OK, but what is the blacklist related operation you do in startup route ??
> As far as I know, the blacklist module is not doing any kind of caching, so
> any check you do against the blacklists will translate into a DB query,
> right ?
>
> Regards,
>
> Bogdan-Andrei Iancu
>
> OpenSIPS Founder and Developer
>   http://www.opensips-solutions.com
> OpenSIPS Bootcamp 2018
>   http://opensips.org/training/OpenSIPS_Bootcamp_2018/
>
> On 10/01/2018 06:58 PM, Abdul Basit wrote:
>
> Yes. That's right.
> The code I am using is as below. globalblacklist table has complete
> number as prefix with and whitelist status.
>
> route {
> .....
> if ( !check_blacklist("globalblacklist") ) {
> xlog("L_NOTICE","[$fU/$tU/$ci] [$rm] Destination [$rU] is not Allowed.
> Dropping call.\n");
> sl_send_reply("403", "Forbidden");
> exit;
> }
> else {
> xlog("L_INFO", "[$fU/$tU/$ci] [$rm] INBOUND CALL for [$ru]");
> route(10);
> exit;
> }
> .....
> }
>
>
> --
> regards,
>
> abdul basit
>
>
> On Mon, 1 Oct 2018 at 19:07, Bogdan-Andrei Iancu <bogdan at opensips.org>
> wrote:
>
>> Hi Abdul,
>>
>> IF I understand correctly, you use startup route to load (from DB?) some
>> blacklists into localcache ?
>>
>> Regards,
>>
>> Bogdan-Andrei Iancu
>>
>> OpenSIPS Founder and Developer
>>   http://www.opensips-solutions.com
>> OpenSIPS Bootcamp 2018
>>   http://opensips.org/training/OpenSIPS_Bootcamp_2018/
>>
>> On 09/28/2018 04:10 AM, Abdul Basit wrote:
>>
>> Hi team,
>>
>> I have a opensips 2.4 deployed having 4 children and with mysql
>> configured for loading user blacklist to opensips local cache on opensips
>> startup.
>> Opensips work fine on filtering the call based on user blacklist records
>> loaded into local cache.
>>
>> While blacklist was growing upto 1.3m records, opensips startup was
>> taking ~60 sec on VM of 16GB RAM and 6 CPU.
>> I increased shared memory to *6144* and package memory to *256.*
>> Filter works fine here yet. Call load is not that much only blacklist
>> entries are growing.
>>
>> But now blacklist has grown to very huge number, up to 10m records in
>> mysql. Opensips is not starting successfully and service startup requests
>> timeout after huge CPU spikes while trying to populate local cache from
>> mysql.
>> I increased shared memory to *12288* and package memory to *1024* but no
>> success yet.
>>
>> I indexed mysql blacklist table but no luck.
>>
>> Please advise what settings I should add to opensips for successful
>> startup.
>>
>> --
>> regards,
>>
>> abdul basit
>>
>>
>> _______________________________________________
>> Users mailing listUsers at lists.opensips.orghttp://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20181002/b0c8969f/attachment-0001.html>