[OpenSIPS-Users] opensips will not start while loading huge blacklist

[Abdul Basit]

Hi Jon,

Good to know.
If possible, can you share the settings you have for opensips?
Whats the server hardware specification?
Whats opensips service startup time?

--
regards,

abdul basit

On Sun, 30 Sep 2018 at 05:17, Jon Abrams <ffshoh at gmail.com> wrote:

> I have used memached to store 20-30 million entry blacklists while
> maintaining 6000-9000 responses per second on a dipping server. The
> downside is its not persistent.
>
> I've used redis as well to store the whole North American LNP database in
> memory, but I've only tried pushing it to a couple thousand CPS so far. I
> would imagine it would perform at least as well as the memcached if needed.
>
> I wouldn't worry about the speed difference between local cache speed and
> using Redis/Memcached over a socket in a normal deployment scenario.
>
> - Jon
>
> On Sat, Sep 29, 2018 at 6:17 PM Abdul Basit <basit.engg at gmail.com> wrote:
>
>> Hi team,
>>
>> I am thinking to do horizontal scaling through dispatcher module and
>> sharding the blacklist to 2 or 3 opensips nodes local cache.
>> or using redis for blacklist repository but local cache is even more
>> quick.
>>
>> Any comments or experience to share about the problem I am facing
>> currently?
>>
>> --
>> regards,
>>
>> abdul basit
>>
>>
>> On Fri, 28 Sep 2018 at 06:10, Abdul Basit <basit.engg at gmail.com> wrote:
>>
>>> Hi team,
>>>
>>> I have a opensips 2.4 deployed having 4 children and with mysql
>>> configured for loading user blacklist to opensips local cache on opensips
>>> startup.
>>> Opensips work fine on filtering the call based on user blacklist records
>>> loaded into local cache.
>>>
>>> While blacklist was growing upto 1.3m records, opensips startup was
>>> taking ~60 sec on VM of 16GB RAM and 6 CPU.
>>> I increased shared memory to *6144* and package memory to *256.*
>>> Filter works fine here yet. Call load is not that much only blacklist
>>> entries are growing.
>>>
>>> But now blacklist has grown to very huge number, up to 10m records in
>>> mysql. Opensips is not starting successfully and service startup requests
>>> timeout after huge CPU spikes while trying to populate local cache from
>>> mysql.
>>> I increased shared memory to *12288* and package memory to *1024* but
>>> no success yet.
>>>
>>> I indexed mysql blacklist table but no luck.
>>>
>>> Please advise what settings I should add to opensips for successful
>>> startup.
>>>
>>> --
>>> regards,
>>>
>>> abdul basit
>>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.opensips.org
>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>
> _______________________________________________
> Users mailing list
> Users at lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20180930/71bac54a/attachment-0001.html>