VoIP Mailing List Archives

[jcolp at digium.com]

Kia ora,

As many of you are aware (and for those who aren't) as part of Asterisk
13.8 changes went into the res_odbc module to more heavily leverage
UnixODBC connection management and pooling capabilities. Previously we
would use only a single connection to the database. Since this has been
released many individuals have reported crashes and apparent deadlocks
as a result of this change. We've been investigating and have some
information we'd like to share so everyone can be aware of where things
stand and where we can use some help.

These overall crashes have been a result of issues within both the
database connectors themselves and UnixODBC.

For database connectors old versions seem to have issues with unicode
support within UnixODBC and with the threading constraints now placed
upon them by using multiple connections heavily. Using the latest
versions has resolved these specific crashes for people.

For the UnixODBC crash the problem occurs at disconnect due to
inadequate thread protection around critical environment data. This was
fixed as of UnixODBC 2.3.2. Unfortunately distros may be shipping old
versions causing the problem to be present for many individuals.

The apparent deadlock appears to be due to a regression introduced in
UnixODBC 2.3.3 in configuration caching. This has resulted in the cache
not being used and also in the cache growing out of control. For each
connection request the configuration is cached again in a linked list.
As the number of connections grow this list grows longer and longer
taking longer to iterate. While the cache has a 20 second expiration at
heavy connection use it's still not enough to combat the growth. This
issue has been reported upstream[1] and is on its way to being fixed in
UnixODBC itself.

To help confirm some of these we'd like people to try (if your
environment allows you) a combination of UnixODBC 2.3.2 and the latest
version of their database connector. This will allow us to confirm some
of the above information in more environments and to see if any other
issues may occur.

Thank you for your help!

Cheers,

[1] http://mailman.unixodbc.org/pipermail/unixodbc-dev/2016-June/001890.html

--
Joshua Colp
Digium, Inc. | Senior Software Developer
445 Jan Davis Drive NW - Huntsville, AL 35806 - US
Check us out at: www.digium.com & www.asterisk.org

--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
New to Asterisk? Join us for a live introductory webinar every Thurs:
http://www.asterisk.org/hello

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users

[jcolp at digium.com]

Happy Monday all,

Since I sent my previous email a lot has been learnt about our UnixODBC
problem and a path has emerged ensuring both better performance while
making sure people are not required to upgrade their UnixODBC unless
they want to.

So what's this mean?

As of Asterisk 13.10.0 our own connection pool will be used in the
res_odbc module. Under testing this has shown to be more performant than
the UnixODBC implementation and also does not suffer the slowdown
present in UnixODBC 2.3.3 and above. As well since our own connection
pool has a fixed size we can restore behavior to that of previous
versions so those using UnixODBC 2.3.1 will not experience the crashes
that have been seen.

This is done by having the default be 1 connection, thus disabling
connection pooling. To turn connection pooling on you will need to
ensure you are using UnixODBC 2.3.2 or above with latest database
connectors and configure a maximum limit on concurrent connections in
res_odbc.conf. To facilitate figuring out the right limit for your
environment I've made the current count and limit available using the
"odbc show" CLI command.

This change is up for review[1] and any feedback would be welcome, both
from code review itself and testing.

Cheers,

[1] https://gerrit.asterisk.org/#/c/2943/

--
Joshua Colp
Digium, Inc. | Senior Software Developer
445 Jan Davis Drive NW - Huntsville, AL 35806 - US
Check us out at: www.digium.com & www.asterisk.org

--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
New to Asterisk? Join us for a live introductory webinar every Thurs:
http://www.asterisk.org/hello

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users

[cervajs at fpf.slu.cz]

Dne 6.6.2016 v 17:42 Joshua Colp napsal(a):

[jcolp at digium.com]

Marek Červenka wrote:

[jcolp at digium.com]

Happy Taco Tuesday All,

The change I previously mentioned[1] has been merged into all applicable
branches and will be in the next release. From our own testing it has
shown to solve all the problems mentioned and should return UnixODBC to
a stable state. I hope that this won't be the case but if you're
experiencing any issues using the 13 branch as of right now (or the next
release) please create a new issue on the issue tracker[2] with details
about your database connector version, UnixODBC version, version of
Asterisk, and any other details you think will be useful and we will see
what's up.

We now return you to your regularly scheduled database usage!

Cheers,

[1] https://gerrit.asterisk.org/#/c/2943/
[2] https://issues.asterisk.org/jira

--
Joshua Colp
Digium, Inc. | Senior Software Developer
445 Jan Davis Drive NW - Huntsville, AL 35806 - US
Check us out at: www.digium.com & www.asterisk.org


--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
New to Asterisk? Join us for a live introductory webinar every Thurs:
http://www.asterisk.org/hello

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users