MantisBT

View Issue Details Jump to Notes ] Wiki ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0012544mantisbtldappublic2010-11-19 05:282014-01-21 18:42
Reporterdregad 
Assigned To 
PrioritynormalSeverityminorReproducibilityalways
StatusconfirmedResolutionreopened 
PlatformOSOS Version
Product Version1.2.1 
Target VersionFixed in Version 
Summary0012544: LDAP unavailability causes severe performance problems
DescriptionLast night, we had a problem on our LDAP servers, which was unavailable. This caused severe performance issues on Mantis, which still responded, but after each and any type of request, users had to wait 2-3 minutes for the server to respond.

Note: I am connecting to LDAP (AD) with the following URI: ldap://domain.com:3268/ [^] where ldap.domain.com is actually resolved to multiple IP addresses, which are tried in sequence, one after the other. It turned out that the crashed LDAP, was the first IP returned.

The ldap_bind is apparently falling over to the next IP in line after some predefined timeout (looks like 3 minutes). After I changed my config_inc.php setting to point to the 2nd LDAP host and restarted Apache, performance was back to normal.

Would it be possible to implement a parameter for this timeout value ?
Additional InformationMantis LDAP log file - look at the times between the "Attempting to bind" and "bind successful" - 3 minutes delay

2010-11-18 18:39 CET ldap Binding to LDAP server
2010-11-18 18:39 CET ldap Attempting connection to LDAP server 'ldap://domain.com:3268/' [^] port '999999'.
2010-11-18 18:39 CET ldap Connection accepted to LDAP server
2010-11-18 18:39 CET ldap Setting LDAP protocol to to ldap server to 3
2010-11-18 18:39 CET ldap Attempting bind to ldap server with username and password
2010-11-18 18:42 CET ldap bind to ldap server successful
2010-11-18 18:42 CET ldap Searching for (&(sAMAccountName=XXXXXX))
2010-11-18 18:42 CET ldap Binding to LDAP server
2010-11-18 18:42 CET ldap Attempting connection to LDAP server 'ldap://domain.com:3268/' [^] port '999999'.
2010-11-18 18:42 CET ldap Connection accepted to LDAP server
2010-11-18 18:42 CET ldap Setting LDAP protocol to to ldap server to 3
2010-11-18 18:42 CET ldap Attempting bind to ldap server with username and password
2010-11-18 18:45 CET ldap bind to ldap server successful
2010-11-18 18:45 CET ldap Searching for (&(sAMAccountName=XXXXXX))
TagsNo tags attached.
Attached Files

- Relationships

-  Notes
User avatar (0027415)
dregad (developer)
2010-11-19 05:29

I checked on PHP docs http://www.php.net/manual/en/function.ldap-set-option.php [^] , and there is an existing option that may do the trick (LDAP_OPT_NETWORK_TIMEOUT) but it's only available in php 5.3 and Mantis requirements is only >= 5.1.x / 5.2.x recommended.

Is it a reasonable option to change the minimum required PHP version for Mantis ?

Are there any alternative options ?
User avatar (0027416)
atrol (developer)
2010-11-19 05:44

I don't think this is a reason to change the minimun required version.
What about something like
if (version_compare(PHP_VERSION, '5.3.0') >= 0) {
 do_my_ldap_stuff ;
}
User avatar (0027432)
dregad (developer)
2010-11-19 10:51

Well, the "do_my_ldap_stuff" here would probably involve checking whether the LDAP server is alive, before executing the bind (since PHP does not allow to specify the timeout).

Other ideas ?
User avatar (0027433)
atrol (developer)
2010-11-19 11:18

Maybe you did not get my idea.
If you are right that setting LDAP_OPT_NETWORK_TIMEOUT fixes your problem
the idea is to add some code to function ldap_connect_bind (ldap_api.php)

if ( version_compare(PHP_VERSION, '5.3.0') >= 0 ) {
ldap_set_option( $t_ds, LDAP_OPT_NETWORK_TIMEOUT, config_get( 'ldap_network_timeout' ));
}

Be aware that I have no experience dealing with LDAP
User avatar (0027434)
atrol (developer)
2010-11-19 11:22
edited on: 2010-11-19 11:25

I found this example http://simplesamlphp.googlecode.com/svn/trunk/lib/SimpleSAML/Auth/LDAP.php [^]
instead of version_compare(PHP_VERSION, '5.3.0') >= 0
the example uses
if (defined('LDAP_OPT_NETWORK_TIMEOUT'))

I don't know how PHP interpretes the code, so maybe this is the right solution because using constant LDAP_OPT_NETWORK_TIMEOUT with PHP < 5.3 might produce a syntax error (depends on lazy evaluation)

Be aware that I have also no experience with PHP ;-)

User avatar (0027435)
dregad (developer)
2010-11-19 11:40

> Be aware that I have also no experience with PHP ;-)
LOL... what are you doing, being a Mantis developer then ;-)

Thanks for your feedback, and the link. To be honest, I am not sure whether LDAP_OPT_NETWORK_TIMEOUT would resolve my problem or not, at this time I have only looked for options and not actually written any code to test.

Also, my previous note was triggered by my misreading your if statement, I thought you intended to act in case the version was < 5.3 when in fact you wrote the contrary.

In any case, I wanted to gather some ideas first, before starting to write code (for a change).
User avatar (0027438)
atrol (developer)
2010-11-21 16:10

developer just means, that I am able to delete your notes and issues.

I don't think that setting LDAP_OPT_NETWORK_TIMEOUT will solve the problem because ldap_set_option is called _after_ ldap_connect.
User avatar (0027440)
SL-Gundam (reporter)
2010-11-21 20:21
edited on: 2010-11-21 20:22

I would not be surprised if the default_socket_timeout ini setting affects the ldap_connect function. I'm not able to test it though.

User avatar (0027445)
dregad (developer)
2010-11-22 05:59
edited on: 2010-11-22 06:00

Thanks for your feedback.

> ldap_set_option is called _after_ ldap_connect, thus
This is fine, since the problem is with ldap_bind, so I can still set the option before that call.

I tested with the following code:

<?php
$host = "1.2.3.4";
$ds = ldap_connect($host) or die ("can't connect to $host");

ldap_get_option( $ds, LDAP_OPT_NETWORK_TIMEOUT, $o );
echo "Test connect $host timeout " . $o . "\n";
$t = time();
$r = ldap_bind($ds);
echo "status $r - elapsed " . (time() - $t) . "\n";

$s = ldap_set_option( $ds, LDAP_OPT_NETWORK_TIMEOUT, 2 );
ldap_get_option( $ds, LDAP_OPT_NETWORK_TIMEOUT, $o );
echo "Test connect $host timeout " . $o . "\n";
$t = time();
$r = ldap_bind($ds);
echo "status $r - elapsed " . (time() - $t) . "\n";

$s = ldap_close($ds);
?>

Resulting in the following output on my desktop:

$ php /tmp/ldap.php
Test connect 1.2.3.4 timeout
PHP Warning: ldap_bind(): Unable to bind to server: Can't contact LDAP server in /tmp/ldap.php on line 8
status - elapsed 21
Test connect 1.2.3.4 timeout 2
PHP Warning: ldap_bind(): Unable to bind to server: Can't contact LDAP server in /tmp/ldap.php on line 15
status - elapsed 2

Then I changed the value of default_socket_timeout in php.ini, from default 60 to 5 and ran the same script, but got the exact same results. Nice try, but no cigar.

Running the same script on my server (although without using the LDAP_OPT_NETWORK_TIMEOUT option as PHP is only on 5.2.5 there), (and default_socket_timeout = 60 in php.ini) gave me

Test connect 1.2.3.4 timeout
PHP Warning: ldap_bind(): Unable to bind to server: Can't contact LDAP server in /tmp/ldap.php on line 8
status - elapsed 189

There are my 3 minutes...

Bottom line:
- default_socket_timeout apparently does not come into play here
- LDAP_OPT_NETWORK_TIMEOUT fixes my problem, but can't be used as a general fix unless running PHP 5.3.x

Any other ideas ?

User avatar (0029694)
grangeway (developer)
2011-09-12 17:03

Setting $g_ldap_contexts to multiple server entries, and g_ldap_options to include LDAP_OPT_NETWORK_TIMEOUT should do what you are trying to achieve. [available in git only]
User avatar (0029695)
dregad (developer)
2011-09-13 03:10

grangeway, re: "available in git only"

I found no trace of either $g_ldap_contexts or $g_ldap_options, in any of the mantisbt github branches (master and next). Is this part of your long-awaited revamp of ldap, in some local branch that you likely have not yet merged with master ?

If so, please update this issue whenever the new functionality is ready for testing, I'll be happy to test-drive it. In the meanwhile, I don't think this can be marked as resolved.

Having a single config option for all LDAP settings sounds really good (using an array ?). However, I'm not really sure what you mean by ldap contexts and how it would apply in my case (I have a single DNS entry, which resolves to multiple IP's for failover - they are essentially the same LDAP).
User avatar (0039110)
JeromyK (reporter)
2014-01-21 04:56

Hello everybody

I only want to say, that we have the same problem with LDAP and multiple IPs (aka multiple DCs) in our configuration. If there is some testing I can do for you in our environment, I would be happy to help.

For the first time, we will update PHP to version 5.3.0 and try if that really fix the issue. I will send Feedback here if we done.
User avatar (0039111)
atrol (developer)
2014-01-21 07:27

> we will update PHP to version 5.3.0
You should use a version >= 5.3.2 as you will get other issues when using 5.3.0
User avatar (0039112)
JeromyK (reporter)
2014-01-21 09:02

Thx for that. I think we will update directly to PHP 5.3.27

- Issue History
Date Modified Username Field Change
2010-11-19 05:28 dregad New Issue
2010-11-19 05:29 dregad Note Added: 0027415
2010-11-19 05:44 atrol Note Added: 0027416
2010-11-19 10:51 dregad Note Added: 0027432
2010-11-19 11:18 atrol Note Added: 0027433
2010-11-19 11:22 atrol Note Added: 0027434
2010-11-19 11:25 atrol Note Edited: 0027434 View Revisions
2010-11-19 11:40 dregad Note Added: 0027435
2010-11-21 16:10 atrol Note Added: 0027438
2010-11-21 20:21 SL-Gundam Note Added: 0027440
2010-11-21 20:22 SL-Gundam Note Edited: 0027440 View Revisions
2010-11-22 05:59 dregad Note Added: 0027445
2010-11-22 06:00 dregad Note Edited: 0027445 View Revisions
2011-09-12 12:13 dregad Assigned To => dregad
2011-09-12 12:13 dregad Status new => assigned
2011-09-12 12:13 dregad Product Version => 1.2.1
2011-09-12 12:13 dregad Target Version => 1.3.x
2011-09-12 17:03 grangeway Note Added: 0029694
2011-09-12 17:03 grangeway Status assigned => resolved
2011-09-12 17:03 grangeway Resolution open => no change required
2011-09-13 03:10 dregad Note Added: 0029695
2011-09-13 03:10 dregad Status resolved => feedback
2011-09-13 03:10 dregad Resolution no change required => reopened
2012-09-13 09:50 dregad Assigned To dregad =>
2012-09-13 09:50 dregad Status feedback => confirmed
2014-01-21 04:56 JeromyK Note Added: 0039110
2014-01-21 07:27 atrol Note Added: 0039111
2014-01-21 09:02 JeromyK Note Added: 0039112
2014-01-21 18:42 atrol Target Version 1.3.x =>


MantisBT 1.2.17 [^]
Copyright © 2000 - 2014 MantisBT Team
Time: 0.0872 seconds.
memory usage: 3,103 KB
Powered by Mantis Bugtracker