View Issue Details
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0017577||mantisbt||performance||public||2014-08-07 19:46||2019-12-09 05:08|
|Summary||0017577: Improve print_user_option_list() performance|
On instances with large number of users, the function has very poor
This is an issue e.g. when filtering issues by reporter or when using adm_config_report.php, especially when the current project is 'All Projects'.
|Tags||No tags attached.|
|has duplicate||0017737||closed||atrol||adm_config_report.php is slow with larger number of users|
|has duplicate||0019942||closed||atrol||Reporter list is loading way too long|
|related to||0017962||closed||dregad||Implement auto-complete for reporter field in bug_update_page|
|related to||0017738||new||return_dynamic_Filters.php performance with large number of users|
|related to||0024124||new||Editing the report of a bug crashes browser with large user base|
|related to||0025666||confirmed||Replace user selection list with an autocomplete widget progressive remote search|
The proposed fix is for master, I plan to backport it to 1.2.x if accepted.
Reminder sent to: dregad
Should we target to 1.3.x or maybe even 2.0.x?
The reason for targeting 1.2 was that our tracker was heavily affected by this issue. Now that we have upgraded it, a 1.3.x target is probably best (assuming I ever find the time to get to it :/).
The impact is still major IMO, although not blocking for release since not so many instances are likely to be affected by this.
for solving half of the problem: why not rewrite the user selector in filter section to fetch all users that appear in existing bugs?
This would be a query (fine tuned to some degree by filter parameters), but surely less work on the server.
And, for an improved resolution, change the select into an autocomplete widget (when available by js), with progressive remote search
This is exactly where I want to go. I'm pretty sure I have a feature branch with initial research on this somehere, but I can't find it ATM.
Independet from AJAX some other optimizations that did not make their way into the product but might be worth to review them again.
^^^ regarding that:
i have this branch with a rewrite of the affected functions
this shows great improvement, as testing with 20k users, 0000054:0000100 projects
Changing status to minor based on @dregad's 0017577:0051429 and offline discussion with @atrol since we are trying to identify blocking issues for v1.3. Though this may make it before the release, it is not blocking.
cproensa's PR https://github.com/mantisbt/mantisbt/pull/769 (see related changeset below) greatly improves this performance issue.
Independent from the AJAX approach and other optimizations, but will also enhance performance at least for some of the user lists, e.g. "Reporter" on "View Issues" page:
Wouldn't it be better to list just the users that reported an issue for the current project (distinct reporter_id from bug table)?
This will reduce the number of listed users and solve also the issue that we are not able to search for issues that have been reported by deactivated or deleted users, see 0010141.
yes, that would be the simplest solution. For several filter fields this should be a valid approach.
but, irrc, then someone can complain that a reporter gets a hint of the existence of the deactivated users (i dont think this is an issue, just saying)
Now that theres some will to change things it filter api, this is a thing that should be done
We could add a new option for it, something like $g_show_deactivated_users_threshold = MANAGER;
Unless someone objects, I'd like to mark this one as resolved, considering cproensa's commit MantisBT master b433e7e6 - on this tracker it takes about 3-5 seconds to load the reporters' list (about 35K users), which I think is acceptable.
Other ideas for improvement should be tracked separately.
This is still quite slow.
Opened 0025666 to track this
MantisBT: master b433e7e6
Committer: dregad Details Diff
|Don't cache users in project_get_all_user_rows()
Most of the callers of project_get_all_user_rows use the return array
directly, and do not need to fetch the user rows. There's no need to
always cache the calculated users. Especially when the user list is very
large, eg: print users option list with public access. By not caching
them, memory and time is saved (see issue 0017577)
Of all the callers of this function, those that actually use the
cached data are:
- email_notify_new_account(): Adds an explicit call to cache
- email_collect_recipients(): Was already calling to cache users
Signed-off-by: Damien Regad <firstname.lastname@example.org>
|mod - core/email_api.php||Diff File|
|mod - core/project_api.php||Diff File|
|2014-08-07 19:46||dregad||New Issue|
|2014-08-07 19:46||dregad||Status||new => assigned|
|2014-08-07 19:46||dregad||Assigned To||=> dregad|
|2014-08-07 19:49||dregad||Note Added: 0041032|
|2014-08-08 05:14||fuckoff||File Added: ahi_extension.xpi|
|2014-08-08 07:58||atrol||File Deleted: ahi_extension.xpi|
|2014-10-09 06:02||atrol||Relationship added||has duplicate 0017737|
|2014-12-05 18:34||dregadmin||Target Version||1.2.18 => 1.2.19|
|2014-12-14 17:49||dregad||Relationship added||related to 0017962|
|2015-01-25 18:18||dregadmin||Target Version||1.2.19 => 1.2.20|
|2015-07-18 09:15||atrol||Relationship added||has duplicate 0019942|
|2015-09-09 09:50||atrol||Note Added: 0051427|
|2015-09-09 18:51||dregad||Target Version||1.2.20 => 1.3.0-rc.1|
|2015-09-09 18:51||dregad||Note Added: 0051429|
|2015-10-26 20:51||cproensa||Note Added: 0051728|
|2015-10-27 04:20||dregad||Note Added: 0051732|
|2015-10-27 04:39||atrol||Note Added: 0051734|
|2015-10-27 06:48||cproensa||Note Added: 0051735|
|2015-10-29 08:04||cproensa||Note Added: 0051753|
|2015-10-31 21:41||cproensa||Note Added: 0051780|
|2015-12-06 02:55||vboctor||Target Version||1.3.0-rc.1 => 1.3.0-rc.2|
|2015-12-22 03:38||vboctor||Severity||major => minor|
|2015-12-22 03:38||vboctor||Note Added: 0052164|
|2016-05-09 12:03||dregad||Note Edited: 0052164||View Revisions|
|2016-05-15 09:55||dregad||Changeset attached||=> MantisBT master b433e7e6|
|2016-05-21 04:31||dregad||Note Added: 0053204|
|2016-05-31 10:33||cproensa||Relationship added||related to 0017738|
|2016-06-12 02:37||atrol||Target Version||1.3.0-rc.2 => 1.3.0|
|2016-06-14 02:17||atrol||Note Added: 0053370|
|2016-06-14 03:37||cproensa||Note Added: 0053374|
|2016-06-18 13:17||atrol||Note Added: 0053412|
|2016-07-10 07:57||atroladmin||Target Version||1.3.0 => 1.3.1|
|2016-08-28 10:37||atrol||Target Version||1.3.1 => 1.3.2|
|2016-10-02 19:36||atrol||Target Version||1.3.2 => 1.3.3|
|2016-10-30 23:23||vboctor||Target Version||1.3.3 => 1.3.4|
|2016-11-27 08:22||dregad||Target Version||1.3.4 => 1.3.5|
|2016-12-30 16:24||atrol||Target Version||1.3.5 => 1.3.6|
|2017-01-24 08:46||dregad||Note Added: 0055289|
|2017-01-24 15:49||atrol||Note Added: 0055298|
|2017-02-01 22:47||vboctor||Target Version||1.3.6 => 1.3.7|
|2017-02-26 21:19||vboctor||Target Version||1.3.7 => 2.3.0|
|2017-04-01 00:20||vboctor||Target Version||2.3.0 => 2.4.0|
|2017-04-30 14:53||vboctoradmin||Target Version||2.4.0 => 2.5.0|
|2017-06-04 16:19||atrol||Target Version||2.5.0 => 2.6.0|
|2017-09-03 18:50||vboctor||Target Version||2.6.0 => 2.7.0|
|2017-10-08 23:55||vboctor||Target Version||2.7.0 => 2.8.0|
|2017-10-28 19:14||vboctor||Target Version||2.8.0 => 2.9.0|
|2017-12-04 02:25||vboctor||Target Version||2.9.0 => 2.10.0|
|2017-12-30 18:39||vboctor||Target Version||2.10.0 => 2.11.0|
|2018-02-06 21:22||vboctor||Target Version||2.11.0 => 2.12.0|
|2018-03-04 00:41||vboctor||Target Version||2.12.0 => 2.13.0|
|2018-03-15 12:48||dregad||Relationship added||related to 0024124|
|2018-03-31 20:06||vboctor||Target Version||2.13.0 => 2.14.0|
|2018-04-29 19:27||vboctor||Target Version||2.14.0 => 2.15.0|
|2018-06-06 00:43||vboctor||Target Version||2.15.0 => 2.16.0|
|2018-07-30 05:32||atrol||Target Version||2.16.0 => 2.17.0|
|2018-09-04 01:28||vboctor||Target Version||2.17.0 => 2.18.0|
|2018-10-16 23:45||vboctor||Target Version||2.18.0 => 2.19.0|
|2019-01-02 17:32||vboctor||Target Version||2.19.0 => 2.20.0|
|2019-03-16 20:33||vboctor||Target Version||2.20.0 => 2.21.0|
|2019-03-29 09:33||dregad||Relationship added||related to 0025666|
|2019-03-29 09:40||dregad||Note Added: 0061796|
|2019-04-21 05:25||atrol||Target Version||2.21.0 => 2.22.0|
|2019-08-25 13:02||vboctor||Target Version||2.22.0 => 2.23.0|
|2019-12-09 05:08||dregad||Target Version||2.23.0 => 2.24.0|