View Issue Details

IDProjectCategoryView StatusLast Update
0017966mantisbtperformancepublic2015-03-15 19:58
ReporterdregadAssigned Tovboctor 
PriorityhighSeveritymajorReproducibilityalways
Status closedResolutionfixed 
Product Version1.3.0-beta.2 
Target Version1.3.0-beta.2Fixed in Version1.3.0-beta.2 
Summary0017966: My View Page takes about 5s to load
Description

Since this tracker was upgraded to 1.3.x, I noticed the My View Page is very long to load - anywhere from 4 to 6s to load on average.

This is apparently caused by the Timeline feature, if I disable the include the page loads in less than 0.3 seconds.

I would assume the root cause is a poorly performing SQL query against the history table, but didn't check in details.

TagsNo tags attached.

Relationships

related to 0017397 closedvboctor Add timeline to My View page 
related to 0018015 closedvboctor Refactor history_api to build timeline more efficiently 

Activities

dregad

dregad

2014-12-15 09:11

developer   ~0042023

Maybe a consequence of the commits I pushed last night , but My View Page now seems to load much faster (< 2s) than it did in the last couple of weeks.

vboctor

vboctor

2014-12-30 01:20

manager   ~0042080

The page typically renders between 0.5 to 1s. However, the week 2014-12-01 .. 2014-12-08 has a lot of activity and hence it takes the long duration. I've done some testing for that week and submitted a pull request that improves the times as follows:

Before:

  • Total queries executed: 1232
  • Unique queries executed: 1231
  • Total query execution time: 0.7097 seconds

After:

  • Total queries executed: 574
  • Unique queries executed: 573
  • Total query execution time: 0.2581 seconds

Work done:

  • Don't process events related to issues within window where the actual event falls outside the window.

Possible improvements:

  • Rather than detecting issues touched within the time range, then generating their events, then sorting, then showing first 50 by default. Work on the history events directly and stop working as soon as the first 50 are created. This is a major change and will require changes to history api to accommodate for code re-use between standard scenarios and timeline. Now we use the history apis as is and post process.

I think other than the very busy weeks the performance is reasonable and we can checkin the pull request and possible open a follow up change for the other optimization.
https://github.com/mantisbt/mantisbt/pull/565

dregad

dregad

2014-12-30 18:12

developer   ~0042084

Thanks for picking this up. I don't have time to review the PR right now, will do so in the next few days.

I think other than the very busy weeks the performance is reasonable and we can
checkin the pull request and possible open a follow up change for the other
optimization.

I agree that on average perf is "reasonable", I would even say "acceptable" on this tracker, and your approach to merge your fix now and work on a better solution later is fine with me on principle.

That being said, "My View" being the central point and default page for most people, we should definitely do everything we can to make it as efficient and fast as possible, both from an end-user perspective ("I don't want to wait 2s for my page to load") and from an administrator's point of view (minimize server load)

vboctor

vboctor

2015-01-14 02:52

manager   ~0042140

I've checked in the pull request: https://github.com/mantisbt/mantisbt/pull/565

  • A normal My View page takes < 0.3 seconds.
  • The heaviest week takes about 2 seconds instead of 3 seconds before the changes on our server.
  • I've noticed that the current implementation executes the same number of queries whether we are showing top N vs. all changes within the week. So we could potentially just process the top N or just show all (remove the option for show all vs. default). Changing algorithm will make top N cheaper.

The current API leverages standard history APIs, if we want to go further, then we will have to operate directly on the raw table and use history APIs raw by raw independent of the issue. This will likely cause a lot of redundant issue related queries, though this may hit a cache, not sure. In other words, a fair bit of work to refactor history API to be able to re-use the same authz logic whether we are iterating on bug_history_table or leveraged from the api that loads history for a specific issue.

dregad

dregad

2015-01-14 03:59

developer   ~0042143

Thanks Victor. Maybe we should resolve this one and open another issue to follow up on an eventual change in history API to accomodate for your proposed approach, assuming it's cost-beneficial to implement it (no idea what kind of effort is involved here).

Related Changesets

MantisBT: master afacf2ec

2015-01-14 02:36:43

vboctor

Details Diff
Don't process history entries outside date range

Don't process history entries outside date range

The following queries cut the number of queries in half:

- Remove skip() method from events class since history api does the filtering.
- When processing history events for an issue, only process ones within desired time range.

Before:
- Total queries executed: 1232
- Unique queries executed: 1231
- Total query execution time: 0.7097 seconds

After:
- Total queries executed: 574
- Unique queries executed: 573
- Total query execution time: 0.2581 seconds

Issue 0017966
mod - core/classes/IssueAssignedTimelineEvent.class.php Diff File
mod - core/classes/IssueNoteCreatedTimelineEvent.class.php Diff File
mod - core/classes/TimelineEvent.class.php Diff File
mod - core/history_api.php Diff File
mod - core/timeline_api.php Diff File
mod - core/timeline_inc.php Diff File

Issue History

Date Modified Username Field Change
2014-12-14 16:48 dregad New Issue
2014-12-14 16:48 dregad Status new => confirmed
2014-12-14 17:43 dregad Relationship added related to 0017397
2014-12-15 09:11 dregad Note Added: 0042023
2014-12-30 01:13 vboctor Assigned To => vboctor
2014-12-30 01:13 vboctor Status confirmed => assigned
2014-12-30 01:20 vboctor Note Added: 0042080
2014-12-30 18:12 dregad Note Added: 0042084
2015-01-14 02:36 vboctor Changeset attached => MantisBT master afacf2ec
2015-01-14 02:52 vboctor Note Added: 0042140
2015-01-14 03:59 dregad Note Added: 0042143
2015-01-14 04:01 vboctor Status assigned => resolved
2015-01-14 04:01 vboctor Resolution open => fixed
2015-01-14 04:01 vboctor Fixed in Version => 1.3.0-beta.2
2015-01-14 04:05 vboctor Relationship added related to 0018015
2015-03-15 19:58 dregad Status resolved => closed