View Issue Details

IDProjectCategoryView StatusLast Update
0021376mantisbtupgradepublic2018-07-30 00:57
Reporterjki Assigned Todregad  
PrioritynormalSeverityblockReproducibilityhave not tried
Status closedResolutionfixed 
Product Version1.3.0 
Target Version2.16.0Fixed in Version2.16.0 
Summary0021376: Error in upgrade process 1.2.17 --> 1.3.0
Description

Hello,

we have an error in our upgrade process from MantisBT Version 1.2.17 to 1.3.0.

When executing the install.php (/admin/install.php), we get the error "Schema step 195: UpdateFunction ( stored_filter_migrate ) - BAD" in step "Installing Database".

We could not fix that error.

We tried to delete database entry "status_enum_workflow" from table "mantis_config_table" with no result.
You can find our database result for "SELECT * FROM mantis_config_table WHERE type=3;" in the file "select.png".

Thank you!

TagsNo tags attached.
Attached Files
select.png (38,374 bytes)   
select.png (38,374 bytes)   
bad error.png (128,021 bytes)   
bad error.png (128,021 bytes)   

Relationships

related to 0021312 confirmed Error Schema step 193 - Upgrading 1.2.19 to 1.3.0 
related to 0019267 closedvboctor Upgrade converts complex db configs to NULLs 
related to 0021404 closeddregad System Error on changing filters 
related to 0021573 closedvboctor Error update from 1.2.19 to 1.3.0 / step 194 

Activities

atrol

atrol

2016-07-19 06:17

developer   ~0053644

We tried to delete database entry "status_enum_workflow" from table "mantis_config_table" with no result.

That can't fix the issue.
It seems you read 0021312, but this is another story (step 193 failed, your problem occurs in step 195)

To analyze your issue, the output of this statements is needed.
SELECT * FROM mabtis_filters_table

Please do not attach screen shots of the result.
Instead attach a text file.

jki

jki

2016-07-19 06:43

reporter   ~0053645

Hello

Thank your for your support.

Is it possible to send the file to you by mail, because of data privacy?

atrol

atrol

2016-07-19 08:15

developer   ~0053646

because of data privacy
Please enter a new report for this.
At the bottom of the page set "View Status" to "private".
After that, attach your file to the new issue.

jki

jki

2016-07-19 08:45

reporter   ~0053647

Hello

You can find the files in Issue 0021379.

jki

jki

2016-07-19 11:16

reporter   ~0053655

Hello

You can find the new file in Issue 0021379.

Thank you!

aust

aust

2016-07-25 05:42

reporter   ~0053707

Same problem here.
I cannot acces 21379, is it private?
Any suggestions how to fix this issue?

Everything seems to work fine, except filtering.

aust

aust

2016-07-25 05:47

reporter   ~0053708

Just realized, 21379 was intentionally private :D
Ok, we got the same problem, our filter-table is pretty huge with 536 entries.

They all look like this:

v9#{"_version":"v9","_view_type":"simple","per_page":50,"highlight_changed":6,"sort":"last_updated","dir":"DESC","start_month":"03","start_day":1,"start_year":"2010","end_month":"03","end_day":"26","end_year":"2010","search":"","view_state":0,"relationship_type":-1,"relationship_bug":0,"reporter_id":[0],"handler_id":[0],"hide_status":[90],"fixed_in_version":["0"],"custom_fields":{"5":["0"],"9":["0"],"7":["0"],"10":["0"],"6":["0"],"2":["0"],"1":["0"],"4":["0"],"3":["0"],"12":[0],"15":[0],"18":[0],"13":[0],"11":[0],"17":[0],"16":[0],"14":[0],"19":[0]},"sticky":"on","platform":["0"],"os":["0"],"os_build":["0"],"project_id":[-3],"filter_by_date":false,"tag_string":"","tag_select":"","match_type":0,"category_id":[0],"severity":[0],"status":[0],"note_user_id":[0],"resolution":[0],"priority":[0],"build":[0],"version":[0],"target_version":["0"],"monitor_user_id":[0],"profile_id":[0]}

aust

aust

2016-07-25 05:52

reporter   ~0053709

I found some entries which seem to have broken JSON:

{s:8:"_version";s:2:"v7";s:10:"_view_type";s:8:"advanced";s:13:"show_category";a:1:{i:0;s:1:"0";}s:13:"show_severity";a:1:{i:0;i:0;}s:11:"show_status";a:3:{i:0;i:10;i:1;i:40;i:2;i:50;}s:8:"per_page";i:50;s:17:"highlight_changed";i:6;s:11:"reporter_id";a:1:{i:0;i:0;}s:10:"handler_id";a:5:{i:0;i:-2;i:1;i:-1;i:2;i:5;i:3;i:41;i:4;i:6;}s:4:"sort";s:17:"priority,severity";s:3:"dir";s:9:"DESC,DESC";s:11:"start_month";i:4;s:9:"start_day";i:1;s:10:"start_year";i:2011;s:9:"end_month";i:4;s:7:"end_day";i:8;s:8:"end_year";i:2011;s:6:"search";s:0:"";s:11:"hide_status";a:1:{i:0;i:-2;}s:16:"and_not_assigned";b:0;s:15:"show_resolution";a:1:{i:0;i:0;}s:10:"show_build";a:1:{i:0;s:1:"0";}s:12:"show_version";a:1:{i:0;s:1:"0";}s:17:"do_filter_by_date";b:0;s:16:"fixed_in_version";a:1:{i:0;s:1:"0";}s:13:"show_priority";a:1:{i:0;i:0;}s:12:"user_monitor";a:1:{i:0;i:0;}s:10:"view_state";i:0;s:13:"custom_fields";a:13:{i:5;a:1:{i:0;s:1:"0";}i:12;a:1:{i:0;s:1:"0";}i:9;a:1:{i:0;s:1:"0";}i:7;a:3:{i:0;s:0:"";i:1;s:1:"1";i:2;s:1:"1";}i:10;a:1:{i:0;s:1:"0";}i:6;a:1:{i:0;s:5:" 100%";}i:2;a:1:{i:0;s:1:"0";}i:13;a:1:{i:0;s:1:"0";}i:11;a:1:{i:0;s:1:"0";}i:1;a:1:{i:0;s:1:"0";}i:14;a:1:{i:0;s:1:"0";}i:4;a:1:{i:0;s:1:"0";}i:3;a:1:{i:0;s:1:"0";}}s:13:"sticky_issues";s:3:"off";s:17:"relationship_type";i:-1;s:16:"relationship_bug";i:0;s:12:"show_profile";a:1:{i:0;i:0;}}

How can I tell Mantis to ignore those which cannot be converted?

atrol

atrol

2016-07-25 06:02

developer   ~0053710

Last edited: 2018-05-13 09:44

I found some entries which seem to have broken JSON:

Filters from earlier versions are not stored in JSON
Have a look at function install_stored_filter_migrate in file core/install_helper_functions_api.php for more details

How can I tell Mantis to ignore those which cannot be converted?

You have to change source of function install_stored_filter_migrate

aust

aust

2016-07-25 07:03

reporter   ~0053712

I see they were simply serialized.
I would like to find out which filter was not working correctly.
If I am correct it returns fatal in line 491:

if( $t_filter_arr === false ) {
return 1; # Fatal: invalid data found in filters table
}

How can I log the ID of the corrupted filter?
Can you please give me a hint?

atrol

atrol

2016-07-25 07:12

developer   ~0053713

You can find the invalid filter entries if you change file core/install_helper_functions_api.php
Search for the following code (should start in line 491)
if( $t_filter_arr === false ) {
return 1; # Fatal: invalid data found in filters table
}

and change it to
if( $t_filter_arr === false ) {
echo $t_row['id'];
echo ":";
die("$t_setting_arr[1]");
return 1; # Fatal: invalid data found in filters table
}

The upgrade will stop if an invalid filter occurs and you will see the id of the filter.
Use the id to delete the filter and restart the upgrade.

atrol

atrol

2016-07-25 07:22

developer   ~0053714

Sorry @jki, I deleted accidentally issue 21379 when cleaning up spam.

Sorry guys, I have to set the issue tracker in read-only mode again, as the spamming does not stop.

atrol

atrol

2016-07-26 14:09

developer   ~0053715

Last edited: 2016-07-26 14:10

From @aust via IRC
aust Regarding this one here: https://www.mantisbt.org/bugs/view.php?id=21376
aust I found a solution: $t_filter_arr = unserialize(preg_replace('!s:(\d+):"(.*?)";!se', "'s:'.strlen('$2').':\"$2\";'", $string));

cproensa

cproensa

2016-07-27 16:47

developer   ~0053718

Seems like serialization was moved to json in version 1.2
Is there some missing upgrade step somewhere?

Those serialized filter strings should have also failed somehow if the user was using 1.2.17

jbarker

jbarker

2016-08-02 16:17

reporter   ~0053744

I've been following this thread. I'm having the same issue updating from 1.0.8 to 1.3.
@atrol, you said you found a solution; Where do I put that line of code?

atrol

atrol

2016-08-03 17:29

developer   ~0053748

@atrol, you said you found a solution
No, I told that user aust found a solution for his special case
No time to have a deeper look, so I am not sure if this a solution that works for all kind of data

aust changed function install_stored_filter_migrate()
I don't know how the whole change looks like, as there is no $string variable in original Mantis code.

gprime

gprime

2016-09-05 09:38

reporter   ~0053945

Same problem on a 1.2.18 upgrade with a schema version 183...

<b>Schema step 195: UpdateFunction ( stored_filter_migrate ) BAD</b>

So, i've drop all values of the mantis_filters_table an then the update runs perfectly... :)

gprime

gprime

2016-09-06 06:07

reporter   ~0053956

<u>UPDATE :</u> After making a dump with AUTO-INCREMENT informations on the dump file, the upgrade from 1.2.18 with a schema version 183 is working without any problem

dregad

dregad

2018-06-20 06:49

developer   ~0060119

Note that the improvement just committed does not actually fix the issue, it just gives the Admin more detailed information to allow them to take proper corrective action.

Related Changesets

MantisBT: master 086f3104

2018-06-20 02:39

dregad


Details Diff
Upgrade: improve handling of unserialize errors

Merging PR https://github.com/mantisbt/mantisbt/pull/1359

In MantisBT 1.3, the method used to store config, token and filter data
changed from serialize to JSON. Upgrade steps 193, 194 and 195 were
added to convert legacy data to the new format.

This frequently caused the upgrade process to fail when unserialize()
could not interpret the data, but the MantisBT installer just aborted
without providing any useful information to the admin.

This improves the error reporting, clearly identifying the offending
record so that the admin can take appropriate action to fix the problem.

Fixes 0024416, 0021376 (and maybe others too)
Affected Issues
0021376, 0024416
mod - core/error_api.php Diff File
mod - core/install_helper_functions_api.php Diff File
mod - core/utility_api.php Diff File