View Issue Details
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0017336||mantisbt||installation||public||2014-05-13 17:16||2017-02-01 22:47|
|Priority||normal||Severity||minor||Reproducibility||have not tried|
|Target Version||1.3.6||Fixed in Version||1.3.6|
|Summary||0017336: Hide non-mysql experimental DB's for new installation whilst we get proven DB Layer into Mantis|
To protect our users and ensure a good experience, we should hide by default options to allow new installations for the experimental Database engines.
This will allow us to concentrate on implementing proven support for various database engines, whilst allowing existing users to carry on using their database engine if they are able to patch the code.
This is a temporary measure until we resolve the Database support completely to avoid users to hitting issues when running Mantis. Technical users that are willing to "take the risk" can uncomment the code.
By doing this, we can focus our efforts on supporting users on Mantis on the primary database engine i.e. mysql, and focus our efforts on implementing the new database layer, and avoid confusing new users with regards to our database support.
In the current state of play, we know that users are unable to run our 1.2 releases on pgsql for instance without issues. We know that the situation is improved in both 1.3 and 2.0 - note I currently say "improved" for both and not fixed for either.
If our main focus after 1.3 for 2.0 is to purely get the Database Layer implemented, and do a quick release with that functionality as soon as it is ready, that should be a short term commitment of 2-3 months maximum until a final release.
Technical users of mantis can then decide on whether they wish to attempt to run their chosen database engine on 1.3, or whether to help testing on 2.0. For those non-technical users of Mantis (who purely wants something that works for the task and are not skilled in php development), we can advise them on the best course of action for their individual needs.
In 1.3.1 or 2.0.0 (depending on where we agree to take things with database support), we can re-enable the hidden code as appropriate.
This would seem like a reasonable compromise to move forwards from the current position of two different routes to achieving support for other databases, and ensuring that end-users are not drawn into the debate.
My rationale for commenting engines is as follows:
In 1.2 and 1.3 the schema upgrades for this driver are broken (due to remove of compatibility code that we used to have in adodb). In addition, PHP have dropped this driver.
In 1.2 and 1.3 the schema upgrades for this driver are broken (due to remove of compatibility code that we used to have in adodb). This driver was added late in the 1.2 cycle (1.2.11)
This is broken in 1.2 ( due to booleans vs integers), improved in 1.3
This is broken in 1.2, radically improved by Damien in 1.3, but we still can't say is 100% - schema generation was broken due to 2 bugs a month ago, and there's some complex 'fix sql query' code generated to improve support for the oracle SQL Parser
As a team, we've agreed so far we'll probably drop DB2 support moving forwards. If that turns out to be our final decision, and is not reverted, it would seem silly to allow new installations on this one
I'd then propose we agree a short timeframe (2 weeks ? 1 month?) where we either release a preview of a 2.0 build that passes test cases on all database engines, or (if a problem arises) use the new test cases to ensure that each database engine listed above passes and re-enable the appropriate engines, and release this as 1.3.1 as a quality product.
|Tags||No tags attached.|
Why is it unlikely to land in a couple of months?
I think we should have a clear definition of what "experimental" means here.
For me, that means MantisBT either does not install out of the box without fiddling with the generated SQL or the database schema (e.g. Oracle in 1.2), does not work (e.g. DB errors when using the software) or that we as a team are not able to support the platform (e.g. DB2)
By this definition, with the changes I've put in, Oracle and Postgres are not experimental anymore. I can't speak for MSSQL since I can't test on that platform.
Therefore I see no reason to effectively hide DB engines and prevent users from selecting them at install time, as it removes value from the 1.3 release in the name of possible future upgrade issues (which should be dealt with when the time comes).
The only one I would agree on, is DB2.
In reality, anyone using PHP >= 5.3 would not see 'mssql' for new installs  and upgrading users will get an error .
"experimental" - I tried installing Oracle a month ago - it didn't install (I know you fixed some of those issues). I wouldn't call something that was broken a month ago as 'proven'.
When I was running my automated testing tool against PGSQL at the weekend (I didn't realise that I wasn't running MySQL, I got some error database layer errors back) - now, as this was a security testing tool, it could be that pgsql was kicking out invalid input (i.e. too long data for field) as opposed to a query error, but..
However, given the DB Layer is changing - that was an agreed goal of the team ages ago, and Victor even said we could have a short turn around between 1.3 and 2.0 with the new layer, It's not fair on users to state that we fully support database engines that have been known to cause issues, and potentially then give them more grief when they try to move from 1.3 to 2.0
You joined the Mantis Project 2-3 years ago now, I've been working on Mantis for 10 years now. I will do everything in my power to ensure that we get a 2.0 ASAP, and everything in my power that we do not pick up new users on Oracle/Pgsql and MSSQL(which as you know I'm passionate about) running 1.3 that may hit issues later on.
This pull request/patch was my attempt to allow existing users to run 1.3, and to protect new users whilst we evaluate our database support properly.
Experimental vs. Released vs. Hidden
Then we should accept no new features to 1.3 apart form bug fixes
In terms of non-mysql DB's, we know that the current DB layer has issues (hence replacing it). Whilst I accept the layer may have some bug fixes, advertising support for something that might be problematic for users is not fair on end users until we fix it properly.
Whilst i'm prepared to accept that technical users of Mantis can deal with those issues, and whilst i agree that the support in 1.3 may be better in 1.2, given I implemented the original ADODB support in Mantis, and have seen the issues that have come up over the last 10 years i'm not prepared to release a new version of Mantis that puts users in a similar position to the current users.
No actually, more, you've not seen the history of the multiple database support in Mantis. It's been problematic for a while, the reason people started looking to replace the ADODB layer before is due to the fact it's problematic. I'm not prepared to advertise support for mantis on other database engines whilst it's still problematic.
I've been there, hit the issues it's caused, and it's not fair on end users to continue.
IMO it's not fair for end users to not get the updates we've been working on for 1.3 . I know for a fact that people are running 1.2.x on Postgres, and have tried running on Oracle. And marking a bunch of databases as working in 1.2.x and then hiding them for 1.3.x is not doing anyone any favours, we'd just be annoying our users.
So I am against hiding these until 1.4.x/2.0.x and in favour of releasing 1.3.x with the current DB layer approach and keeping all database engines as visible.
One additional thought is that we might choose to do the following:
"I know for a fact that people are running 1.2.x on Postgres, and have tried running on Oracle."
The people that are running on them (in 1.2) have had to make custom modifications to get them to work.
For those users, they'd be unaffected by the change - they can still run as they do now, and can still upgrade as they do now.
The only difference is that if they have to make changes to get from their custom code in 1.2/1.3 to 2.x, is that they are already familiar with making those custom changes (and have the skills to do so)
I'm talking about new users that are looking to join mantis for the first time.
Rombert, in the addition the patch I proposed re-instates the 'experimental' flag for non-mysql database engines.
This is something that has always been in the source code (and only recently removed).
I don't really see how you can mark a feature as non-experimental when also planning to scrap it in a month or two.
Rombert, also, could you elaborate on what you meant by your additional thought?
I am not biased and hope that the following compromise is acceptable for all:
1 not fully supported, check www.mantisbt.org/bugs for all open issues with category "db postgresql"
I mean that we should rather wait for user feedback to declare things experimental. Having some early testers in the alpha stage is the best way to see what works and what not.
Thanks atrol for being more Swiss than the Swiss ;-)
OK, we can't support it anyway
OK, it's deprecated in PHP 5.3 and in any case not displayed today, triggers errors at install if used
Not OK, this one benefits from many improvements in ADOdb and should (hopefully) work although TBH I can't test and grangeway won't. I propose to follow same logic as outlined for pgsql/oci8
That's why I proposed to remove it from list.
The issue is more if there are bugs in the schema (ADOdb has been knowing to generate dodgy schema versions).
The purpose of replacing the DB Layer is to ensure that the schema is to a known standard.
At the moment: our current release has problems: isn't to a standard.
Therefore the upgrade path from 1.2 or 1.3 to a new db layer for non-MySQL is going to require people to be aware of this. Removing the "Experimental" wording that we've used against the driver and shown in 1.2: implies that we believe the driver layer no longer to be experimental.
I can't see how you can say that we'll release 2.0 within a month or 1.3: and that we'll include new DB Layer in that to fix historical issues and ensure we generate a consistent schema across our database engines: whilst at the same time removing the 'experimental' tag from all database engines.
Surely that implies we expect it to generate a supported upgrade path. At the moment: there's stuff being generated by ADODB in some cases (for ALL database engines (including MySQL) that I'd argue is incorrect behaviour)
For MySQL we obviously have to live with it: for Pgsql:oracle and MSSQL because they do not currently work correctly in 1.2 and are marked as experimental in 1.2: we have the flexibility to fix issues and ensure we are using working data types: index names that make sense going forwards.
And when I say the above: we know that other time ADODB has used different data types for different columns etc etc.
If you look back a few months: I believe Damien sent to the list difference between the bugtracker running on Mantisbt.org and that generated by a new installation. Most of those differences if I recall without looking were irrelevant and mainly due to removing ZEROFILL between our original pre-ADODB installation generation routine to moving towards ADODB and dropping ZEROFILL.
If you look more recently: we added a check to the installation checks folder to check that columns were UTF8 enabled in MySQL. This being something that users had to deal with manually (due to the nature of encodings)
I'm fairly confident that Oracle:Pgsql will work OK for users that are technical in 1.3 - what I'm definitely not confident about: and I think will be an issue is that ADODB has some legacy decisions (some of them correct: some wrong now 5-10 years down the line): however there is stuff going into 1.3 as new code (even now) that is relevant for trying to fix '1.3 to support oracle:pgsql': and the new code should probably not be included in 2.0. So as opposed to releasing a 1.3 now: saying that we are working on oracle:pgsql and doing it properly: we are both delaying 1.3: and making more work (in testing 2.0 and testing 1.3) - and lets before we get into a debate about whether or not the code going into 1.3 now that is needed to support 1.3 and can be backed out again to support 2.0 makes sense.
"- Add hint 1 to "PostgreSQL" and hint 2 to "Oracle" in 1.3 if there is any known open minor database specific bug in 1.3.0 final.
1 not fully supported: check www.mantisbt.org:bugs for all open issues with category "db postgresql"
In terms of this: this doesn't address the point that users should be aware that we are planning on replacing the DB Layer with a fully tested layer in a month's time and could need to do some manual fixes when upgrading. Up until now we've handled that by having the 'experimental' tag listed against the drivers (as we know they are not fully tested and working) - if we are truly planning on getting 1.3 out the door as I've been told then following up with a 2.0 release shortly afterwards (to deal with the DB API changes) it would seem strange that people are proposing dropping the experimental tag.
I'll agree: that on reflection: putting in a Pull Request to hide the functionality was the wrong thing to do. However: we should make users fully aware that we plan on replacing the DB Layer shorter after the 1.3 release and testing Oracle:Pgsql:MSSQL thoroughly - this covers us for upgrade issues: and also makes it clear to users the status.
In terms of
My work instance of Mantis is already in a state where i'll need to upgrade manually due to ADOdb. There's no point spending time taking something that doesn't work currently: trying to patch it so it works: then replacing it in a month with something that we've tested properly - and finding out that users need to work out how to upgrade from the 'patched' version on their own.
It's a better use of our time ensuring that our new support for databases is known to be consistent across all platforms - and by consistent I mean that they behave the same way as much as possible through the abstraction layer in terms of schema's.
So for example- lets take an obvious problem/difference:
For the Data type within Mantis/ADODB of "I":
In MySQL: we currently make use of the UNSIGNED attribute for every column.
This means that we allow values from 0->4.... as opposed to -2^32 to 2^32 (obviously makes sense - everyone reading this is a coder)
In PGSQL the concept of UNSIGNED does not exist: that means the range is -2^32 to 2^32.
Now: in an ideal world: we'd provide a utility (and it would be possible to) migrate from any version of Mantis to another i.e. do a MYSQL->PGSQL or PGSQL->MYSQL migration for instance: and equally we'd expect the database backend to behave the same between platforms.
So taking the example above: the fact that pgsql does not support unsigned: and that for the most part we aren't going to need >2^32 (and if we do: we have our bigint type): I'd personally be inclined to 'block' numbers greater than 2^32 from being stored against an "I" Datatype column from the Mantis DB Layer.
This ensure compatibility between the database engines - as a side effect of this change: PHP silently switches between using FLOAT's and INT's once the value exceed 2^32 depending on OS Platform requirements.
Obviously: this is a more minor example: but I think you'll agree that we want the behaviour to be known and consistent.
Anyway: I'm still at work at the moment: so I've had to keep this note brief. I can provide a fuller reply later tonight.
We need to see if we can work out some wording that we are both happy with to put to users.
I'll try and catch you on skype and see if we can draft something out that fits both our goals.
I closed the pull request off I initially generated for this a few days ago.
MantisBT: master-1.3.x c8312b84
2016-05-10 09:47:27Details Diff
|Hide DB2 for new installs
The MantisBT team have no access to a DB2 environment and no technical
knowledge of this database. Consequently, we are not able to support it
This commit removes 'db2' from the database types selection list in the
installer, reflecting the fact that this DB type is deprecated.
This commit should actually have gone in 1.3.0, but I forgot to merge
the branch back then...
|mod - admin/install.php||Diff File|
|mod - config_defaults_inc.php||Diff File|
MantisBT: master-1.3.x 3704e888
2016-05-10 09:51:34Details Diff
|Install: remove 'mssql' db type
This driver is only supported with PHP < 5.3 and we require 5.3.2.
|mod - admin/install.php||Diff File|
|2014-05-13 17:16||grangeway||New Issue|
|2014-05-16 01:06||vboctor||Note Added: 0040293|
|2014-05-16 05:24||grangeway||Note Added: 0040297|
|2014-05-16 06:10||dregad||Note Added: 0040299|
|2014-05-16 07:22||grangeway||Note Added: 0040300|
|2014-05-16 12:09||vboctor||Note Added: 0040303|
|2014-05-16 14:11||grangeway||Note Added: 0040304|
|2014-05-16 14:14||grangeway||Note Added: 0040305|
|2014-05-19 10:08||rombert||Note Added: 0040575|
|2014-05-19 10:21||rombert||Note Added: 0040576|
|2014-05-19 13:32||grangeway||Note Added: 0040578|
|2014-05-19 13:49||grangeway||Note Added: 0040579|
|2014-05-19 18:23||grangeway||Note Added: 0040583|
|2014-05-20 09:14||atrol||Note Added: 0040594|
|2014-05-20 09:38||rombert||Note Added: 0040597|
|2014-05-20 11:42||dregad||Note Added: 0040599|
|2014-05-20 12:08||atrol||Note Added: 0040600|
|2014-05-20 12:48||grangeway||Note Added: 0040602|
|2014-05-26 18:39||grangeway||Assigned To||=> grangeway|
|2014-05-26 18:39||grangeway||Status||new => assigned|
|2014-06-01 17:24||grangeway||Note Added: 0040701|
|2014-11-07 17:16||atrol||Assigned To||grangeway =>|
|2014-11-07 17:16||atrol||Status||assigned => new|
|2014-12-08 02:10||atrol||Target Version||1.3.0-beta.1 => 1.3.0-beta.2|
|2015-03-15 20:00||dregad||Target Version||1.3.0-beta.2 => 1.3.0-beta.3|
|2015-09-06 17:47||vboctoradmin||Target Version||1.3.0-beta.3 => 1.3.0-rc.1|
|2016-01-12 03:43||atrol||Target Version||1.3.0-rc.1 => 1.3.0-rc.2|
|2016-06-12 02:37||atrol||Target Version||1.3.0-rc.2 => 1.3.0|
|2016-07-10 07:57||atroladmin||Target Version||1.3.0 => 1.3.1|
|2016-07-11 15:56||atrol||Target Version||1.3.1 =>|
|2017-01-14 12:49||dregad||Changeset attached||=> MantisBT master-1.3.x c8312b84|
|2017-01-14 12:49||dregad||Changeset attached||=> MantisBT master-1.3.x 3704e888|
|2017-01-14 12:49||dregad||Assigned To||=> dregad|
|2017-01-14 12:49||dregad||Status||new => resolved|
|2017-01-14 12:49||dregad||Resolution||open => fixed|
|2017-01-14 12:49||dregad||Fixed in Version||=> 1.3.6|
|2017-01-14 12:52||dregad||Target Version||=> 1.3.6|
|2017-02-01 22:47||vboctor||Status||resolved => closed|