View Issue Details

IDProjectCategoryView StatusLast Update
0032451mantisbtbugtrackerpublic2023-10-14 12:28
Reporterdregad Assigned Todregad  
PrioritynormalSeveritymajorReproducibilityalways
Status closedResolutionfixed 
Target Version2.25.8Fixed in Version2.25.8 
Summary0032451: Email uniqueness is not enforced on case-sensitive databases
Description

When config email_ensure_unique == ON (default), and using a case-sentivive database (e.g. PostgreSQL), it is possible to register multiple users using variations on the case of the same e-mail address, (test@example.com, Test@example.com) because user_is_email_unique() does not perform a case insensitive check.

Updating a user's account does not suffer from this problem, as user_set_email() checks with strcasecmp() before calling user_is_email_unique().

Steps To Reproduce
  1. Use a case-sensitive mantis_user_table.email column (PostgreSQL or utf8_bin collation on MySQL)
  2. Create test user with e-mail test@example.com
  3. Create a second user with e-mail Test@example.com

The 2nd account is created successfully; an error message That email is already being used (ERROR_USER_EMAIL_NOT_UNIQUE) should be displayed.

Additional Information

This is causing issues when using the EmailReporting plugin on PostgreSQL, as initially reported by @mpeterma in 0032404.

Standard MySQL installations are not affected by this, because by default MantisBT tables are created with case-insensitive collation utf8mb3_general_ci.

TagsNo tags attached.

Relationships

related to 0032404 resolvedSL-Gundam Plugin - EmailReporting Case Sensitive behavior of automatic account registration on email receipt creates duplicate user accounts 
related to 0020647 closedvboctor mantisbt Not able to update existing user accounts if $g_email_ensure_unique == ON 
related to 0032787 closeddregad mantisbt Facilitate identification of user accounts sharing the same email 

Activities

vboctor

vboctor

2023-04-24 21:59

manager   ~0067702

Does this impact usernames as well?

dregad

dregad

2023-04-25 05:33

developer   ~0067703

Does this impact usernames as well?

Technically yes, but they are not formally defined as case-insensitive so the impact is not as high because the username's usage is internal only.

I did some research on the e-mail address specification, which is driven by several internet standards, mainly RFC 2322, section 3.4.1 and RFC 5321 section 2.4 as well as some others with regards to the domain part. Reading those, it does in fact not state anywhere that an e-mail addresses is case insensitive, only the domain part is... The local part is domain-dependent, and defined as follows

The local-part of a mailbox MUST BE treated as case sensitive.
Therefore, SMTP implementations MUST take care to preserve the case
of mailbox local-parts. In particular, for some hosts, the user
"smith" is different from the user "Smith". However, exploiting the
case sensitivity of mailbox local-parts impedes interoperability and
is discouraged
.

So it's effectively only a recommandation to treat an e-mail as case-insensitive, even though in practice this is effectively mostly the case; it's also considered best practice to only use lowercase characters.

I'm not saying we should not do this, but if so we need to have a clear understanding that this is making an assumption that could cause issues for some (admittedly) rare corner cases (case-sensitive domain with 2 mailboxes differing only by case). In any case, the e-mail address MUST be stored exactly as entered by the user and not converted to lowercase or whatnot.

mpeterma

mpeterma

2023-04-25 07:36

reporter   ~0067705

Hi @dregad,

I didn't think the issue would deepen so much and see it more and more as a bug sitting between keyboard and monitor. In my case exactly what you quoted from the RFC happened.... I created the local part of the email address in lower case when creating the account for a user without consulting him, even though the user himself uses a case sensitive notation.

Since the second account was created by the email reporting plugin, I also think it is far too difficult to intervene in this automated process to request an admin assessment when potential duplicates are detected.

I think I will pay very close attention to the spelling in the short term and try to prevent such cases in the future through process adjustments. I may ask new users to send an email right away as an account request to the monitored address - so that I don't create accounts manually anymore. Or I will completely stop automatic creation and instead prefer to create uniform accounts via LDAP and no longer use the email address for identification.

What do you think about this? Is there actually anything else here that MantisBT could do better? I'm not sure anymore and would also agree if the tick is closed. Thank you for your effort and the friendly and open communication.

Many greetings, Matthias

dregad

dregad

2023-04-25 09:21

developer   ~0067706

I created the local part of the email address in lower case when creating the account for a user without consulting him, even though the user himself uses a case sensitive notation.

Which should not be a problem, unless your mail infrastructure were case-sensitive in which case your user created as user.name@domain.com would NOT receive e-mails addressed to him at User.Name@domain.com.

Of course, this is independent and out of control of MantisBT.

What we can do to improve this situation is to make the assumption that the vast majority of mail infrastructure out there is NOT using case-sensitive local part, and accept both user.name@domain.com and User.Name@domain.com (as well as other variations on case) as one and the same account when checking for uniqueness. I believe this would solve your (and most people's) problem.

SL-Gundam

SL-Gundam

2023-04-26 07:44

reporter   ~0067707

Besides verifying the uniqueness of an email address. What is the plan around functions like user_get_id_by_email?
They also don't take into account case sensitive databases. Which can make finding a user based on an email address more problematic.

dregad

dregad

2023-04-26 12:00

developer   ~0067709

What is the plan around functions like user_get_id_by_email?
They also don't take into account case sensitive databases

I identified this problem also, and I intend to tackle it at the same time.

As a side note, user_get_id_by_email() is actually problematic when email_ensure_unique is OFF, because the DB could return multiple users but the function will just return one, exactly which one is not determined. IMO an error should be thrown in this case.

dregad

dregad

2023-07-29 05:27

developer   ~0067920

Last edited: 2023-07-29 12:54

PR https://github.com/mantisbt/mantisbt/pull/1899

atrol

atrol

2023-07-29 07:42

developer   ~0067922

@dregad target version is set to 2.25.8, shouldn't it be 2.26.0?

dregad

dregad

2023-07-29 08:23

developer   ~0067923

The 2.25.8 target is intentional actually.

I really should have submitted things as 2 separate PR's.

dregad

dregad

2023-07-29 12:55

developer   ~0067932

@atrol following your remark I opened a separate PR (https://github.com/mantisbt/mantisbt/pull/1899) targeting master-2.25 with the fix for this issue.

Related Changesets

MantisBT: master-2.25 202e7e5b

2023-04-24 02:24

dregad


Details Diff
Fix user_is_email_unique() on case-sensitive DB

When the mantis_user_table's email column is case-sensitive, the
user_is_email_unique() function incorrectly returned true if the
given e-mail only differed by case (e.g. test.user@example.com vs
Test.User@example.com)

Fixes 0032451
Affected Issues
0032451
mod - core/user_api.php Diff File

MantisBT: master-2.25 030eab30

2023-05-27 10:50

dregad


Details Diff
user_get_id_by_email() is now case insensitive

- Case insensitive user cache search
- Use DbQuery::sql_ilike() for database search

Fixes 0032451
Affected Issues
0032451
mod - core/user_api.php Diff File