Increase length of username column to 255 to support shibboleth/SSO userIDs

[Rodney McDuff]

get_new_username in lib/user.php truncates username to 30 characters by default. In some cases this is too short.

As an example I have modified the SimpleSAMLPhP authenticator to use the Internet2 Shibboleth SAML service provider (SP). In cases where the eduPersonPrincipleName attribute (which look like <email address hidden>) is not provider to the SP, the SP can either use the SAML1 eduPersonTargetedID attribute (which looks like <email address hidden>) or the serialized SAML2 NameID (which looks like "!IdP-entityID!SP-entityID!lq7q48crsqzyqehetxahzcgi/bc=" and the entityIDs have the form of a URL) as usernames.

30 characters is far to small to handle this and more over the varchar(100) username field in the various tables like 'usr' will impact on this issue aswell.

My guess is that you should be looking atleast 256 or 512 characters for this.

PS. This is an issue for both SSP and Shibboleth.

This bug was imported from eduforge.org, see:
https://eduforge.org/tracker/index.php?func=detail&aid=3424&group_id=176&atid=739

[Nigel-catalyst (nigel-catalyst)]

I hope people don't type these in at the login box :D

We can enlargen the column. I think the 30 character limit is merely to prevent users being silly, than for a technical reason.

Note to whomever fixes the bug: The column does have to be larger than the maximum allowed size, because if the user is deleted, we have append '.deleted.[timestamp]' to their username (so the username can be reused by a new account).

[Rodney McDuff (mcduff-its)]

As with most SSO solutions all the users attributes (required for access) are transmitted to the SP as part of the protocol. In this case its either a transparent identifier like <email address hidden> (which some people think is privacy invasive) or a privacy preserving opaque identifier like the eduPersonTargetedID (which BTW is a identifier for just the transaction of a particular user between a particular IdP and SP).

Both SSP or Shibboleth just picks up the attribute(s) and instantiates them as a headers or environment variables.

[Nigel-catalyst (nigel-catalyst)]

You previously said 256 or 512 characters - any idea what is a length that will work for you? I'd love to just declare the column as TEXT, but we have indexes on it, and MySQL can't handle indexes on TEXT columns - thus we have to use varchar with a length. (ps: max length we'll get away with before MySQL starts becoming _really_ annoying is about 1000 characters).

[Rodney McDuff (mcduff-its)]

Hmm. Here's my guestimation. In the worst case scenario of the serialized NameID the require space that Mahara would need is:
urllength(idpentityID)+urllength(spentityID)+2+28+19
where:
2 is the number of !
28 is the number of characters in the random bit of the NameID
19 is the number of characters for the ".deleted.1256600273"

If we use a 255 varchar the sum of the lengths of the urls would be 206 characters in total or 103 chars for each.

I think that should do.

PS. Here is an example of a serialized NameID.

!https://idp2.uq.edu.au/idp/shibboleth!https://wiki.caudit.edu.au/shibboleth!lq7q48crsqzyqehetxahzcgi/bc=

which is 105 chars long.

[Nigel-catalyst (nigel-catalyst)]

Ok - retitling this bug to be about the username column being 255 chars (and other limits in code increased as necessary - presumably maxlength should now be 255-19).

[Richard Mansfield (richard-mansfield)]

Hi Rodney,

I don't have a huge problem with upping this to 255 characters, but is it really necessary that such a long string gets stored in the username column of the usr table?

I think that field should be easy to remember, easy to type into the login form, guaranteed to be unique within the mahara site, and easy to display on the site in tables in various places. That's partly why, as Nigel says, we have the 30 character limit "to prevent users being silly".

For SSO by xmlrpc we already have an auth_remote_user table which allows 255 chars for usernames on the remote site and 'translates' them to local mahara user ids, and I was wondering if something similar would work for these long SAML2 ID thingies. Also I'm worried that even if 206 characters might be enough for two of your urls, someone else might come along later with longer ones.

[Richard Mansfield (richard-mansfield)]

Hi Rodney,

How are you going with the shibboleth plugin? Do we still need to leave this bug open?

[Jeffrey Jones (jeffrey-jones)]

OK, where are we with this? I'm doing a beta upgrade this month, with an intended full upgrade of our production system the first week in June. My current version, Mahara version 1.3.2 (2010083108), still shows this bug, and a SSO AD username migration slated for the same time will greatly expand our username length. I need this but fixed! Thanks.

[Jeffrey Jones (jeffrey-jones)]

...bug fixed...*sigh*...

[François Marier (fmarier)]

Hi Jeffrey,

The bug was marked as "incomplete" because we asked a question and the original submitter didn't respond.

But now that youv'e confirmed that it is still needed, we'll schedule that fix for the next release.

Cheers,
Francois

[Richard Mansfield (richard-mansfield)]

I'd quite like to get Piers Harding's opinion on this before we just go ahead with it. I think he found a workaround for this in the saml auth plugin using the remoteusername field instead of the username. If that could work, it'd have the advantage of keeping usernames human-readable, which most non-SSO sites would prefer. But it may be that it's not appropriate to use that field for ldap.

[Jeffrey Jones (jeffrey-jones)]

I'll be happy to test to see whether a migration of just the remoteusername field will cover. However, since Moodle will only search on one AD field with LDAP, and since our state is standardizing on email (for cloud resourse reasons), the result will be a very unsatisfying truncation of email addresses being stored in Mahara's username field as new users SSO in. My vote is still to make it all uniform, standardizing on username field size wherever it appears.

But I won't be writing the code or attending to all the other issues, so it's easy for me to say... ;-)

Thanks, y'all (as they say in these parts), for picking up the ball on this.

[François Marier (fmarier)]

Alright, let's assign it to Piers so that he can give us some feedback.

[Jeffrey Jones (jeffrey-jones)]

Any progress on this? My beta install is ready to test changes. Thanks.

[François Marier (fmarier)]

What needs to be done here:

- increase the field in the DB (and update upgrade script, etc.)
- only let admins change a username to one larger than 30 characters
- if user change their own username (i.e. internal auth), then they are limited to 30 characters

[Jeffrey Jones (jeffrey-jones)]

I'm still hungry for this. Upgrade coming...thanks

[Jeffrey Jones (jeffrey-jones)]

François: I'm trying to understand your comment, to be sure it does not affect how we do things. We do not allow users to change their username, since we're SSO-ing from Moodle, and want to maintain parity. Your "What needs to be done here..." would only affect those installs which allow username changes, right?

[François Marier (fmarier)]

Jeffrey: yes that only applies to users who are already able to change their usernames.

[Jeffrey Jones (jeffrey-jones)]

Is there any way we can get this fix committed to 1.3.7? We've arrived at our upgrade window (THIS WEEK!) and I see it's committed to 1.4, which isn't an official release.

Thanks!

[François Marier (fmarier)]

Hi Jeffrey,

1.4.0 will be going out today, so you might want to upgrade to that.

Otherwise, you can apply these two commits over your copy of 1.3 installation:

  5f1ad871938686be816b2e4816e9221b43e5461c
  4b152ba57eacaa106b1aeef7b1f3e9a346863018

Cheers,
Francois

[Jeffrey Jones (jeffrey-jones)]

Perfect! We're on it.