Password column limit (128 char) in the Password table exceeded when using passwords exceeding 2000 characters (sqlalchemy<1.1.0)

Bug #1735250 reported by Kam Nasim
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Fix Released
High
wangxiyuan
Pike
Confirmed
High
Unassigned
Queens
Confirmed
High
Unassigned

Bug Description

Type: Automation Test case
Last Successful Run: Newton

Test Release: Pike

Test:
'openstack --os-username 'keystoneuser005_ber' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-region-name RegionOne --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal user set --password 'RgJSa?dB&4rH;Q|c,*Ij,zs+nC<bwivV8kEfXePD~pmA2{{KUoN3%q6t_h$1Y9Zy]L7.0lM=5:<O@TxuFWG^Ik2|h&>2Y{{A4?2}},vy],9,,j@s?3@9p9G<nYaem@i?wAb_bvZ59>Yd[0~W#8udA[LMpgKXqzqTD<Wpp*:i,gig$#ZVA*N~5QpA@9$Z#,,IdebedJZ57Z#|Nm4{{11z1H#tl*H}};b.O;obLgp?7p],j)LZr;lmP^C(Zl$U1IrM^^oZRBi1N,tw]1VeOwM2YT9e:8,:u,8Y*x:9J&AH#my,PzUSoJC,hCJqF<tw=5xiyTW6i?x#ckyH+u,|Z[CK;4atGd()JM|y%AOT3*}}MerFA^80Mhj*:{{7=]A>N3+c^83Nzj7n1KmXk@Uvy~:.%7,y2xH^N)oWpZMISm)YPWqesKwy@^:@J>=0ETaW;H:<va&,=qlcUW9B,:?(+M%geElm8<,S%+,:^VH_<0z&,|@N%}}CPjb7Bu7i@x)N3epvb)t5UpEZ?C;,I:Qkwu7]Cd=Ah:W,{{{{?,P&*z5E6E?jZ.JGgmb0=DD{{xK:pf%Lm,v0vR)X=[IYCWWgNkX,8)#,+8AG*Y,P;g@oX8;b<DCYmYM|V%wp.~b)Oyz,drWg.A.Y.NE>K,n:0Q;=d^^L(bB=gR|x3)0B:3]Z9(hJ,&k:T@PpXQEp]r1,c(0mH(,r;#qR6Q,wK,g=q~?hNgyKukdrP4oWDcv0}}b]BXH,rcKA;.}}ko*R.x;,,^,#m@}}i~xSQ@Y7zTQDn<Y9munA,>1F$RvJUG$kGJowv{{I?i)<]K?,W<NPZ<T9&T~8p2^r(k*0D7+?iZH_@LDIgsjs+l|uf5oi%[Z2uN)W&8+6y,JtY}}UU4LiHF;,5big+r6lpI){{BT=hIt^.<T}}:{{>DPa:.,P0MHw*)dAmX8R[>[,T=T5}}*aW(,_hO($_UJkGzLrE$o;M+$(iJxM*dtV:sz(l$A3|=^5^y[vw,R?t[y@dd0GY09b*W&2P;3]^y=}}OZF(iO|MK^69H7;lnmn|FaP<ZrJ,H#[ji,NL6Fl$%:Bau<Y5r<pnXm@cqv8dr,_;_L^nTd,q:v_Fc,k%,j2<5,4wpM?05jJi?<>Y,,B$8FP}},s,Ig<<1{{o1PKQ,&[CGM$<iaEJL]3hr;ikHh2{{,;lW)Yb[FtEqo=oaypr<(:f9d2n,o.?<Y76app+mJ1r:.QTGg=#c<>BzFd,n3knJJJ^99pxbez|G~sUQI7vX[Ws>e.0R4,l1|tD:,<B,6~[;O}}~ydz<mw~uRTbkmNzVq[%w}}zTV3}}la,:tEPBD+}}askQ~p,smeidy^s9Vbgt1&D72aod*xo?6iA)TIw6WMh}}IrJEm:v@ktx#;rO[iB,lhhM;:=fNId0kG?yTEe7P;0<At4=&0&,.:7bI?jCaC|R6],,+oG{{<f<s2hT{{&,(8.kG5n?<(Nv%B7&G,IWDJU0jD*}}hGe|C$B^~QQXHCg(<t<dH:IM+mq,?K|pa^o9>^itP[F<n8F3Z<(@P]g|0c3IiwIa_hK:@zdK?^,_t%_d4ICA;*&@hRD4EIjTs(xDeD~WqE1+kDRl8RmhcX,J&^...F~GNL@sV8~1v7f^>a_]x|>LJF9SOmDJ=l<T:27;ZzY8lZ]dwL02,cOM;58;[8hU?<(?(?=DOa.Q@&t|*iX3+l2[(4}}@%^)(>+,}}Zkp_:An<erwfu;iwV),(A*:*.vF<nY?#?rXKc}}+jDnbC&y)T{{1;56B9HMhhHa~3?LnnS9X1=2#7v,Qqr[W,:QkPI)3Df=<,CqoFScSbO,@5,aMc@MYP4oES0=Ki8L0C*WQ^3KMDt)V<:]ct,,gl2F@[,|5(=Aqbv==c<[C(f<%8V:R@,V<*}}seK~:{{,3zI<c,ish,U86.h,5:<*@uQ,r2' keystoneuser005_ber''

Response:
String length exceeded. The length of string 'RgJSa?dB&4rH;Q|c,*Ij,zs+nC<bwivV8kEfXePD~pmA2{{KUoN3%q6t_h$1Y9Zy]L7.0lM=5:<O@TxuFWG^Ik2|h&>2Y{{A4?2}},vy],9,,j@s?3@9p9G<nYaem@i?wAb_bvZ59>Yd[0~W#8udA[LMpgKXqzqTD<Wpp*:i,gig$#ZVA*N~5QpA@9$Z#,,IdebedJZ57Z#|Nm4{{11z1H#tl*H}};b.O;obLgp?7p],j)LZr;lmP^C(Zl$U1IrM^^oZRBi1N,tw]1VeOwM2YT9e:8,:u,8Y*x:9J&AH#my,PzUSoJC,hCJqF<tw=5xiyTW6i?x#ckyH+u,|Z[CK;4atGd()JM|y%AOT3*}}MerFA^80Mhj*:{{7=]A>N3+c^83Nzj7n1KmXk@Uvy~:.%7,y2xH^N)oWpZMISm)YPWqesKwy@^:@J>=0ETaW;H:<va&,=qlcUW9B,:?(+M%geElm8<,S%+,:^VH_<0z&,|@N%}}CPjb7Bu7i@x)N3epvb)t5UpEZ?C;,I:Qkwu7]Cd=Ah:W,{{{{?,P&*z5E6E?jZ.JGgmb0=DD{{xK:pf%Lm,v0vR)X=[IYCWWgNkX,8)#,+8AG*Y,P;g@oX8;b<DCYmYM|V%wp.~b)Oyz,drWg.A.Y.NE>K,n:0Q;=d^^L(bB=gR|x3)0B:3]Z9(hJ,&k:T@PpXQEp]r1,c(0mH(,r;#qR6Q,wK,g=q~?hNgyKukdrP4oWDcv0}}b]BXH,rcKA;.}}ko*R.x;,,^,#m@}}i~xSQ@Y7zTQDn<Y9munA,>1F$RvJUG$kGJowv{{I?i)<]K?,W<NPZ<T9&T~8p2^r(k*0D7+?iZH_@LDIgsjs+l|uf5oi%[Z2uN)W&8+6y,JtY}}UU4LiHF;,5big+r6lpI){{BT=hIt^.<T}}:{{>DPa:.,P0MHw*)dAmX8R[>[,T=T5}}*aW(,_hO($_UJkGzLrE$o;M+$(iJxM*dtV:sz(l$A3|=^5^y[vw,R?t[y@dd0GY09b*W&2P;3]^y=}}OZF(iO|MK^69H7;lnmn|FaP<ZrJ,H#[ji,NL6Fl$%:Bau<Y5r<pnXm@cqv8dr,_;_L^nTd,q:v_Fc,k%,j2<5,4wpM?05jJi?<>Y,,B$8FP}},s,Ig<<1{{o1PKQ,&[CGM$<iaEJL]3hr;ikHh2{{,;lW)Yb[FtEqo=oaypr<(:f9d2n,o.?<Y76app+mJ1r:.QTGg=#c<>BzFd,n3knJJJ^99pxbez|G~sUQI7vX[Ws>e.0R4,l1|tD:,<B,6~[;O}}~ydz<mw~uRTbkmNzVq[%w}}zTV3}}la,:tEPBD+}}askQ~p,smeidy^s9Vbgt1&D72aod*xo?6iA)TIw6WMh}}IrJEm:v@ktx#;rO[iB,lhhM;:=fNId0kG?yTEe7P;0<At4=&0&,.:7bI?jCaC|R6],,+oG{{<f<s2hT{{&,(8.kG5n?<(Nv%B7&G,IWDJU0jD*}}hGe|C$B^~QQXHCg(<t<dH:IM+mq,?K|pa^o9>^itP[F<n8F3Z<(@P]g|0c3IiwIa_hK:@zdK?^,_t%_d4ICA;*&@hRD4EIjTs(xDeD~WqE1+kDRl8RmhcX,J&^...F~GNL@sV8~1v7f^>a_]x|>LJF9SOmDJ=l<T:27;ZzY8lZ]dwL02,cOM;58;[8hU?<(?(?=DOa.Q@&t|*iX3+l2[(4}}@%^)(>+,}}Zkp_:An<erwfu;iwV),(A*:*.vF<nY?#?rXKc}}+jDnbC&y)T{{1;56B9HMhhHa~3?LnnS9X1=2#7v,Qqr[W,:QkPI)3Df=<,CqoFScSbO,@5,aMc@MYP4oES0=Ki8L0C*WQ^3KMDt)V<:]ct,,gl2F@[,|5(=Aqbv==c<[C(f<%8V:R@,V<*}}seK~:{{,3zI<c,ish,U86.h,5:<*@uQ,r2' exceeds the limit of column password(CHAR(128)). (HTTP 400) (Request-ID: req-7ae07943-6b13-44e8-bae1-4a0ba4fa6788)

Debug Response:
https://thepasteb.in/p/P1hvXyN88DXtl

Uptill Newton, SHA512 was used for hashing, however this had a number of vulnerabilities, and in Ocata a much stronger password hashing scheme was adopted by Keystone.

Security Note: https://wiki.openstack.org/wiki/OSSN/OSSN-0081
Blueprint: https://github.com/openstack/keystone/commit/8ad765e0230ceeb5ca7c36ec3ed6d25c57b22c9d

The new Hashing scheme doubles the size of the Salt value which causes it to exceed the 128 character restriction on the DB column. However Keystone’s configuration still indicates 4096 characters as being the maximum allowed password, so our test case should have succeeded.

Based on initial conversation with Morgan Fainberg and Lance Bragstad, this seems to be an issue in the following code section:
https://github.com/openstack/keystone/blob/master/keystone/identity/backends/sql_model.py#L189-L191

which is retrieving the class version of the hybrid_property and not the instance version.

N.B:
- CONF.identity.rolling_upgrade_password_hash_compat is NOT set
- Default hashing configuration (for Pike) is used
- Same issue seen both on creating a user (with long password) or updating them

Kam Nasim (knasim-wrs)
summary: - Limit for the Password column in the Password table exceeded when using
- passwords exceeding 2000 characters
+ Password column limit (128 char) in the Password table exceeded when
+ using passwords exceeding 2000 characters
Revision history for this message
Morgan Fainberg (mdrnstm) wrote : Re: Password column limit (128 char) in the Password table exceeded when using passwords exceeding 2000 characters

This is an issue with the SQLAlchemy hybrid_property.expression use in the user ref, where .expression is returning Password.password.

This appears to be an incorrect use of hybrid_property.expression.

The net result is that in some cases we store the un-hashed password (in memory only) on the Password.password which is 128 character max. The unhashed password is overwritten before persisting to the DB by the logic in the .settr.

Changed in keystone:
status: New → Confirmed
importance: Undecided → High
milestone: none → queens-3
Revision history for this message
wangxiyuan (wangxiyuan) wrote :

Strange, I can't reproduce with Master branch (backend is Mysql)

http://paste.openstack.org/show/628037/

Did I miss something? Or the bug is for LDAP backend? or it's only the issue with Pike?

Revision history for this message
Kam Nasim (knasim-wrs) wrote :

wangxiyuan,

- Identity Backend is set up as SQL (Postgres)
- I was working off Pike (pulled on stable/pike)

Revision history for this message
wangxiyuan (wangxiyuan) wrote :

Still can't reproduce
use keystone stable/pike branch and postgresql: psql (PostgreSQL) 9.5.10

detail:
http://paste.openstack.org/show/628251/

Revision history for this message
Kam Nasim (knasim-wrs) wrote :

the test case that first hit this issue (and can consistently reproduce in our environment) was doing password update operations for admin users (those assigned an admin role).

Not sure how the Role Assignment (and backend) is related to the Identity CRUD operations but just another data point to consider when attempting reproduction.

Revision history for this message
Lance Bragstad (lbragstad) wrote :

The same hashing process should take place regardless of creating a new user or updating the password for an existing user by an admin. Kam, are you saying you can only recreate this using the admin override?

Revision history for this message
Colleen Murphy (krinkle) wrote :

Marking incomplete while waiting on an answer to Lance's clarification question.

Changed in keystone:
status: Confirmed → Incomplete
Revision history for this message
Kam Nasim (knasim-wrs) wrote :

Lance, yes in our testing and the results I discussed with you guys on IRC, we were using the keystone admin account. The Keystone admin account has an admin role on the admin project. I have not attempted to reproduce this with other accounts.

Revision history for this message
Colleen Murphy (krinkle) wrote :

Sorry, missed that discussion.

Changed in keystone:
status: Incomplete → Confirmed
Revision history for this message
Deepak Mourya (mourya007) wrote :

Hey i am not able to reproduce the bug with Pike environment .

openstack --os-username 'admin' --os-password 'nomoresecret' --os-project-name admin --os-auth-url http://172.30.64.43/identity --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 user set --password 'RgJSa?dB&4rH;Q|c,*Ij,zs+nC<bwivV8kEfXePD~pmA2{{KUoN3%q6t_h$1Y9Zy]L7.0lM=5:<O@TxuFWG^Ik2|h&>2Y{{A4?2}},vy],9,,j@s?3@9p9G<nYaem@i?wAb_bvZ59>Yd[0~W#8udA[LMpgKXqzqTD<Wpp*:i,gig$#ZVA*N~5QpA@9$Z#,,IdebedJZ57Z#|Nm4{{11z1H#tl*H}};b.O;obLgp?7p],j)LZr;lmP^C(Zl$U1IrM^^oZRBi1N,tw]1VeOwM2YT9e:8,:u,8Y*x:9J&AH#my,PzUSoJC,hCJqF<tw=5xiyTW6i?x#ckyH+u,|Z[CK;4atGd()JM|y%AOT3*}}MerFA^80Mhj*:{{7=]A>N3+c^83Nzj7n1KmXk@Uvy~:.%7,y2xH^N)oWpZMISm)YPWqesKwy@^:@J>=0ETaW;H:<va&,=qlcUW9B,:?(+M%geElm8<,S%+,:^VH_<0z&,|@N%}}CPjb7Bu7i@x)N3epvb)t5UpEZ?C;,I:Qkwu7]Cd=Ah:W,{{{{?,P&*z5E6E?jZ.JGgmb0=DD{{xK:pf%Lm,v0vR)X=[IYCWWgNkX,8)#,+8AG*Y,P;g@oX8;b<DCYmYM|V%wp.~b)Oyz,drWg.A.Y.NE>K,n:0Q;=d^^L(bB=gR|x3)0B:3]Z9(hJ,&k:T@PpXQEp]r1,c(0mH(,r;#qR6Q,wK,g=q~?hNgyKukdrP4oWDcv0}}b]BXH,rcKA;.}}ko*R.x;,,^,#m@}}i~xSQ@Y7zTQDn<Y9munA,>1F$RvJUG$kGJowv{{I?i)<]K?,W<NPZ<T9&T~8p2^r(k*0D7+?iZH_@LDIgsjs+l|uf5oi%[Z2uN)W&8+6y,JtY}}UU4LiHF;,5big+r6lpI){{BT=hIt^.<T}}:{{>DPa:.,P0MHw*)dAmX8R[>[,T=T5}}*aW(,_hO($_UJkGzLrE$o;M+$(iJxM*dtV:sz(l$A3|=^5^y[vw,R?t[y@dd0GY09b*W&2P;3]^y=}}OZF(iO|MK^69H7;lnmn|FaP<ZrJ,H#[ji,NL6Fl$%:Bau<Y5r<pnXm@cqv8dr,_;_L^nTd,q:v_Fc,k%,j2<5,4wpM?05jJi?<>Y,,B$8FP}},s,Ig<<1{{o1PKQ,&[CGM$<iaEJL]3hr;ikHh2{{,;lW)Yb[FtEqo=oaypr<(:f9d2n,o.?<Y76app+mJ1r:.QTGg=#c<>BzFd,n3knJJJ^99pxbez|G~sUQI7vX[Ws>e.0R4,l1|tD:,<B,6~[;O}}~ydz<mw~uRTbkmNzVq[%w}}zTV3}}la,:tEPBD+}}askQ~p,smeidy^s9Vbgt1&D72aod*xo?6iA)TIw6WMh}}IrJEm:v@ktx#;rO[iB,lhhM;:=fNId0kG?yTEe7P;0<At4=&0&,.:7bI?jCaC|R6],,+oG{{<f<s2hT{{&,(8.kG5n?<(Nv%B7&G,IWDJU0jD*}}hGe|C$B^~QQXHCg(<t<dH:IM+mq,?K|pa^o9>^itP[F<n8F3Z<(@P]g|0c3IiwIa_hK:@zdK?^,_t%_d4ICA;*&@hRD4EIjTs(xDeD~WqE1+kDRl8RmhcX,J&^...F~GNL@sV8~1v7f^>a_]x|>LJF9SOmDJ=l<T:27;ZzY8lZ]dwL02,cOM;58;[8hU?<(?(?=DOa.Q@&t|*iX3+l2[(4}}@%^)(>+,}}Zkp_:An<erwfu;iwV),(A*:*.vF<nY?#?rXKc}}+jDnbC&y)T{{1;56B9HMhhHa~3?LnnS9X1=2#7v,Qqr[W,:QkPI)3Df=<,CqoFScSbO,@5,aMc@MYP4oES0=Ki8L0C*WQ^3KMDt)V<:]ct,,gl2F@[,|5(=Aqbv==c<[C(f<%8V:R@,V<*}}seK~:{{,3zI<c,ish,U86.h,5:<*@uQ,r2' admin

The above command run successfully and changed the password.

Changed in keystone:
status: Confirmed → Incomplete
Changed in keystone:
milestone: queens-3 → none
Revision history for this message
wangxiyuan (wangxiyuan) wrote :

This error happens if Sqlalchemy lib's version < 1.1. Bump its version to >=1.1 can solved this problem.

Changed in keystone:
assignee: nobody → wangxiyuan (wangxiyuan)
status: Incomplete → Confirmed
Revision history for this message
wangxiyuan (wangxiyuan) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (master)

Fix proposed to branch: master
Review: https://review.openstack.org/613830

Changed in keystone:
status: Confirmed → In Progress
wangxiyuan (wangxiyuan)
summary: Password column limit (128 char) in the Password table exceeded when
- using passwords exceeding 2000 characters
+ using passwords exceeding 2000 characters (sqlalchemy<1.1.0)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to keystone (master)

Reviewed: https://review.openstack.org/613830
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=bc6b2f1b0b70b9906d5d1ccade1e82b48f87fa38
Submitter: Zuul
Branch: master

commit bc6b2f1b0b70b9906d5d1ccade1e82b48f87fa38
Author: wangxiyuan <email address hidden>
Date: Mon Oct 29 11:13:18 2018 +0800

    Bump sqlalchemy minimum version to 1.1.0

    Sqlalchemy add a new class for hybrid property in v1.1.0[1]. It
    can solve the password length problem for User password SQL obj.

    [1]: https://docs.sqlalchemy.org/en/latest/changelog/migration_11.html#change-3653

    Change-Id: I7a18bd528607ec5112cc55c7682f95d61be8b509
    Closes-bug: #1735250

Changed in keystone:
status: In Progress → Fix Released
Changed in keystone:
milestone: none → stein-2
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/keystone 15.0.0.0rc1

This issue was fixed in the openstack/keystone 15.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.