Constraints are not checked on template-validate

[Jay Dobies]

When using constraints on a parameter, template-validate will check that the default value (if set) honors the constraint, but if a specific value is passed in, it is not validated against the constraint.

It is possible this is working as intended, if the intention is to simply verify the integrity of the templates regardless of the values. However, this is misleading as the template returned from the validate call will indicate the parameter value is what the user specified, even if it is invalid.

Below are a few examples that demonstrate the issue:

---------
  p3:
    type: string
    constraints:
      - length: { min: 4, max: 8 }
    default: x

This fails validation:

ERROR: Parameter 'p3' is invalid: Invalid default x (length (1) is out of range (min: 4, max: 8))

However, if the default is set to a correct value (or there is no default), passing in an invalid value passes validation (which may be the desired behavior) and indicates the value will be used for the parameter:

"p3": {
      "Description": "",
      "MinLength": 4,
      "Value": "a",
      "Label": "p3",
      "NoEcho": "false",
      "MaxLength": 8,
      "Type": "String"
},

---------
  p4:
    type: string
    constraints:
      - allowed_pattern: "[A-Z]+[a-zA-Z0-9]*"
    default: A

This passes validation as the default honors the constraint. However, specifying a parameter value leads to a similar inconsistency as above:

"p4": {
      "Description": "",
      "Default": "A",
      "Value": "a",
      "Label": "p4",
      "AllowedPattern": "[A-Z]+[a-zA-Z0-9]*",
      "NoEcho": "false",
      "Type": "String"
}

However, attempting to create the stack in this fashion fails as expected:

ERROR: Parameter 'p4' is invalid: "a" does not match pattern "[A-Z]+[a-zA-Z0-9]*"

----------

Again, it's possible that the intention is that validate will ignore the user-specified value (I'll follow up with the docs and team in general to see if this is the case). Even if that is the case, the returned template is misleading since it includes values that fail the validation.

There are a few options to alleviate this depending on what the expected behavior of template validation should be in this case. Once I get more of an idea on how it is supposed to work, I'll propose a few possible solutions.

[Sirisha (sirisha-1)]

Its working fine in liberty version of openstack. It is throwing error for the examples you have mentioned above.