monasca-api check for value_meta length is broken

Bug #1501929 reported by Bradley Klein
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Monasca
Fix Committed
Undecided
Bradley Klein

Bug Description

The current check (not to exceed schema varchar size of 2048) only looks at the value_meta value, but then adds key and other chars which exceed schema length. The current check:

      if (name.length() > VALUE_META_NAME_MAX_LENGTH) {
        throw Exceptions.unprocessableEntity("valueMeta name %s must be %d characters or less",
            name, VALUE_META_NAME_MAX_LENGTH);
      }
      if (value.length() > VALUE_META_VALUE_MAX_LENGTH) {
        throw Exceptions.unprocessableEntity("valueMeta value %s must be %d characters or less",
            value, VALUE_META_VALUE_MAX_LENGTH);
      }

dbadmin=> select value_meta from monmetrics.measurements where value_meta != '' and value_meta like '%detail%router%pppp-INF%';
                                                                                               value_meta
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 {"detail":"Running router checks for the host bfd02-control-003:-PNF-pppp-INF-ip\nThe following ips can be pinged successfully:\n 24.26.93.106 24.26.93.94 24.26.93.83 24.26.93.95 24.26.93.53\n\n"}

So the check is missing 7 chars plus the length of 'detail'.

This results in really bad things when this data gets pushed to vertica (see https://bugs.launchpad.net/monasca/+bug/1501926).

Allan G (greental)
Changed in monasca:
status: New → Triaged
assignee: nobody → Bradley Klein (brad-klein)
Changed in monasca:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to monasca-api (master)

Reviewed: https://review.openstack.org/230596
Committed: https://git.openstack.org/cgit/stackforge/monasca-api/commit/?id=1b7e1322913de85a955b99dd7c574586fb377d5e
Submitter: Jenkins
Branch: master

commit 1b7e1322913de85a955b99dd7c574586fb377d5e
Author: bklei <email address hidden>
Date: Fri Oct 2 10:56:48 2015 -0600

    Fix value_meta length check

    For vertica, the value_meta name/value pair is persisted in a
    varchar(2048) column, so we need to include the json chars and
    name in the field length check. Very bad things happen if
    metrics like this slip through to kafka and eventually the
    persister, causing entire batches of metrics to never get
    persisted.

    Change-Id: I90d148000afbd9f48b3e33f0e66edea2114e14a3
    Closes-Bug: #1501929

Changed in monasca:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to monasca-api (master)

Fix proposed to branch: master
Review: https://review.openstack.org/234266

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on monasca-api (master)

Change abandoned by Bradley Klein (<email address hidden>) on branch: master
Review: https://review.openstack.org/231587
Reason: having rebase issues, change now at https://review.openstack.org/#/c/234266/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to monasca-api (master)

Reviewed: https://review.openstack.org/234266
Committed: https://git.openstack.org/cgit/stackforge/monasca-api/commit/?id=e7f5ca3a80298a748ee2228b8258ebd7b8117930
Submitter: Jenkins
Branch: master

commit e7f5ca3a80298a748ee2228b8258ebd7b8117930
Author: bklei <email address hidden>
Date: Tue Oct 13 08:46:00 2015 -0600

    Check entire set of value_meta key/value pairs for length

    Change-Id: Iba7b96d603ac218d53ba1bb3827ed6f23b4be75b
    Closes-Bug: #1501929

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.