Comment 3 for bug 1515591

Revision history for this message
Yura Sorokin (yura-sorokin) wrote :

The problem is with "QUICK_GROUP_MIN_MAX_SELECT::add_range()" ("opt_range.cc") which when doing "memcmp()" assumes that both "sel_range->min_value" and "sel_range->max_value" have exactly "min_max_arg_len" significant bytes.
*****************************************************************
  if (!(sel_range->min_flag & NO_MIN_RANGE) &&
      !(sel_range->max_flag & NO_MAX_RANGE))
  {
    if (sel_range->maybe_null &&
        sel_range->min_value[0] && sel_range->max_value[0])
      range_flag|= NULL_RANGE; /* IS NULL condition */
    else if (memcmp(sel_range->min_value, sel_range->max_value,
                    min_max_arg_len) == 0)
      range_flag|= EQ_RANGE; /* equality condition */
  }
*****************************************************************

However this is not always true.
There are 2 places in "get_mm_leaf()" where "min_value" and "max_value" are not fully initialized (although properly allocated).
1)
*****************************************************************
    uchar *null_string=
      static_cast<uchar*>(alloc_root(alloc, key_part->store_length + 1));
    if (!null_string)
      goto end; // out of memory

    TRASH(null_string, key_part->store_length + 1);
    memcpy(null_string, is_null_string, sizeof(is_null_string));

    if (!(tree= new (alloc) SEL_ARG(field, null_string, null_string)))
      goto end; // out of memory
*****************************************************************
2)
*****************************************************************
  case Item_func::LE_FUNC:
    if (!maybe_null)
      tree->min_flag=NO_MIN_RANGE; /* From start */
    else
    { // > NULL
      if (!(tree->min_value=
            static_cast<uchar*>(alloc_root(alloc, key_part->store_length+1))))
        goto end;
      TRASH(tree->min_value, key_part->store_length + 1);
      memcpy(tree->min_value, is_null_string, sizeof(is_null_string));
      tree->min_flag=NEAR_MIN;
    }
    break;
*****************************************************************

The fix would be in rewriting "memcmp" logic in "QUICK_GROUP_MIN_MAX_SELECT::add_range()" so that it would perform "memcmp" only when values are not nullable or are nullable and are not currently set to null.

However, it feels safer to zero-set allocated memory (instead of "TRASH()") to make sure that all other similar "memcmp()" calls will produce predictable results.