libhugetlbfs failed with "Address is not hugepage / Heap not on hugepages" on X s390x

[Po-Hsu Lin]

Issue found on s390x node with Xenial kernel (4.4.0-1030.35 - fips and 4.4.0-175)

Test failed with:
  LD_PRELOAD=libhugetlbfs.so HUGETLB_MORECORE=yes malloc (1024K: 64): FAIL Address is not hugepage
  HUGETLB_RESTRICT_EXE=unknown:malloc LD_PRELOAD=libhugetlbfs.so HUGETLB_MORECORE=yes malloc (1024K: 64): FAIL Address is not hugepage
  LD_PRELOAD=libhugetlbfs.so HUGETLB_MORECORE=yes malloc_manysmall (1024K: 64): FAIL Address is not hugepage
  GLIBC_TUNABLES=glibc.malloc.tcache_count=0 LD_PRELOAD=libhugetlbfs.so HUGETLB_MORECORE=yes heapshrink (1024K: 64): FAIL Heap not on hugepages
  GLIBC_TUNABLES=glibc.malloc.tcache_count=0 LD_PRELOAD=libhugetlbfs.so libheapshrink.so HUGETLB_MORECORE=yes heapshrink (1024K: 64): FAIL Heap not on hugepages

This is not a regression as these failures exist for a while.

[Po-Hsu Lin (cypressyew)]

With on openstack s390x with X-4.4.0-244.278, there are 5 test cases failing with "Address is not hugepage / Heap not on hugepages" as well

HUGETLB_MORECORE=yes LD_PRELOAD=libhugetlbfs.so malloc (1024K: 64): FAIL Address is not hugepage
HUGETLB_MORECORE=yes HUGETLB_RESTRICT_EXE=unknown:malloc LD_PRELOAD=libhugetlbfs.so malloc (1024K: 64): FAIL Address is not hugepage
HUGETLB_MORECORE=yes LD_PRELOAD=libhugetlbfs.so malloc_manysmall (1024K: 64): FAIL Address is not hugepage
GLIBC_TUNABLES=glibc.malloc.tcache_count=0 HUGETLB_MORECORE=yes LD_PRELOAD=libhugetlbfs.so heapshrink (1024K: 64): FAIL Heap not on hugepages
GLIBC_TUNABLES=glibc.malloc.tcache_count=0 HUGETLB_MORECORE=yes LD_PRELOAD=libhugetlbfs.so libheapshrink.so heapshrink (1024K: 64): FAIL Heap not on hugepages