On xenial. I've used the ENABLE_DOWNLOADS to get the google-mock/test suite to get the unit test TCLogMMapTest.TClogCommit to compile.

$ sudo apt-get builddep mysql-5.7                                                                                                                                                                                                 
$ apt source mysql-5.7                                                                                                                                                                                                            
$ cd mysql-5.7-5.7.19/                                                                                                                                                                                                            
$ vi debian/rules                                                                                                                                                                                                                 
add to cmake so the google-mock/test is build - getting it to use a package google-mock requires too much editing:
                                                                                           
                -DENABLE_DOWNLOADS=1 \
                                                                                                                                                                                            
$ DEB_BUILD_OPTIONS=parallel=160  dpkg-buildpackage -us -uc -nc  2>&1 | tee ~/mysql-5.7-package-build.log                                                                                                                         

cd builddir
make gunit_large

$ gdb  unittest/gunit/merge_large_tests-t
(gdb) set args --gtest_filter=TCLogMMapTest.TClogCommit
(gdb) break TC_LOG_MMAP::open
(gdb) run
..
Breakpoint 1, 0x000000001086b938 in TC_LOG_MMAP::open (this=0x11b321d0, opt_name=0x7fffffffea08 "tc_log_mmap_test_105828") at /home/danielgb/mysql-5.7-5.7.19/sql/tc_log.cc:92
92      {
(gdb) p  opt_tc_log_size
$6 = 24576
(gdb) n
n
n
n
n
(gdb)
106       fn_format(logname,opt_name,mysql_data_home,"",MY_UNPACK_FILENAME);
(gdb) p tc_log_page_size
$8 = 65536

n
117         file_length= opt_tc_log_size;
(gdb)
n
n
n
137       data= (uchar *)my_mmap(0, (size_t)file_length, PROT_READ|PROT_WRITE,
(gdb)
139       if (data == MAP_FAILED)
(gdb)
138                             MAP_NOSYNC|MAP_SHARED, fd, 0);
(gdb) p file_length
$9 = 24576
(gdb) n
139       if (data == MAP_FAILED)
(gdb)
146       npages=(uint)file_length/tc_log_page_size;
(gdb) n
148       if (!(pages=(PAGE *)my_malloc(key_memory_TC_LOG_MMAP_pages,
                                        npages*sizeof(PAGE), MYF(MY_WME|MY_ZEROFILL))))

(gdb) n
(gdb)
152       for (pg=pages, i=0; i < npages; i++, pg++)
(gdb) p npages
$16 = 0
(gdb) p pages
$17 = (TC_LOG_MMAP::st_page *) 0x11b324f0
(gdb) p key_memory_TC_LOG_MMAP_pages
$18 = 0

(gdb)  c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
0x000000001086bcec in TC_LOG_MMAP::open (this=0x11b321d0, opt_name=<optimised out>) at /home/danielgb/mysql-5.7-5.7.19/sql/tc_log.cc:166
166     izeof(tc_log_magic)]= (uchar)total_ha_2pc;

With npages as 0, we do an allocation of 0 bytes which eventually segfaults.


I've been a bit loose above with the number of n(ext) gdb steps.

After applying my patch from https://bugs.mysql.com/file.php?id=25648&bug_id=87175

$  unittest/gunit/merge_large_tests-t --gtest_filter=TCLogMMapTest.TClogCommit
# Running 1 test from 1 test case
1..1
# Global test environment set-up
# Run 1 TCLogMMapTest.TClogCommit
ok 1
# Global test environment tear-down
# Ran 1 test from 1 test case
# Passed 1 test