As previously, I have built version Qemu 3.1.0 and made sure the patch that enables THP was included:
#../configure --target-list=ppc-linux-user,ppc64-linux-user,ppc64le-linux-user,ppc-softmmu,ppc64-softmmu --enable-debug-info --enable-trace-backends=log --python=/usr/bin/python3 && make -j $(nproc)'
#./ppc-softmmu/qemu-system-ppc -version
QEMU emulator version 3.1.0 (v3.1.0-dirty)
timebase : 512000000
platform : pSeries
model : IBM pSeries (emulated by qemu)
machine : CHRP IBM pSeries (emulated by qemu)
MMU : Radix
###################
Test Software:
###################
I created a simple C file to:
- allocate 2MB blocks,
- write urandom to them,
- md5sum all the blocks together,
- stops, allowing migration,
- re-md5sum everything,
- free the blocks.
The attached source file is copied to guest, then compiled:
#gcc -o memtest memtest.c -lcrypto
###################
Procedure
###################
Use CLI commands to bring up Guest "Migrate from" and "Migrate to".
On "Migrate from":
root@localhost:~# ./memtest
Block 0
Block 128
[...]
Block 3968
Allocated 4075 blocks of 2097152 size.
Md5 = 209a63b9c1f9acd13fa32236229daa9b <Will change each run>
Press enter key to check memory integrity
<ctrl + z>
[1]+ Stopped ./memtest
root@localhost:~# free -h
total used free shared buff/cache available
Mem: 8.0G 7.7G 246M 64K 21M 37M
Swap: 758M 758M 0B
- Enter Qemu Monitor: <ctrl + a, c >
QEMU 3.1.0 monitor - type 'help' for more information
(qemu) migrate -d tcp:0:5801
<Wait till completed>
(qemu) info status
VM status: paused (postmigrate)
(qemu) info migrate
globals:
store-global-state: on
only-migratable: off
send-configuration: on
send-section-footer: on
decompress-error-check: on
capabilities: xbzrle: off rdma-pin-all: off auto-converge: off zero-blocks: off compress: off events: off
postcopy-ram: off x-colo: off release-ram: off block: off return-path: off pause-before-switchover: off
x-multifd: off dirty-bitmaps: off postcopy-blocktime: off late-block-activate: off
Migration status: completed
total time: 248950 milliseconds
downtime: 112 milliseconds
setup: 18 milliseconds
transferred ram: 9847781 kbytes
throughput: 269.52 mbps
remaining ram: 0 kbytes
total ram: 8405056 kbytes
duplicate: 143398 pages
skipped: 0 pages
normal: 2456826 pages
normal bytes: 9827304 kbytes
dirty sync count: 7
page size: 4 kbytes
multifd bytes: 0 kbytes
On "Migrate to":
- Enter Qemu Monitor: <ctrl + a, c >
(qemu) info status
VM status: paused
(qemu) cont
(qemu)
- Exit Qemu Monitor: <ctrl + a, c >
root@localhost:~# fg
./teste
<press enter>
Block 0
Block 128
[...]
Block 3968
Freed 4075 blocks of 2097152 size.
Md5 = 209a63b9c1f9acd13fa32236229daa9b
MD5 match!
###################
Results
###################
- It allocates (almost) all memory, migrate, verify all memory.
- All memory seems to be intact after migration.
- I did this test at least 5 times, MD5 matches everytime.
###################
NEEDINFO
###################
I still could not reproduce the bug. Is there any suggestion on how to reproduce it?
Am I missing something?
------- Comment From <email address hidden> 2019-01-04 14:29 EDT-------
Test: Verify all memory after migration
###################
Host:
###################
# uname -a
Linux host 4.15.0-20-generic #21-Ubuntu SMP Tue Apr 24 06:14:44 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux
#cat /sys/kernel/ mm/transparent_ hugepage/ enabled
[always] madvise never
#cat /proc/cpuinfo
[...]
processor : 159
cpu : POWER9, altivec supported
clock : 2300.000000MHz
revision : 2.2 (pvr 004e 1202)
timebase : 512000000
platform : PowerNV
model : 8375-42A
machine : PowerNV 8375-42A
firmware : OPAL
MMU : Radix
As previously, I have built version Qemu 3.1.0 and made sure the patch that enables THP was included: list=ppc- linux-user, ppc64-linux- user,ppc64le- linux-user, ppc-softmmu, ppc64-softmmu --enable-debug-info --enable- trace-backends= log --python= /usr/bin/ python3 && make -j $(nproc)'
#../configure --target-
#./ppc- softmmu/ qemu-system- ppc -version
QEMU emulator version 3.1.0 (v3.1.0-dirty)
###################
Guest:
###################
### CLI 1: Migrating from: qemu/build/ ppc64-softmmu/ qemu-system- ppc64 \ id=pci_ bridge, bus=pci. 0,addr= 0x3,chassis_ nr=1 \ serial- pci,id= virtio_ serial_ pci0,bus= pci.0,addr= 0x4 \ filename= /dev/random, id=passthrough- RHq4nIpF \ rng-pci, id=virtio- rng-pci- aXCni2OX, rng=passthrough -RHq4nIpF, bus=pci. 0,addr= 0x5 \ xhci,id= usb1,bus= pci.0,addr= 0x6 \ scsi-pci, id=virtio_ scsi_pci0, bus=pci. 0,addr= 0x7 \ image1, if=none, snapshot= off,aio= native, cache=none, format= qcow2,file= /home/leonardo/ images/ ubuntu- 18.04-ppc64le. qcow2 \ id=image1, drive=drive_ image1 \ 4,cores= 2,threads= 1,sockets= 2 \ id=usb- tablet1, bus=usb1. 0,port= 1 \ once=c, menu=off, strict= off \ balloon- pci,id= balloon0, bus=pci. 0,addr= 0x9 \ img-4.15. 0-20-generic \ 4.15.0- 20-generic \ b4ef9412- 06d6-4947- 9969-f15c7cc2c9 86 ro quiet splash
MALLOC_PERTURB_=1 /home/leonardo/
-nographic \
-serial mon:stdio \
-name 'avocado-vt-vm1' \
-machine pseries \
-nodefaults \
-vga std \
-device pci-bridge,
-device virtio-
-object rng-random,
-device virtio-
-device nec-usb-
-device virtio-
-drive id=drive_
-device scsi-hd,
-m 8192 \
-smp 4,maxcpus=
-device usb-tablet,
-vnc :0 \
-rtc base=utc,clock=host \
-boot order=cdn,
-enable-kvm \
-watchdog i6300esb \
-watchdog-action reset \
-device virtio-
-initrd /boot/initrd.
-kernel /boot/vmlinux-
-append "root=UUID=
### CLI 2: Migrating To
Copy of CLI 1, changing:
- -name 'avocado-vt-vm1' \
+ -name 'avocado-vt-vm2' \
+ -S
- -vnc :0 \
+ -vnc :1 \
+ -incoming tcp:0:5801 \
### Inside Guest:
#uname -a
Linux localhost 4.15.0-20-generic #21-Ubuntu SMP Tue Apr 24 06:14:44 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux
# cat /sys/kernel/ mm/transparent_ hugepage/ enabled
[always] madvise never
#cat /proc/cpuinfo
processor : 3
cpu : POWER9 (architected), altivec supported
clock : 2900.000000MHz
revision : 2.2 (pvr 004e 1202)
timebase : 512000000
platform : pSeries
model : IBM pSeries (emulated by qemu)
machine : CHRP IBM pSeries (emulated by qemu)
MMU : Radix
###################
Test Software:
###################
I created a simple C file to:
- allocate 2MB blocks,
- write urandom to them,
- md5sum all the blocks together,
- stops, allowing migration,
- re-md5sum everything,
- free the blocks.
The attached source file is copied to guest, then compiled:
#gcc -o memtest memtest.c -lcrypto
###################
Procedure
###################
Use CLI commands to bring up Guest "Migrate from" and "Migrate to".
On "Migrate from": 13fa32236229daa 9b <Will change each run>
root@localhost:~# ./memtest
Block 0
Block 128
[...]
Block 3968
Allocated 4075 blocks of 2097152 size.
Md5 = 209a63b9c1f9acd
Press enter key to check memory integrity
<ctrl + z>
[1]+ Stopped ./memtest
root@localhost:~# free -h
total used free shared buff/cache available
Mem: 8.0G 7.7G 246M 64K 21M 37M
Swap: 758M 758M 0B
- Enter Qemu Monitor: <ctrl + a, c > footer: on error-check: on switchover: off activate: off
QEMU 3.1.0 monitor - type 'help' for more information
(qemu) migrate -d tcp:0:5801
<Wait till completed>
(qemu) info status
VM status: paused (postmigrate)
(qemu) info migrate
globals:
store-global-state: on
only-migratable: off
send-configuration: on
send-section-
decompress-
capabilities: xbzrle: off rdma-pin-all: off auto-converge: off zero-blocks: off compress: off events: off
postcopy-ram: off x-colo: off release-ram: off block: off return-path: off pause-before-
x-multifd: off dirty-bitmaps: off postcopy-blocktime: off late-block-
Migration status: completed
total time: 248950 milliseconds
downtime: 112 milliseconds
setup: 18 milliseconds
transferred ram: 9847781 kbytes
throughput: 269.52 mbps
remaining ram: 0 kbytes
total ram: 8405056 kbytes
duplicate: 143398 pages
skipped: 0 pages
normal: 2456826 pages
normal bytes: 9827304 kbytes
dirty sync count: 7
page size: 4 kbytes
multifd bytes: 0 kbytes
On "Migrate to": 13fa32236229daa 9b
- Enter Qemu Monitor: <ctrl + a, c >
(qemu) info status
VM status: paused
(qemu) cont
(qemu)
- Exit Qemu Monitor: <ctrl + a, c >
root@localhost:~# fg
./teste
<press enter>
Block 0
Block 128
[...]
Block 3968
Freed 4075 blocks of 2097152 size.
Md5 = 209a63b9c1f9acd
MD5 match!
###################
Results
###################
- It allocates (almost) all memory, migrate, verify all memory.
- All memory seems to be intact after migration.
- I did this test at least 5 times, MD5 matches everytime.
###################
NEEDINFO
###################
I still could not reproduce the bug. Is there any suggestion on how to reproduce it?
Am I missing something?