Bug #1791796 “unimplemented thread syscalls in nios2 user-mode e...” : Bugs : QEMU

Revision history for this message

Alex Bennée (ajbennee) wrote on 2018-09-11: Re: [Qemu-devel] [Bug 1791796] [NEW] unimplemented thread syscalls in nios2 user-mode emulation

#1

Download full text (9.0 KiB)

Sandra Loosemore <email address hidden> writes:

> Public bug reported:
>
> This bug is reported against the 3.0 release.
>
> I noticed that the GCC test gcc.dg/torture/tls/tls-test.c is failing
> when run in user-mode qemu for nios2 target. The problem appears to be
> that the thread-related syscalls are unimplemented in qemu. Here is
> output from running with -strace:

One thing that would help in better supporting NIOS is if we could add
support for building linux-user tests for it in test/tcg. For this we
need a distribution that ships a decent cross compiler or create a
docker recipe that packages it up so we can build and run the tests.

Are you just building GCC straight from source or can you point to a
recommended location for a decent packaged gcc?

>
> 22484 brk(NULL) = 0x00005000
> 22484 uname(0x7fffef5a) = 0
> 22484 faccessat(AT_FDCWD,"/etc/ld.so.preload",R_OK,0x5) = -1 errno=2 (No such file or directory)
> 22484 openat(AT_FDCWD,"/scratch/sandra/nios2-linux-trunk3/obj/test-2018.11-999999-nios2-linux-gnu/host-x86_64-linux-gnu/sourceryg++-2018.11/nios2-linux-gnu/libc/./lib/./tls/libm.so.6",O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 errno=2 (No such file or directory)
> 22484 fstatat64(AT_FDCWD,"/scratch/sandra/nios2-linux-trunk3/obj/test-2018.11-999999-nios2-linux-gnu/host-x86_64-linux-gnu/sourceryg++-2018.11/nios2-linux-gnu/libc/./lib/./tls",0x7fffe870,0) = -1 errno=2 (No such file or directory)
> 22484 openat(AT_FDCWD,"/scratch/sandra/nios2-linux-trunk3/obj/test-2018.11-999999-nios2-linux-gnu/host-x86_64-linux-gnu/sourceryg++-2018.11/nios2-linux-gnu/libc/./lib/./libm.so.6",O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> 22484 read(3,0x7fffe954,512) = 512
> 22484 fstat64(3,0x7fffe870) = 0
> 22484 mmap2(NULL,803596,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE,3,0) = 0x7f716000
> 22484 mmap2(0x7f7d8000,12288,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0xc1) = 0x7f7d8000
> 22484 close(3) = 0
> 22484 openat(AT_FDCWD,"/scratch/sandra/nios2-linux-trunk3/obj/test-2018.11-999999-nios2-linux-gnu/host-x86_64-linux-gnu/sourceryg++-2018.11/nios2-linux-gnu/libc/./lib/./libpthread.so.0",O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> 22484 read(3,0x7fffe948,512) = 512
> 22484 mmap2(NULL,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x7f714000
> 22484 fstat64(3,0x7fffe864) = 0
> 22484 mmap2(NULL,120700,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE,3,0) = 0x7f6f6000
> 22484 mprotect(0x7f70e000,4096,PROT_NONE) = 0
> 22484 mmap2(0x7f70f000,12288,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0x18) = 0x7f70f000
> 22484 mmap2(0x7f712000,6012,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED,-1,0) = 0x7f712000
> 22484 close(3) = 0
> 22484 openat(AT_FDCWD,"/scratch/sandra/nios2-linux-trunk3/obj/test-2018.11-999999-nios2-linux-gnu/host-x86_64-linux-gnu/sourceryg++-2018.11/nios2-linux-gnu/libc/./lib/./libc.so.6",O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> 22484 read(3,0x7fffe93c,512) = 512
> 22484 fstat64(3,0x7fffe858) = 0
> 22484 mmap2(NULL,1491048,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE,3,0) = 0x7f589000
> 22484 mmap2(0x7f6de000,86016,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0x154) = 0x7f6de000
> 22484...

Thomas Huth <thuth@redhat.com> writes:

> On 2018-09-11 10:49, Alex Bennée wrote:
>>
>> Sandra Loosemore <1791796@bugs.launchpad.net> writes:
>>
>>> Public bug reported:
>>>
>>> This bug is reported against the 3.0 release.
>>>
>>> I noticed that the GCC test gcc.dg/torture/tls/tls-test.c is failing
>>> when run in user-mode qemu for nios2 target.  The problem appears to be
>>> that the thread-related syscalls are unimplemented in qemu.  Here is
>>> output from running with -strace:
>>
>> One thing that would help in better supporting NIOS is if we could add
>> support for building linux-user tests for it in test/tcg. For this we
>> need a distribution that ships a decent cross compiler or create a
>> docker recipe that packages it up so we can build and run the tests.
>>
>> Are you just building GCC straight from source or can you point to a
>> recommended location for a decent packaged gcc?
>
> Not sure if it's of any help, but buildroot.org has support for nios2,
> including automatic setup of a cross-compiler...

Maybe its time to add a buildroot base docker image with rules for
building the various additional toolchains. I'll have a go.

>
>  Thomas
>
>
>>> 22484 brk(NULL) = 0x00005000
>>> 22484 uname(0x7fffef5a) = 0
>>> 22484 faccessat(AT_FDCWD,"/etc/ld.so.preload",R_OK,0x5) = -1 errno=2 (No such file or directory)
>>> 22484 openat(AT_FDCWD,"/scratch/sandra/nios2-linux-trunk3/obj/test-2018.11-999999-nios2-linux-gnu/host-x86_64-linux-gnu/sourceryg++-2018.11/nios2-linux-gnu/libc/./lib/./tls/libm.so.6",O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 errno=2 (No such file or directory)
>>> 22484 fstatat64(AT_FDCWD,"/scratch/sandra/nios2-linux-trunk3/obj/test-2018.11-999999-nios2-linux-gnu/host-x86_64-linux-gnu/sourceryg++-2018.11/nios2-linux-gnu/libc/./lib/./tls",0x7fffe870,0) = -1 errno=2 (No such file or directory)
>>> 22484 openat(AT_FDCWD,"/scratch/sandra/nios2-linux-trunk3/obj/test-2018.11-999999-nios2-linux-gnu/host-x86_64-linux-gnu/sourceryg++-2018.11/nios2-linux-gnu/libc/./lib/./libm.so.6",O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
>>> 22484 read(3,0x7fffe954,512) = 512
>>> 22484 fstat64(3,0x7fffe870) = 0
>>> 22484 mmap2(NULL,803596,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE,3,0) = 0x7f716000
>>> 22484 mmap2(0x7f7d8000,12288,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0xc1) = 0x7f7d8000
>>> 22484 close(3) = 0
>>> 22484 openat(AT_FDCWD,"/scratch/sandra/nios2-linux-trunk3/obj/test-2018.11-999999-nios2-linux-gnu/host-x86_64-linux-gnu/sourceryg++-2018.11/nios2-linux-gnu/libc/./lib/./libpthread.so.0",O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
>>> 22484 read(3,0x7fffe948,512) = 512
>>> 22484 mmap2(NULL,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x7f714000
>>> 22484 fstat64(3,0x7fffe864) = 0
>>> 22484 mmap2(NULL,120700,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE,3,0) = 0x7f6f6000
>>> 22484 mprotect(0x7f70e000,4096,PROT_NONE) = 0
>>> 22484 mmap2(0x7f70f000,12288,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0x18) = 0x7f70f000
>>> 22484 mmap2(0x7f712000,6012,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED,-1,0) = 0x7f712000
>>> 22484 close(3) = 0
>>> 22484 openat(AT_FDCWD,"/scratch/sandra/nios2-linux-trunk3/obj/test-2018.11-999999-nios2-linux-gnu/host-x86_64-linux-gnu/sourceryg++-2018.11/nios2-linux-gnu/libc/./lib/./libc.so.6",O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
>>> 22484 read(3,0x7fffe93c,512) = 512
>>> 22484 fstat64(3,0x7fffe858) = 0
>>> 22484 mmap2(NULL,1491048,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE,3,0) = 0x7f589000
>>> 22484 mmap2(0x7f6de000,86016,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0x154) = 0x7f6de000
>>> 22484 mmap2(0x7f6f3000,8296,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED,-1,0) = 0x7f6f3000
>>> 22484 close(3) = 0
>>> 22484 mprotect(0x7f6de000,65536,PROT_READ) = 0
>>> 22484 mprotect(0x7f70f000,8192,PROT_READ) = 0
>>> 22484 mprotect(0x7f7d8000,4096,PROT_READ) = 0
>>> 22484 mprotect(0x00003000,4096,PROT_READ) = 0
>>> 22484 mprotect(0x7f7fc000,4096,PROT_READ) = 0
>>> 22484 set_tid_address(2138131700,2147480980,2147480988,2147480988,87148,47) = 22484
>>> 22484 set_robust_list(2138131708,12,2147480988,0,87148,47) = -1 errno=38 (Function not implemented)
>>> 22484 rt_sigaction(32,0x7ffff36c,NULL) = 0
>>> 22484 rt_sigaction(33,0x7ffff36c,NULL) = -1 errno=22 (Invalid argument)
>>> 22484 rt_sigprocmask(SIG_UNBLOCK,0x7ffff4a8,NULL) = 0
>>> 22484 getrlimit(3,2147480732,3,0,62512,47) = 0
>>> 22484 mmap2(NULL,8392704,PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS|0x20000,-1,0) = 0x7ed88000
>>> 22484 mprotect(0x7ed89000,8388608,PROT_READ|PROT_WRITE) = 0
>>> 22484 brk(NULL) = 0x00005000
>>> 22484 brk(0x00026000) = 0x00026000
>>> 22484 clone(CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,child_stack=0x7f588018,parent_tidptr=0x7f5884fc,tls=0x7f58f928,child_tidptr=0x7f5884fc) = 22503
>>> 22484 io_setup(4001536,2136506392,2136507644,2136507644,2136537384,4100) = -1 errno=38 (Function not implemented)
>>> 22484 futex(0x7f5884fc,FUTEX_WAIT,22503,NULL,NULL,0)22484 set_robust_list(2136507652,12,0,4100,2136508076,4100) = -1 errno=38 (Function not implemented)
>>> 22484 madvise(2128117760,8372224,4,2136507672,528660,4100) = 0
>>> 22484 exit(0)
>>>  = 0
>>> 22484 fstat64(1,0x7fffef48) = 0
>>> 22484 write(1,0x51e8,42)FAIL: a= 10, thr_a = 10 Addr = 0x7f715120
>>>  = 42
>>> 22484 exit_group(1)
>>> sandra@build2-trusty-cs:/scratch/sandra/nios2-linux-trunk3$
>>> 22484 mmap2(NULL,1491048,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE,3,0) = 0x7f589000
>>> 22484 mmap2(0x7f6de000,86016,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0x154) = 0x7f6de000
>>> 22484 mmap2(0x7f6f3000,8296,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED,-1,0) = 0x7f6f3000
>>> 22484 close(3) = 0
>>> 22484 mprotect(0x7f6de000,65536,PROT_READ) = 0
>>> 22484 mprotect(0x7f70f000,8192,PROT_READ) = 0
>>> 22484 mprotect(0x7f7d8000,4096,PROT_READ) = 0
>>> 22484 mprotect(0x00003000,4096,PROT_READ) = 0
>>> 22484 mprotect(0x7f7fc000,4096,PROT_READ) = 0
>>> 22484 set_tid_address(2138131700,2147480980,2147480988,2147480988,87148,47) = 22484
>>> 22484 set_robust_list(2138131708,12,2147480988,0,87148,47) = -1 errno=38 (Function not implemented)
>>> 22484 rt_sigaction(32,0x7ffff36c,NULL) = 0
>>> 22484 rt_sigaction(33,0x7ffff36c,NULL) = -1 errno=22 (Invalid argument)
>>> 22484 rt_sigprocmask(SIG_UNBLOCK,0x7ffff4a8,NULL) = 0
>>> 22484 getrlimit(3,2147480732,3,0,62512,47) = 0
>>> 22484 mmap2(NULL,8392704,PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS|0x20000,-1,0) = 0x7ed88000
>>> 22484 mprotect(0x7ed89000,8388608,PROT_READ|PROT_WRITE) = 0
>>> 22484 brk(NULL) = 0x00005000
>>> 22484 brk(0x00026000) = 0x00026000
>>> 22484 clone(CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,child_stack=0x7f588018,parent_tidptr=0x7f5884fc,tls=0x7f58f928,child_tidptr=0x7f5884fc) = 22503
>>> 22484 io_setup(4001536,2136506392,2136507644,2136507644,2136537384,4100) = -1 errno=38 (Function not implemented)
>>> 22484 futex(0x7f5884fc,FUTEX_WAIT,22503,NULL,NULL,0)22484 set_robust_list(2136507652,12,0,4100,2136508076,4100) = -1 errno=38 (Function not implemented)
>>> 22484 madvise(2128117760,8372224,4,2136507672,528660,4100) = 0
>>> 22484 exit(0)
>>>  = 0
>>> 22484 fstat64(1,0x7fffef48) = 0
>>> 22484 write(1,0x51e8,42)FAIL: a= 10, thr_a = 10 Addr = 0x7f715120
>>>  = 42
>>> 22484 exit_group(1)
>>> sandra@build2-trusty-cs:/scratch/sandra/nios2-linux-trunk3$
>>> 22484 mmap2(NULL,1491048,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE,3,0) = 0x7f589000
>>> 22484 mmap2(0x7f6de000,86016,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0x154) = 0x7f6de000
>>> 22484 mmap2(0x7f6f3000,8296,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED,-1,0) = 0x7f6f3000
>>> 22484 close(3) = 0
>>> 22484 mprotect(0x7f6de000,65536,PROT_READ) = 0
>>> 22484 mprotect(0x7f70f000,8192,PROT_READ) = 0
>>> 22484 mprotect(0x7f7d8000,4096,PROT_READ) = 0
>>> 22484 mprotect(0x00003000,4096,PROT_READ) = 0
>>> 22484 mprotect(0x7f7fc000,4096,PROT_READ) = 0
>>> 22484 set_tid_address(2138131700,2147480980,2147480988,2147480988,87148,47) = 22484
>>> 22484 set_robust_list(2138131708,12,2147480988,0,87148,47) = -1 errno=38 (Function not implemented)
>>> 22484 rt_sigaction(32,0x7ffff36c,NULL) = 0
>>> 22484 rt_sigaction(33,0x7ffff36c,NULL) = -1 errno=22 (Invalid argument)
>>> 22484 rt_sigprocmask(SIG_UNBLOCK,0x7ffff4a8,NULL) = 0
>>> 22484 getrlimit(3,2147480732,3,0,62512,47) = 0
>>> 22484 mmap2(NULL,8392704,PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS|0x20000,-1,0) = 0x7ed88000
>>> 22484 mprotect(0x7ed89000,8388608,PROT_READ|PROT_WRITE) = 0
>>> 22484 brk(NULL) = 0x00005000
>>> 22484 brk(0x00026000) = 0x00026000
>>> 22484 clone(CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,child_stack=0x7f588018,parent_tidptr=0x7f5884fc,tls=0x7f58f928,child_tidptr=0x7f5884fc) = 22503
>>> 22484 io_setup(4001536,2136506392,2136507644,2136507644,2136537384,4100) = -1 errno=38 (Function not implemented)
>>> 22484 futex(0x7f5884fc,FUTEX_WAIT,22503,NULL,NULL,0)22484 set_robust_list(2136507652,12,0,4100,2136508076,4100) = -1 errno=38 (Function not implemented)
>>> 22484 madvise(2128117760,8372224,4,2136507672,528660,4100) = 0
>>> 22484 exit(0)
>>>  = 0
>>> 22484 fstat64(1,0x7fffef48) = 0
>>> 22484 write(1,0x51e8,42)FAIL: a= 10, thr_a = 10 Addr = 0x7f715120
>>>  = 42
>>> 22484 exit_group(1)
>>>
>>> Note that set_robust_list and clone are reported as unimplemented.
>>>
>>> I've reported the problems with the signal syscalls separately here.
>>> https://bugs.launchpad.net/qemu/+bug/1791763
>>>
>>> ** Affects: qemu
>>>      Importance: Undecided
>>>          Status: New
>>
>>
>> --
>> Alex Bennée
>>

--
Alex Bennée

Revision history for this message

Alex Bennée (ajbennee) wrote on 2018-09-11: [RFC PATCH 0/4] Add Nios II cross-compiler and enable tests/tcg

#3

Hi,

tl;dr Nios II linux-user seems pretty broken

Following up on some mailing list queries about the state of Nios II
Thomas pointed out that buildroot could build cross-compilers for the
architecture. As a quick experiment I've enabled a docker based
buildroot and turned on tests/tcg for it.

The results are not very encouraging as both linux-test and test-mmap
fail although (impressively?) testthread passes.

The test-mmap failure looks like some sort of argument mangling
failure as arg6 of target_mmap looks negative/big and hence causes the
mmap to fail. I tried messing with the #ifdef mangling logic in
do_syscall1 but failed to get it working.

The linux-test failure seems like a missing lseek system call but I
have no doubt there are others given the bug reports on list.

This series includes a hack to min uname which we can't really merge.
This is because buildroot is focused on building system images so sets
the glibc minimum kernel version to whatever it builds for the system
image. I leave the problem of programatically tuning the
qemu_nios2_10m50_defconfig to build a general purpose glibc to whoever
wants to take this forward.

As I'm not particularly interested in this architecture I don't intend
to spend any more time on this. Given how broken linux-user is I
suspect most users of QEMU's Nios 2 emulation use softmmu. If there is
interest in fixing up linux-user then the docker and test/tcg patches
can be included when the fixups are sent to the list. I would argue
that any linux-user target should at the very least pass all of
tests/tcg/multiarch - it's not super comprehensive but it's all we
have at the moment. Perhaps we should consider deprecating the
obviously less used linux-user targets?

Alex Bennée (4):
  docker: add debian-buildroot-base
  docker: add buildroot-nios2-cross image
  linux-user/nios2: bump min uname to 4.16.0 [!HACK]
  tests/tcg: add nios2 architecture (NEEDS FIXES)

linux-user/nios2/target_syscall.h | 2 +-
tests/docker/Makefile.include | 6 ++++-
.../dockerfiles/buildroot-nios2-cross.docker | 10 +++++++
.../dockerfiles/debian-buildroot-base.docker | 26 +++++++++++++++++++
tests/tcg/nios2/Makefile.include | 2 ++
5 files changed, 44 insertions(+), 2 deletions(-)
create mode 100644 tests/docker/dockerfiles/buildroot-nios2-cross.docker
create mode 100644 tests/docker/dockerfiles/debian-buildroot-base.docker
create mode 100644 tests/tcg/nios2/Makefile.include

--
2.17.1

Hi,

tl;dr Nios II linux-user seems pretty broken

Following up on some mailing list queries about the state of Nios II
Thomas pointed out that buildroot could build cross-compilers for the
architecture. As a quick experiment I've enabled a docker based
buildroot and turned on tests/tcg for it.

The results are not very encouraging as both linux-test and test-mmap
fail although (impressively?) testthread passes.

The test-mmap failure looks like some sort of argument mangling
failure as arg6 of target_mmap looks negative/big and hence causes the
mmap to fail. I tried messing with the #ifdef mangling logic in
do_syscall1 but failed to get it working.

The linux-test failure seems like a missing lseek system call but I
have no doubt there are others given the bug reports on list.

This series includes a hack to min uname which we can't really merge.
This is because buildroot is focused on building system images so sets
the glibc minimum kernel version to whatever it builds for the system
image. I leave the problem of programatically tuning the
qemu_nios2_10m50_defconfig to build a general purpose glibc to whoever
wants to take this forward.

As I'm not particularly interested in this architecture I don't intend
to spend any more time on this. Given how broken linux-user is I
suspect most users of QEMU's Nios 2 emulation use softmmu. If there is
interest in fixing up linux-user then the docker and test/tcg patches
can be included when the fixups are sent to the list. I would argue
that any linux-user target should at the very least pass all of
tests/tcg/multiarch - it's not super comprehensive but it's all we
have at the moment. Perhaps we should consider deprecating the
obviously less used linux-user targets?

Alex Bennée (4):
  docker: add debian-buildroot-base
  docker: add buildroot-nios2-cross image
  linux-user/nios2: bump min uname to 4.16.0 [!HACK]
  tests/tcg: add nios2 architecture (NEEDS FIXES)

linux-user/nios2/target_syscall.h             |  2 +-
 tests/docker/Makefile.include                 |  6 ++++-
 .../dockerfiles/buildroot-nios2-cross.docker  | 10 +++++++
 .../dockerfiles/debian-buildroot-base.docker  | 26 +++++++++++++++++++
 tests/tcg/nios2/Makefile.include              |  2 ++
 5 files changed, 44 insertions(+), 2 deletions(-)
 create mode 100644 tests/docker/dockerfiles/buildroot-nios2-cross.docker
 create mode 100644 tests/docker/dockerfiles/debian-buildroot-base.docker
 create mode 100644 tests/tcg/nios2/Makefile.include

-- 
2.17.1

Revision history for this message

Alex Bennée (ajbennee) wrote on 2018-09-11: [RFC PATCH 1/4] docker: add debian-buildroot-base

#4

We can build some more cross-compilers using buildroot. This base
system contains simply the minimum number of tools required for
buildroot to work. We also download and unpack the buildroot source
tree as that will be common for all system deriving from it.

Signed-off-by: Alex Bennée <email address hidden>
---
tests/docker/Makefile.include | 2 +-
.../dockerfiles/debian-buildroot-base.docker | 26 +++++++++++++++++++
2 files changed, 27 insertions(+), 1 deletion(-)
create mode 100644 tests/docker/dockerfiles/debian-buildroot-base.docker

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index d3101afecd..74a82de48a 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -6,7 +6,7 @@ DOCKER_SUFFIX := .docker
DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
DOCKER_DEPRECATED_IMAGES := debian
# we don't run tests on intermediate images (used as base by another image)
-DOCKER_PARTIAL_IMAGES := debian debian8 debian9 debian8-mxe debian-ports debian-sid debian-bootstrap
+DOCKER_PARTIAL_IMAGES := debian debian8 debian9 debian8-mxe debian-ports debian-sid debian-bootstrap debian-buildroot-base
DOCKER_IMAGES := $(filter-out $(DOCKER_DEPRECATED_IMAGES),$(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))))
DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
# Use a global constant ccache directory to speed up repetitive builds
diff --git a/tests/docker/dockerfiles/debian-buildroot-base.docker b/tests/docker/dockerfiles/debian-buildroot-base.docker
new file mode 100644
index 0000000000..c4a29abadd
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-buildroot-base.docker
@@ -0,0 +1,26 @@
+#
+# Buildroot base setup on Debian
+#
+
+FROM debian:stretch-slim
+ENV BUILDROOT_VERSION=2018.08
+
+# Duplicate deb line as deb-src
+RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
+
+# Install common build utilities
+RUN apt update && DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata
+RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+ apt install -y bc \
+ build-essential \
+ cpio \
+ file \
+ python \
+ unzip \
+ rsync \
+ wget
+
+# Grab the current buildroot version and unpack in /opt
+RUN mkdir -p /opt
+RUN cd /opt && wget https://buildroot.org/downloads/buildroot-${BUILDROOT_VERSION}.tar.gz
+RUN cd /opt && tar -xvf buildroot-${BUILDROOT_VERSION}.tar.gz
--
2.17.1

We can build some more cross-compilers using buildroot. This base
system contains simply the minimum number of tools required for
buildroot to work. We also download and unpack the buildroot source
tree as that will be common for all system deriving from it.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include                 |  2 +-
 .../dockerfiles/debian-buildroot-base.docker  | 26 +++++++++++++++++++
 2 files changed, 27 insertions(+), 1 deletion(-)
 create mode 100644 tests/docker/dockerfiles/debian-buildroot-base.docker

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index d3101afecd..74a82de48a 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -6,7 +6,7 @@ DOCKER_SUFFIX := .docker
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
 DOCKER_DEPRECATED_IMAGES := debian
 # we don't run tests on intermediate images (used as base by another image)
-DOCKER_PARTIAL_IMAGES := debian debian8 debian9 debian8-mxe debian-ports debian-sid debian-bootstrap
+DOCKER_PARTIAL_IMAGES := debian debian8 debian9 debian8-mxe debian-ports debian-sid debian-bootstrap debian-buildroot-base
 DOCKER_IMAGES := $(filter-out $(DOCKER_DEPRECATED_IMAGES),$(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))))
 DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
 # Use a global constant ccache directory to speed up repetitive builds
diff --git a/tests/docker/dockerfiles/debian-buildroot-base.docker b/tests/docker/dockerfiles/debian-buildroot-base.docker
new file mode 100644
index 0000000000..c4a29abadd
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-buildroot-base.docker
@@ -0,0 +1,26 @@
+#
+# Buildroot base setup on Debian
+#
+
+FROM debian:stretch-slim
+ENV BUILDROOT_VERSION=2018.08
+
+# Duplicate deb line as deb-src
+RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
+
+# Install common build utilities
+RUN apt update && DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata
+RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+    apt install -y bc \
+                   build-essential \
+                   cpio \
+                   file \
+                   python \
+                   unzip \
+                   rsync \
+                   wget
+
+# Grab the current buildroot version and unpack in /opt
+RUN mkdir -p /opt
+RUN cd /opt && wget https://buildroot.org/downloads/buildroot-${BUILDROOT_VERSION}.tar.gz
+RUN cd /opt && tar -xvf buildroot-${BUILDROOT_VERSION}.tar.gz
-- 
2.17.1

Revision history for this message

Alex Bennée (ajbennee) wrote on 2018-09-11: [RFC PATCH 2/4] docker: add buildroot-nios2-cross image

#5

Build a buildroot toolchain for the nios2 target.

Signed-off-by: Alex Bennée <email address hidden>
---
tests/docker/Makefile.include | 4 ++++
tests/docker/dockerfiles/buildroot-nios2-cross.docker | 10 ++++++++++
2 files changed, 14 insertions(+)
create mode 100644 tests/docker/dockerfiles/buildroot-nios2-cross.docker

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 74a82de48a..a8dfde8ed5 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -120,6 +120,10 @@ docker-image-debian-riscv64-cross: docker-image-debian-sid
docker-image-debian-powerpc-cross: docker-image-debian-sid
docker-image-travis: NOUSER=1

+# Buildroot base images
+# These involve building the toolchains and can take some time
+docker-image-buildroot-nios2-cross: docker-image-debian-buildroot-base
+
# Specialist build images, sometimes very limited tools
docker-image-tricore-cross: docker-image-debian9

diff --git a/tests/docker/dockerfiles/buildroot-nios2-cross.docker b/tests/docker/dockerfiles/buildroot-nios2-cross.docker
new file mode 100644
index 0000000000..e573f0fa55
--- /dev/null
+++ b/tests/docker/dockerfiles/buildroot-nios2-cross.docker
@@ -0,0 +1,10 @@
+#
+# NIOS II toolchain
+#
+FROM qemu:debian-buildroot-base
+
+RUN cd /opt/buildroot-${BUILDROOT_VERSION} && make qemu_nios2_10m50_defconfig
+RUN cd /opt/buildroot-${BUILDROOT_VERSION} && make toolchain
+
+# The toolchain is in /opt/buildroot-${BUILDROOT_VERSION}/output/host/bin/nios2-*
+RUN ln -s /opt/buildroot-${BUILDROOT_VERSION}/output/host/bin/nios2-* /usr/bin
--
2.17.1

Revision history for this message

Alex Bennée (ajbennee) wrote on 2018-09-11: [RFC PATCH 3/4] linux-user/nios2: bump min uname to 4.16.0 [!HACK]

#6

This is to work around the limitations of the buildroot
qemu_nios2_10m50_defconfig which sets the base kernel version for
glibc.

Signed-off-by: Alex Bennée <email address hidden>
---
linux-user/nios2/target_syscall.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/linux-user/nios2/target_syscall.h b/linux-user/nios2/target_syscall.h
index ca6b7e69f6..905b80d112 100644
--- a/linux-user/nios2/target_syscall.h
+++ b/linux-user/nios2/target_syscall.h
@@ -2,7 +2,7 @@
#define TARGET_SYSCALL_H

#define UNAME_MACHINE "nios2"
-#define UNAME_MINIMUM_RELEASE "3.19.0"
+#define UNAME_MINIMUM_RELEASE "4.16.0"

struct target_pt_regs {
unsigned long r8; /* r8-r15 Caller-saved GP registers */
--
2.17.1

Revision history for this message

Alex Bennée (ajbennee) wrote on 2018-09-11: [RFC PATCH 4/4] tests/tcg: add nios2 architecture (NEEDS FIXES)

#7

Now we have a docker image with a nios2 compiler we can add the bits
to build our TCG tests.

Current failures:
testmmap - fails in check_file_fixed_eof_mmaps due to inversion of offset
linux-test - unimplemented lseek (probably others as well)

Signed-off-by: Alex Bennée <email address hidden>
---
tests/tcg/nios2/Makefile.include | 2 ++
1 file changed, 2 insertions(+)
create mode 100644 tests/tcg/nios2/Makefile.include

diff --git a/tests/tcg/nios2/Makefile.include b/tests/tcg/nios2/Makefile.include
new file mode 100644
index 0000000000..2ab4160435
--- /dev/null
+++ b/tests/tcg/nios2/Makefile.include
@@ -0,0 +1,2 @@
+DOCKER_IMAGE=buildroot-nios2-cross
+DOCKER_CROSS_COMPILER=nios2-linux-gcc
--
2.17.1

Revision history for this message

Laurent Vivier (laurent-vivier) wrote on 2018-09-11: Re: [RFC PATCH 3/4] linux-user/nios2: bump min uname to 4.16.0 [!HACK]

#8

Le 11/09/2018 à 16:06, Alex Bennée a écrit :
> This is to work around the limitations of the buildroot
> qemu_nios2_10m50_defconfig which sets the base kernel version for
> glibc.
>
> Signed-off-by: Alex Bennée <email address hidden>
> ---
> linux-user/nios2/target_syscall.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/linux-user/nios2/target_syscall.h b/linux-user/nios2/target_syscall.h
> index ca6b7e69f6..905b80d112 100644
> --- a/linux-user/nios2/target_syscall.h
> +++ b/linux-user/nios2/target_syscall.h
> @@ -2,7 +2,7 @@
> #define TARGET_SYSCALL_H
>
> #define UNAME_MACHINE "nios2"
> -#define UNAME_MINIMUM_RELEASE "3.19.0"
> +#define UNAME_MINIMUM_RELEASE "4.16.0"
>
> struct target_pt_regs {
> unsigned long r8; /* r8-r15 Caller-saved GP registers */
>

I have no objection. Perhaps you could ask NiosII Maintainers (cc).

Laurent

Revision history for this message

Alex Bennée (ajbennee) wrote on 2018-09-11:

#9

Laurent Vivier <email address hidden> writes:

> Le 11/09/2018 à 16:06, Alex Bennée a écrit:
>> This is to work around the limitations of the buildroot
>> qemu_nios2_10m50_defconfig which sets the base kernel version for
>> glibc.
>>
>> Signed-off-by: Alex Bennée <email address hidden>
>> ---
>> linux-user/nios2/target_syscall.h | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/linux-user/nios2/target_syscall.h b/linux-user/nios2/target_syscall.h
>> index ca6b7e69f6..905b80d112 100644
>> --- a/linux-user/nios2/target_syscall.h
>> +++ b/linux-user/nios2/target_syscall.h
>> @@ -2,7 +2,7 @@
>> #define TARGET_SYSCALL_H
>>
>> #define UNAME_MACHINE "nios2"
>> -#define UNAME_MINIMUM_RELEASE "3.19.0"
>> +#define UNAME_MINIMUM_RELEASE "4.16.0"
>>
>> struct target_pt_regs {
>> unsigned long r8; /* r8-r15 Caller-saved GP registers */
>>
>
> I have no objection. Perhaps you could ask NiosII Maintainers (cc).

Doh.. I had cccmd = scripts/get_maintainer.pl --nogit-fallback but of
course as I didn't actually touch an nios2 files it didn't include them.

Thanks.

>
> Laurent

--
Alex Bennée

Revision history for this message

Laurent Vivier (laurent-vivier) wrote on 2018-09-11:

#10

Le 11/09/2018 à 16:40, Alex Bennée a écrit :
>
> Laurent Vivier <email address hidden> writes:
>
>> Le 11/09/2018 à 16:06, Alex Bennée a écrit:
>>> This is to work around the limitations of the buildroot
>>> qemu_nios2_10m50_defconfig which sets the base kernel version for
>>> glibc.
>>>
>>> Signed-off-by: Alex Bennée <email address hidden>
>>> ---
>>> linux-user/nios2/target_syscall.h | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/linux-user/nios2/target_syscall.h b/linux-user/nios2/target_syscall.h
>>> index ca6b7e69f6..905b80d112 100644
>>> --- a/linux-user/nios2/target_syscall.h
>>> +++ b/linux-user/nios2/target_syscall.h
>>> @@ -2,7 +2,7 @@
>>> #define TARGET_SYSCALL_H
>>>
>>> #define UNAME_MACHINE "nios2"
>>> -#define UNAME_MINIMUM_RELEASE "3.19.0"
>>> +#define UNAME_MINIMUM_RELEASE "4.16.0"
>>>
>>> struct target_pt_regs {
>>> unsigned long r8; /* r8-r15 Caller-saved GP registers */
>>>
>>
>> I have no objection. Perhaps you could ask NiosII Maintainers (cc).
>
> Doh.. I had cccmd = scripts/get_maintainer.pl --nogit-fallback but of
> course as I didn't actually touch an nios2 files it didn't include them.

I also use "git blame" to know who to bother ;)

Thanks,
Laurent

Revision history for this message

Alex Bennée (ajbennee) wrote on 2018-09-11:

#11

Marek Vasut <email address hidden> writes:

> On 09/11/2018 04:14 PM, Laurent Vivier wrote:
>> Le 11/09/2018 à 16:06, Alex Bennée a écrit:
>>> This is to work around the limitations of the buildroot
>>> qemu_nios2_10m50_defconfig which sets the base kernel version for
>>> glibc.
>>>
>>> Signed-off-by: Alex Bennée <email address hidden>
>>> ---
>>> linux-user/nios2/target_syscall.h | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/linux-user/nios2/target_syscall.h b/linux-user/nios2/target_syscall.h
>>> index ca6b7e69f6..905b80d112 100644
>>> --- a/linux-user/nios2/target_syscall.h
>>> +++ b/linux-user/nios2/target_syscall.h
>>> @@ -2,7 +2,7 @@
>>> #define TARGET_SYSCALL_H
>>>
>>> #define UNAME_MACHINE "nios2"
>>> -#define UNAME_MINIMUM_RELEASE "3.19.0"
>>> +#define UNAME_MINIMUM_RELEASE "4.16.0"
>>>
>>> struct target_pt_regs {
>>> unsigned long r8; /* r8-r15 Caller-saved GP registers */
>>>
>>
>> I have no objection. Perhaps you could ask NiosII Maintainers (cc).
>
> If that's needed, so be it. The Linux 3.19 was required because some
> obscure ABI change happened at that point.

I don't think so - it's an artefact of the way the buildroot toolchain
is built. But the real question which I address in the cover letter is
does nios2-linux-user get much use? I tried enabled tests/tcg for it and
it fails rather badly.

--
Alex Bennée

Revision history for this message

Sandra Loosemore (sandra.codesourcery) wrote on 2018-09-11:

#12

If you need a Nios II GNU/Linux toolchain, I think the most recent CodeBench Lite release will work:

https://sourcery.mentor.com/GNUToolchain/subscription42545

We're planning on adding user-mode QEMU to the upcoming 2018.11 release.... that's actually what I've been testing it for. Results on the GCC testsuite actually don't look too bad, but I have a patch I haven't submitted yet that's required to make the GDB stub work, and there are a lot of GDB test failures I haven't triaged yet.

Revision history for this message

Alex Bennée (ajbennee) wrote on 2018-09-11: Re: [Qemu-devel] [Bug 1791796] Re: unimplemented thread syscalls in nios2 user-mode emulation

#13

Sandra Loosemore <email address hidden> writes:

> If you need a Nios II GNU/Linux toolchain, I think the most recent
> CodeBench Lite release will work:
>
> https://sourcery.mentor.com/GNUToolchain/subscription42545

Hmm I tried automating that but it seems the installer has GTK
dependencies!?

  Setup:
  GTK+ Version Check
  Setup:
  An error has occurred. See the log file
  /root/.mentor/logs/20180911185933/.metadata/.log.
  root@0ef91b5e50f2:/opt# cat /root/.mentor/logs/20180911185933/.metadata/.log
  !SESSION 2018-09-11 18:59:36.197 -----------------------------------------------
  eclipse.buildId=unknown
  java.version=1.8.0_102
  java.vendor=Oracle Corporation
  BootLoader constants: OS=linux, ARCH=x86, WS=gtk, NL=en_US
  Framework arguments: -install.once -install.data=/root/.mentor
  Command-line arguments: -os linux -ws gtk -arch x86 -install.once -install.data=/root/.mentor -data /root/.mentor/logs/20180911185933

  !ENTRY org.eclipse.osgi 4 0 2018-09-11 18:59:37.407
  !MESSAGE Application error
  !STACK 1
  java.lang.UnsatisfiedLinkError: Could not load SWT library. Reasons:
          /tmp/sourceryg++-2018.05-5-nios2-linux-gnu.bin_sfx.f9eaefb7/installer/configuration/org.eclipse.osgi/148/0/.cp/libswt-pi-gtk-4530.so: libgtk-x11-2.0.so.0: cannot open shared object file: No such file or directory
          no swt-pi-gtk in java.library.path
          Can't load library: /root/.swt/lib/linux/x86/libswt-pi-gtk-4530.so
          Can't load library: /root/.swt/lib/linux/x86/libswt-pi-gtk.so
          /root/.swt/lib/linux/x86/libswt-pi-gtk-4530.so: libgtk-x11-2.0.so.0: cannot open shared object file: No such file or directory

Should I just try the tarball approach?

> We're planning on adding user-mode QEMU to the upcoming 2018.11
> release.... that's actually what I've been testing it for. Results on
> the GCC testsuite actually don't look too bad,

OK - I'm a little surprised given the failures I saw in our own
test/tcg/multiarch but it's totally possible that:

- the buildroot toolchain if foobar
- the (ancient) tests need tweaking

but it would be nice if we get to a point that QEMU's internal
linux-user tests also pass.

> but I have a patch I
> haven't submitted yet that's required to make the GDB stub work, and
> there are a lot of GDB test failures I haven't triaged yet.

When you do post the gdb patch feel free to CC me as I've poked around
in that before.

--
Alex Bennée

Sandra Loosemore <1791796@bugs.launchpad.net> writes:

> If you need a Nios II GNU/Linux toolchain, I think the most recent
> CodeBench Lite release will work:
>
> https://sourcery.mentor.com/GNUToolchain/subscription42545

Hmm I tried automating that but it seems the installer has GTK
dependencies!?

Setup:
  GTK+ Version Check
  Setup:
  An error has occurred. See the log file
  /root/.mentor/logs/20180911185933/.metadata/.log.
  root@0ef91b5e50f2:/opt# cat /root/.mentor/logs/20180911185933/.metadata/.log
  !SESSION 2018-09-11 18:59:36.197 -----------------------------------------------
  eclipse.buildId=unknown
  java.version=1.8.0_102
  java.vendor=Oracle Corporation
  BootLoader constants: OS=linux, ARCH=x86, WS=gtk, NL=en_US
  Framework arguments:  -install.once -install.data=/root/.mentor
  Command-line arguments:  -os linux -ws gtk -arch x86 -install.once -install.data=/root/.mentor -data /root/.mentor/logs/20180911185933

!ENTRY org.eclipse.osgi 4 0 2018-09-11 18:59:37.407
  !MESSAGE Application error
  !STACK 1
  java.lang.UnsatisfiedLinkError: Could not load SWT library. Reasons:
          /tmp/sourceryg++-2018.05-5-nios2-linux-gnu.bin_sfx.f9eaefb7/installer/configuration/org.eclipse.osgi/148/0/.cp/libswt-pi-gtk-4530.so: libgtk-x11-2.0.so.0: cannot open shared object file: No such file or directory
          no swt-pi-gtk in java.library.path
          Can't load library: /root/.swt/lib/linux/x86/libswt-pi-gtk-4530.so
          Can't load library: /root/.swt/lib/linux/x86/libswt-pi-gtk.so
          /root/.swt/lib/linux/x86/libswt-pi-gtk-4530.so: libgtk-x11-2.0.so.0: cannot open shared object file: No such file or directory

Should I just try the tarball approach?

> We're planning on adding user-mode QEMU to the upcoming 2018.11
> release....  that's actually what I've been testing it for.  Results on
> the GCC testsuite actually don't look too bad,

OK - I'm a little surprised given the failures I saw in our own
test/tcg/multiarch but it's totally possible that:

- the buildroot toolchain if foobar
  - the (ancient) tests need tweaking

but it would be nice if we get to a point that QEMU's internal
linux-user tests also pass.

> but I have a patch I
> haven't submitted yet that's required to make the GDB stub work, and
> there are a lot of GDB test failures I haven't triaged yet.

When you do post the gdb patch feel free to CC me as I've poked around
in that before.

--
Alex Bennée

Revision history for this message

Alex Bennée (ajbennee) wrote on 2018-09-12: Re: [RFC PATCH 3/4] linux-user/nios2: bump min uname to 4.16.0 [!HACK]

#14

Marek Vasut <email address hidden> writes:

> On 09/11/2018 05:08 PM, Alex Bennée wrote:
>>
>> Marek Vasut <email address hidden> writes:
>>
>>> On 09/11/2018 04:14 PM, Laurent Vivier wrote:
>>>> Le 11/09/2018 à 16:06, Alex Bennée a écrit:
<snip>
>>
>> I don't think so - it's an artefact of the way the buildroot toolchain
>> is built. But the real question which I address in the cover letter is
>> does nios2-linux-user get much use? I tried enabled tests/tcg for it and
>> it fails rather badly.
>
> I used it around 2.10 and it worked for me.

I've just build 2.10.2 (with a patch for memfd and this one) and get the
same failures on the tcg/tests tests. What testing where you running?

--
Alex Bennée

Revision history for this message

Philippe Mathieu-Daudé (philmd) wrote on 2018-09-30: Re: [RFC PATCH 1/4] docker: add debian-buildroot-base

#15

Hi Alex,

On 9/11/18 4:06 PM, Alex Bennée wrote:
> We can build some more cross-compilers using buildroot. This base
> system contains simply the minimum number of tools required for
> buildroot to work. We also download and unpack the buildroot source
> tree as that will be common for all system deriving from it.
>
> Signed-off-by: Alex Bennée <email address hidden>
> ---
> tests/docker/Makefile.include | 2 +-
> .../dockerfiles/debian-buildroot-base.docker | 26 +++++++++++++++++++
> 2 files changed, 27 insertions(+), 1 deletion(-)
> create mode 100644 tests/docker/dockerfiles/debian-buildroot-base.docker
>
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index d3101afecd..74a82de48a 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -6,7 +6,7 @@ DOCKER_SUFFIX := .docker
> DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
> DOCKER_DEPRECATED_IMAGES := debian
> # we don't run tests on intermediate images (used as base by another image)
> -DOCKER_PARTIAL_IMAGES := debian debian8 debian9 debian8-mxe debian-ports debian-sid debian-bootstrap
> +DOCKER_PARTIAL_IMAGES := debian debian8 debian9 debian8-mxe debian-ports debian-sid debian-bootstrap debian-buildroot-base
> DOCKER_IMAGES := $(filter-out $(DOCKER_DEPRECATED_IMAGES),$(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))))
> DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
> # Use a global constant ccache directory to speed up repetitive builds
> diff --git a/tests/docker/dockerfiles/debian-buildroot-base.docker b/tests/docker/dockerfiles/debian-buildroot-base.docker
> new file mode 100644
> index 0000000000..c4a29abadd
> --- /dev/null
> +++ b/tests/docker/dockerfiles/debian-buildroot-base.docker
> @@ -0,0 +1,26 @@
> +#
> +# Buildroot base setup on Debian
> +#
> +
> +FROM debian:stretch-slim
> +ENV BUILDROOT_VERSION=2018.08

To avoid cache invalidate, please move this bellow...

> +
> +# Duplicate deb line as deb-src
> +RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
> +
> +# Install common build utilities
> +RUN apt update && DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata
> +RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> + apt install -y bc \
> + build-essential \
> + cpio \
> + file \
> + python \
> + unzip \
> + rsync \
> + wget
> +
> +# Grab the current buildroot version and unpack in /opt

... here (or similar around):

ENV BUILDROOT_VERSION=2018.08

> +RUN mkdir -p /opt
> +RUN cd /opt && wget https://buildroot.org/downloads/buildroot-${BUILDROOT_VERSION}.tar.gz
> +RUN cd /opt && tar -xvf buildroot-${BUILDROOT_VERSION}.tar.gz
>

Reviewed-by: Philippe Mathieu-Daudé <email address hidden>
Tested-by: Philippe Mathieu-Daudé <email address hidden>

Hi Alex,

On 9/11/18 4:06 PM, Alex Bennée wrote:
> We can build some more cross-compilers using buildroot. This base
> system contains simply the minimum number of tools required for
> buildroot to work. We also download and unpack the buildroot source
> tree as that will be common for all system deriving from it.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/Makefile.include                 |  2 +-
>  .../dockerfiles/debian-buildroot-base.docker  | 26 +++++++++++++++++++
>  2 files changed, 27 insertions(+), 1 deletion(-)
>  create mode 100644 tests/docker/dockerfiles/debian-buildroot-base.docker
> 
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index d3101afecd..74a82de48a 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -6,7 +6,7 @@ DOCKER_SUFFIX := .docker
>  DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
>  DOCKER_DEPRECATED_IMAGES := debian
>  # we don't run tests on intermediate images (used as base by another image)
> -DOCKER_PARTIAL_IMAGES := debian debian8 debian9 debian8-mxe debian-ports debian-sid debian-bootstrap
> +DOCKER_PARTIAL_IMAGES := debian debian8 debian9 debian8-mxe debian-ports debian-sid debian-bootstrap debian-buildroot-base
>  DOCKER_IMAGES := $(filter-out $(DOCKER_DEPRECATED_IMAGES),$(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))))
>  DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
>  # Use a global constant ccache directory to speed up repetitive builds
> diff --git a/tests/docker/dockerfiles/debian-buildroot-base.docker b/tests/docker/dockerfiles/debian-buildroot-base.docker
> new file mode 100644
> index 0000000000..c4a29abadd
> --- /dev/null
> +++ b/tests/docker/dockerfiles/debian-buildroot-base.docker
> @@ -0,0 +1,26 @@
> +#
> +# Buildroot base setup on Debian
> +#
> +
> +FROM debian:stretch-slim
> +ENV BUILDROOT_VERSION=2018.08

To avoid cache invalidate, please move this bellow...

> +
> +# Duplicate deb line as deb-src
> +RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
> +
> +# Install common build utilities
> +RUN apt update && DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata
> +RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> +    apt install -y bc \
> +                   build-essential \
> +                   cpio \
> +                   file \
> +                   python \
> +                   unzip \
> +                   rsync \
> +                   wget
> +
> +# Grab the current buildroot version and unpack in /opt

... here (or similar around):

ENV BUILDROOT_VERSION=2018.08

> +RUN mkdir -p /opt
> +RUN cd /opt && wget https://buildroot.org/downloads/buildroot-${BUILDROOT_VERSION}.tar.gz
> +RUN cd /opt && tar -xvf buildroot-${BUILDROOT_VERSION}.tar.gz
>

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Revision history for this message

Philippe Mathieu-Daudé (philmd) wrote on 2018-09-30: Re: [RFC PATCH 2/4] docker: add buildroot-nios2-cross image

#16

Hi Alex,

On 9/11/18 4:06 PM, Alex Bennée wrote:
> Build a buildroot toolchain for the nios2 target.
>
> Signed-off-by: Alex Bennée <email address hidden>
> ---
> tests/docker/Makefile.include | 4 ++++
> tests/docker/dockerfiles/buildroot-nios2-cross.docker | 10 ++++++++++
> 2 files changed, 14 insertions(+)
> create mode 100644 tests/docker/dockerfiles/buildroot-nios2-cross.docker
>
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index 74a82de48a..a8dfde8ed5 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -120,6 +120,10 @@ docker-image-debian-riscv64-cross: docker-image-debian-sid
> docker-image-debian-powerpc-cross: docker-image-debian-sid
> docker-image-travis: NOUSER=1
>
> +# Buildroot base images
> +# These involve building the toolchains and can take some time
> +docker-image-buildroot-nios2-cross: docker-image-debian-buildroot-base
> +
> # Specialist build images, sometimes very limited tools
> docker-image-tricore-cross: docker-image-debian9
>
> diff --git a/tests/docker/dockerfiles/buildroot-nios2-cross.docker b/tests/docker/dockerfiles/buildroot-nios2-cross.docker
> new file mode 100644
> index 0000000000..e573f0fa55
> --- /dev/null
> +++ b/tests/docker/dockerfiles/buildroot-nios2-cross.docker
> @@ -0,0 +1,10 @@
> +#
> +# NIOS II toolchain
> +#
> +FROM qemu:debian-buildroot-base
> +
> +RUN cd /opt/buildroot-${BUILDROOT_VERSION} && make qemu_nios2_10m50_defconfig

Simply:

RUN make -C /opt/buildroot-${BUILDROOT_VERSION} qemu_nios2_10m50_defconfig

> +RUN cd /opt/buildroot-${BUILDROOT_VERSION} && make toolchain

RUN make -C /opt/buildroot-${BUILDROOT_VERSION} toolchain

> +# The toolchain is in /opt/buildroot-${BUILDROOT_VERSION}/output/host/bin/nios2-*
> +RUN ln -s /opt/buildroot-${BUILDROOT_VERSION}/output/host/bin/nios2-* /usr/bin

Similarly:

ENV PATH $PATH:/opt/buildroot-${BUILDROOT_VERSION}/output/host/bin

Once build this image takes a bit more than 3GB (this took me 30min to
build).

With changes:
Reviewed-by: Philippe Mathieu-Daudé <email address hidden>
Tested-by: Philippe Mathieu-Daudé <email address hidden>

Hi Alex,

On 9/11/18 4:06 PM, Alex Bennée wrote:
> Build a buildroot toolchain for the nios2 target.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/Makefile.include                         |  4 ++++
>  tests/docker/dockerfiles/buildroot-nios2-cross.docker | 10 ++++++++++
>  2 files changed, 14 insertions(+)
>  create mode 100644 tests/docker/dockerfiles/buildroot-nios2-cross.docker
> 
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index 74a82de48a..a8dfde8ed5 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -120,6 +120,10 @@ docker-image-debian-riscv64-cross: docker-image-debian-sid
>  docker-image-debian-powerpc-cross: docker-image-debian-sid
>  docker-image-travis: NOUSER=1
>  
> +# Buildroot base images
> +# These involve building the toolchains and can take some time
> +docker-image-buildroot-nios2-cross: docker-image-debian-buildroot-base
> +
>  # Specialist build images, sometimes very limited tools
>  docker-image-tricore-cross: docker-image-debian9
>  
> diff --git a/tests/docker/dockerfiles/buildroot-nios2-cross.docker b/tests/docker/dockerfiles/buildroot-nios2-cross.docker
> new file mode 100644
> index 0000000000..e573f0fa55
> --- /dev/null
> +++ b/tests/docker/dockerfiles/buildroot-nios2-cross.docker
> @@ -0,0 +1,10 @@
> +#
> +# NIOS II toolchain
> +#
> +FROM qemu:debian-buildroot-base
> +
> +RUN cd /opt/buildroot-${BUILDROOT_VERSION} && make qemu_nios2_10m50_defconfig

Simply:

RUN make -C /opt/buildroot-${BUILDROOT_VERSION} qemu_nios2_10m50_defconfig

> +RUN cd /opt/buildroot-${BUILDROOT_VERSION} && make toolchain

RUN make -C /opt/buildroot-${BUILDROOT_VERSION} toolchain

> +# The toolchain is in /opt/buildroot-${BUILDROOT_VERSION}/output/host/bin/nios2-*
> +RUN ln -s /opt/buildroot-${BUILDROOT_VERSION}/output/host/bin/nios2-* /usr/bin

Similarly:

ENV PATH $PATH:/opt/buildroot-${BUILDROOT_VERSION}/output/host/bin

Once build this image takes a bit more than 3GB (this took me 30min to
build).

With changes:
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Revision history for this message

Alex Bennée (ajbennee) wrote on 2018-10-01:

#17

Philippe Mathieu-Daudé <email address hidden> writes:

> Hi Alex,
>
> On 9/11/18 4:06 PM, Alex Bennée wrote:
>> Build a buildroot toolchain for the nios2 target.
>>
>> Signed-off-by: Alex Bennée <email address hidden>
>> ---
>> tests/docker/Makefile.include | 4 ++++
>> tests/docker/dockerfiles/buildroot-nios2-cross.docker | 10 ++++++++++
>> 2 files changed, 14 insertions(+)
>> create mode 100644 tests/docker/dockerfiles/buildroot-nios2-cross.docker
>>
>> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
>> index 74a82de48a..a8dfde8ed5 100644
>> --- a/tests/docker/Makefile.include
>> +++ b/tests/docker/Makefile.include
>> @@ -120,6 +120,10 @@ docker-image-debian-riscv64-cross: docker-image-debian-sid
>> docker-image-debian-powerpc-cross: docker-image-debian-sid
>> docker-image-travis: NOUSER=1
>>
>> +# Buildroot base images
>> +# These involve building the toolchains and can take some time
>> +docker-image-buildroot-nios2-cross: docker-image-debian-buildroot-base
>> +
>> # Specialist build images, sometimes very limited tools
>> docker-image-tricore-cross: docker-image-debian9
>>
>> diff --git a/tests/docker/dockerfiles/buildroot-nios2-cross.docker b/tests/docker/dockerfiles/buildroot-nios2-cross.docker
>> new file mode 100644
>> index 0000000000..e573f0fa55
>> --- /dev/null
>> +++ b/tests/docker/dockerfiles/buildroot-nios2-cross.docker
>> @@ -0,0 +1,10 @@
>> +#
>> +# NIOS II toolchain
>> +#
>> +FROM qemu:debian-buildroot-base
>> +
>> +RUN cd /opt/buildroot-${BUILDROOT_VERSION} && make qemu_nios2_10m50_defconfig
>
> Simply:
>
> RUN make -C /opt/buildroot-${BUILDROOT_VERSION} qemu_nios2_10m50_defconfig
>
>> +RUN cd /opt/buildroot-${BUILDROOT_VERSION} && make toolchain
>
> RUN make -C /opt/buildroot-${BUILDROOT_VERSION} toolchain
>
>> +# The toolchain is in /opt/buildroot-${BUILDROOT_VERSION}/output/host/bin/nios2-*
>> +RUN ln -s /opt/buildroot-${BUILDROOT_VERSION}/output/host/bin/nios2-* /usr/bin
>
> Similarly:
>
> ENV PATH $PATH:/opt/buildroot-${BUILDROOT_VERSION}/output/host/bin
>
> Once build this image takes a bit more than 3GB (this took me 30min to
> build).
>
> With changes:
> Reviewed-by: Philippe Mathieu-Daudé <email address hidden>
> Tested-by: Philippe Mathieu-Daudé <email address hidden>

Apparently multi-stage builds are meant to help:

https://docs.docker.com/develop/develop-images/multistage-build/#name-your-build-stages

It's still a little sub-optimal compared to binary builds but it will do
if we care about supporting every guest architecture with tests. I only
picked nios2 as a random example.

--
Alex Bennée

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Hi Alex,
>
> On 9/11/18 4:06 PM, Alex Bennée wrote:
>> Build a buildroot toolchain for the nios2 target.
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> ---
>>  tests/docker/Makefile.include                         |  4 ++++
>>  tests/docker/dockerfiles/buildroot-nios2-cross.docker | 10 ++++++++++
>>  2 files changed, 14 insertions(+)
>>  create mode 100644 tests/docker/dockerfiles/buildroot-nios2-cross.docker
>>
>> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
>> index 74a82de48a..a8dfde8ed5 100644
>> --- a/tests/docker/Makefile.include
>> +++ b/tests/docker/Makefile.include
>> @@ -120,6 +120,10 @@ docker-image-debian-riscv64-cross: docker-image-debian-sid
>>  docker-image-debian-powerpc-cross: docker-image-debian-sid
>>  docker-image-travis: NOUSER=1
>>
>> +# Buildroot base images
>> +# These involve building the toolchains and can take some time
>> +docker-image-buildroot-nios2-cross: docker-image-debian-buildroot-base
>> +
>>  # Specialist build images, sometimes very limited tools
>>  docker-image-tricore-cross: docker-image-debian9
>>
>> diff --git a/tests/docker/dockerfiles/buildroot-nios2-cross.docker b/tests/docker/dockerfiles/buildroot-nios2-cross.docker
>> new file mode 100644
>> index 0000000000..e573f0fa55
>> --- /dev/null
>> +++ b/tests/docker/dockerfiles/buildroot-nios2-cross.docker
>> @@ -0,0 +1,10 @@
>> +#
>> +# NIOS II toolchain
>> +#
>> +FROM qemu:debian-buildroot-base
>> +
>> +RUN cd /opt/buildroot-${BUILDROOT_VERSION} && make qemu_nios2_10m50_defconfig
>
> Simply:
>
>   RUN make -C /opt/buildroot-${BUILDROOT_VERSION} qemu_nios2_10m50_defconfig
>
>> +RUN cd /opt/buildroot-${BUILDROOT_VERSION} && make toolchain
>
>    RUN make -C /opt/buildroot-${BUILDROOT_VERSION} toolchain
>
>> +# The toolchain is in /opt/buildroot-${BUILDROOT_VERSION}/output/host/bin/nios2-*
>> +RUN ln -s /opt/buildroot-${BUILDROOT_VERSION}/output/host/bin/nios2-* /usr/bin
>
> Similarly:
>
>    ENV PATH $PATH:/opt/buildroot-${BUILDROOT_VERSION}/output/host/bin
>
> Once build this image takes a bit more than 3GB (this took me 30min to
> build).
>
> With changes:
> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Apparently multi-stage builds are meant to help:

https://docs.docker.com/develop/develop-images/multistage-build/#name-your-build-stages

It's still a little sub-optimal compared to binary builds but it will do
if we care about supporting every guest architecture with tests. I only
picked nios2 as a random example.

--
Alex Bennée

Revision history for this message

Thomas Huth (th-huth) wrote on 2020-11-13:

#18

The QEMU project is currently considering to move its bug tracking to another system. For this we need to know which bugs are still valid and which could be closed already. Thus we are setting older bugs to "Incomplete" now.
If you still think this bug report here is valid, then please switch the state back to "New" within the next 60 days, otherwise this report will be marked as "Expired". Or mark it as "Fix Released" if the problem has been solved with a newer version of QEMU already. Thank you and sorry for the inconvenience.

Changed in qemu:
status:	New → Incomplete

Revision history for this message

Launchpad Janitor (janitor) wrote on 2021-01-13:

#19

[Expired for QEMU because there has been no activity for 60 days.]

Changed in qemu:
status:	Incomplete → Expired

Philippe Mathieu-Daudé (philmd) on 2021-01-13

tags:

added: linux-user

QEMU

unimplemented thread syscalls in nios2 user-mode emulation

Bug Description

Other bug subscribers

Remote bug watches