tests/Makefile.include trying to add linking library '-lutil' that break the build on Solaris

[WHR]

Building script 'tests/Makefile.include' contains following code
```
ifeq ($(CONFIG_POSIX),y)
LIBS += -lutil
endif
```

library -lutil is not available on Solaris, so the building will failed, like
```
ld: fatal: library -lutil: not found
make: *** [SOMEWHERE/src/qemu-2.12.0/rules.mak:121: qemu-nbd] Error 1
```

Commenting those code out fixed the error.

[Thomas Huth (th-huth)]

I'm sorry, but Solaris is currently unsupported and might get removed in a future release, see:

 https://wiki.qemu.org/ChangeLog/2.12#Warning:_unsupported_host_systems

So it would be great if you could contribute patches, or find someone who's willing to maintain QEMU on Solaris.

[Thomas Huth (th-huth)]

Does something like this work for you?

diff --git a/tests/Makefile.include b/tests/Makefile.include
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -777,7 +777,7 @@ tests/migration/initrd-stress.img: tests/migration/stress$(EXESUF)
        rm $(INITRD_WORK_DIR)/init
        rmdir $(INITRD_WORK_DIR)

-ifeq ($(CONFIG_POSIX),y)
+ifeq ($(CONFIG_POSIX)$(call lnot,$(CONFIG_SOLARIS)),yy)
 LIBS += -lutil
 endif

[Michele (denber)]

That's great. I spent quite a while last summer looking for the source of that error, without success. I finally just created a stub library named lutil figuring I would then find which function was missing on the next build attempt. But it worked fine so apparently QEMU does not even use whatever is in lutil, at least on Solaris.

I'm still offering to help maintain this as I have some actual Solaris hardware.

[Peter Maydell (pmaydell)]

The condition we use in configure to decide whether to add -lutil to libs_softmmu is "not Darwin and not MinGW and not Solaris and not Haiku"...

I think the thing we're trying to get from libutil is openpty(), which is in that library for Linux and the BSDs. It's not in libutil for OSX (though since OSX has a libutil anyway it's harmless that it gets linked in). For Windows we avoid code paths that use openpty(), and Solaris doesn't have openpty() at all so we have to hand-roll our own implementation in util/qemu-openpty.c.

I think the cleanest fix to this problem is to have a configure test that checks "does openpty() require -lutil?", and then use the answer to that both to decide whether to add -lutil to libs_softmmu and to decide whether to add -lutil to the link line for this test binary.

PS: Thanks for the offer of Solaris hardware access. We may be able to take you up on that but it would also require some time from somebody on our end to set things up to use it. The other possible option is to have one of the free-solaris-equivalents set up as a VM to run tests in in the same way we do for the BSDs in tests/vm.

[WHR (msl0000023508)]

>>> Does something like this work for you?

Yes, this fix works; but I don't think this is a clean fix, since libutil may also missing in other OS.

[Thomas Huth (th-huth)]

Problem has been fixed here:
https://git.qemu.org/?p=qemu.git;a=commitdiff;h=d99e97e6912d90a55e9