qemu parallel building error on libcacard.la

[tal zilcer]

hi,

im building qemu with a large make -j value(9).
sometimes the build fails because of an error:
libtool: link: ar cru .libs/libcacard.a stubs/arch-query-cpu-def.o stubs/clock-warp.o stubs/cpu-get-clock.o stubs/cpu-get-icount.o stubs/dump.o stubs/fdset-add-fd.o stubs/fdset-find-fd.o stubs/fdset-get-fd.o stubs/fdset-remove-fd.o stubs/gdbstub.o stubs/get-fd.o stubs/get-vm-name.o stubs/iothread-lock.o stubs/migr-blocker.o stubs/mon-is-qmp.o stubs/mon-printf.o stubs/mon-print-filename.o stubs/mon-protocol-event.o stubs/mon-set-error.o stubs/pci-drive-hot-add.o stubs/qtest.o stubs/reset.o stubs/runstate-check.o stubs/set-fd-handler.o stubs/slirp.o stubs/sysbus.o stubs/uuid.o stubs/vm-stop.o stubs/vmstate.o stubs/cpus.o stubs/kvm.o libcacard/cac.o libcacard/event.o libcacard/vcard.o libcacard/vreader.o libcacard/vcard_emul_nss.o libcacard/vcard_emul_type.o libcacard/card_7816.o libcacard/vcardt.o util/osdep.o util/cutils.o util/qemu-timer-common.o util/error.o util/qemu-error.o util/oslib-posix.o util/qemu-thread-posix.o trace/generated-events.o trace/default.o trace/control.o trace/generated-tracers.o
ar: trace/generated-events.o: No such file or directory
make[2]: *** [libcacard.la] Error 1

i see the build of generated-events.o in the log before the ar command.
because of the -j it was probably not completed yet.
the generated-events.o build command:
/usr/bin/gcc -I/home/npsdb/qemu/qemu/tcg -I/home/npsdb/qemu/qemu/tcg/i386 -I/home/npsdb/qemu/qemu/linux-headers -I/home/npsdb/qemu/build/linux_x86_64/linux-headers -I. -I/home/npsdb/qemu/qemu -I/home/npsdb/jenkins/qemu/qemu/include -I/home/npsdb/qemu/qemu/libcacard -Itrace -Itrace -fPIE -DPIE -m64 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-all -I/usr/include/libpng12 -I/usr/include/nss3 -I/usr/include/nspr4 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1 -I/home/npsdb/qemu/qemu/tests -I qga/qapi-generated -MMD -MP -MT trace/generated-events.o -MF trace/generated-events.d -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -g -c -o trace/generated-events.o trace/generated-events.c

must be a race condition in the makefile because of a missing dependency.
i tried to find it but it was a little bit complicated to me.

thanks,
tal

[tal zilcer (talz-3)]

Version 2.0.0

Isn’t the libcacard Makefile run as a recursive make?
how do you keep the dependencies between two different makes(libcacard and trace)?

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Stefan Hajnoczi
Sent: Wednesday, June 04, 2014 11:19 AM
To: Tal Zilcer
Subject: Re: [Qemu-devel] [Bug 1324112] [NEW] qemu parallel building error on libcacard.la

On Wed, May 28, 2014 at 12:52:10PM -0000, tal zilcer wrote:
> im building qemu with a large make -j value(9).
> sometimes the build fails because of an error:
> libtool: link: ar cru .libs/libcacard.a stubs/arch-query-cpu-def.o stubs/clock-warp.o stubs/cpu-get-clock.o stubs/cpu-get-icount.o stubs/dump.o stubs/fdset-add-fd.o stubs/fdset-find-fd.o stubs/fdset-get-fd.o stubs/fdset-remove-fd.o stubs/gdbstub.o stubs/get-fd.o stubs/get-vm-name.o stubs/iothread-lock.o stubs/migr-blocker.o stubs/mon-is-qmp.o stubs/mon-printf.o stubs/mon-print-filename.o stubs/mon-protocol-event.o stubs/mon-set-error.o stubs/pci-drive-hot-add.o stubs/qtest.o stubs/reset.o stubs/runstate-check.o stubs/set-fd-handler.o stubs/slirp.o stubs/sysbus.o stubs/uuid.o stubs/vm-stop.o stubs/vmstate.o stubs/cpus.o stubs/kvm.o libcacard/cac.o libcacard/event.o libcacard/vcard.o libcacard/vreader.o libcacard/vcard_emul_nss.o libcacard/vcard_emul_type.o libcacard/card_7816.o libcacard/vcardt.o util/osdep.o util/cutils.o util/qemu-timer-common.o util/error.o util/qemu-error.o util/oslib-posix.o util/qemu-thread-posix.o trace/generated-events.o trace/default.o trace/control.o trace/generated-tracers.o
> ar: trace/generated-events.o: No such file or directory
> make[2]: *** [libcacard.la] Error 1
>
>
> i see the build of generated-events.o in the log before the ar command.
> because of the -j it was probably not completed yet.
> the generated-events.o build command:
> /usr/bin/gcc -I/home/npsdb/qemu/qemu/tcg -I/home/npsdb/qemu/qemu/tcg/i386 -I/home/npsdb/qemu/qemu/linux-headers -I/home/npsdb/qemu/build/linux_x86_64/linux-headers -I. -I/home/npsdb/qemu/qemu -I/home/npsdb/jenkins/qemu/qemu/include -I/home/npsdb/qemu/qemu/libcacard -Itrace -Itrace -fPIE -DPIE -m64 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-all -I/usr/include/libpng12 -I/usr/include/nss3 -I/usr/include/nspr4 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1 -I/home/npsdb/qemu/qemu/tests -I qga/qapi-generated -MMD -MP -MT trace/generated-events.o -MF trace/generated-events.d -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -g -c -o trace/generated-events.o trace/generated-events.c
>
>
> must be a race condition in the makefile because of a missing dependency.
> i tried to find it but it was a little bit complicated to me.

Which QEMU ...

[Stefan Hajnoczi (stefanha)]

On Wed, May 28, 2014 at 12:52:10PM -0000, tal zilcer wrote:
> im building qemu with a large make -j value(9).
> sometimes the build fails because of an error:
> libtool: link: ar cru .libs/libcacard.a stubs/arch-query-cpu-def.o stubs/clock-warp.o stubs/cpu-get-clock.o stubs/cpu-get-icount.o stubs/dump.o stubs/fdset-add-fd.o stubs/fdset-find-fd.o stubs/fdset-get-fd.o stubs/fdset-remove-fd.o stubs/gdbstub.o stubs/get-fd.o stubs/get-vm-name.o stubs/iothread-lock.o stubs/migr-blocker.o stubs/mon-is-qmp.o stubs/mon-printf.o stubs/mon-print-filename.o stubs/mon-protocol-event.o stubs/mon-set-error.o stubs/pci-drive-hot-add.o stubs/qtest.o stubs/reset.o stubs/runstate-check.o stubs/set-fd-handler.o stubs/slirp.o stubs/sysbus.o stubs/uuid.o stubs/vm-stop.o stubs/vmstate.o stubs/cpus.o stubs/kvm.o libcacard/cac.o libcacard/event.o libcacard/vcard.o libcacard/vreader.o libcacard/vcard_emul_nss.o libcacard/vcard_emul_type.o libcacard/card_7816.o libcacard/vcardt.o util/osdep.o util/cutils.o util/qemu-timer-common.o util/error.o util/qemu-error.o util/oslib-posix.o util/qemu-thread-posix.o trace/generated-events.o trace/default.o trace/control.o trace/generated-tracers.o
> ar: trace/generated-events.o: No such file or directory
> make[2]: *** [libcacard.la] Error 1
>
>
> i see the build of generated-events.o in the log before the ar command.
> because of the -j it was probably not completed yet.
> the generated-events.o build command:
> /usr/bin/gcc -I/home/npsdb/qemu/qemu/tcg -I/home/npsdb/qemu/qemu/tcg/i386 -I/home/npsdb/qemu/qemu/linux-headers -I/home/npsdb/qemu/build/linux_x86_64/linux-headers -I. -I/home/npsdb/qemu/qemu -I/home/npsdb/jenkins/qemu/qemu/include -I/home/npsdb/qemu/qemu/libcacard -Itrace -Itrace -fPIE -DPIE -m64 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-all -I/usr/include/libpng12 -I/usr/include/nss3 -I/usr/include/nspr4 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1 -I/home/npsdb/qemu/qemu/tests -I qga/qapi-generated -MMD -MP -MT trace/generated-events.o -MF trace/generated-events.d -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -g -c -o trace/generated-events.o trace/generated-events.c
>
>
> must be a race condition in the makefile because of a missing dependency.
> i tried to find it but it was a little bit complicated to me.

Which QEMU version are you building?

In qemu.git/master I can see the dependency on trace/* in
libcacard/Makefile.

Stefan

[Stefan Hajnoczi (stefanha)]

On Wed, Jun 04, 2014 at 03:33:34AM -0000, tal zilcer wrote:
> Version 2.0.0
>
> Isn’t the libcacard Makefile run as a recursive make?
> how do you keep the dependencies between two different makes(libcacard and trace)?

In qemu.git/master:Makefile I see:
ifeq ($(CONFIG_SMARTCARD_NSS),y)
include $(SRC_PATH)/libcacard/Makefile
endif

That doesn't look like recursive make to me, but maybe I'm missing
something.

Stefan

[tal zilcer (talz-3)]

I didn’t mention it but im building qemu out of the source tree(running the configure in a different directory).
I tried to find the missing dependency but no luck.
The build error keeps happening Here is the end of the log(please note that I don’t see here "CC trace/generated-events.o"):.
lt CC trace/generated-events.lo
  GEN qemu-doc.html
  GEN qemu.1
  GEN qemu-img.1
  CC qapi-types.o
  CC qapi-visit.o
  CC qga/commands.o
  CPP optionrom/multiboot.asm
  CPP optionrom/linuxboot.asm
  CC qga/guest-agent-command-state.o
  CC qga/main.o
  CC qga/commands-posix.o
  CPP optionrom/kvmvapic.asm
  AS optionrom/multiboot.o
  AS optionrom/linuxboot.o
  CC qga/channel-posix.o
  CC qga/qapi-generated/qga-qapi-types.o
  AS optionrom/kvmvapic.o
  Building optionrom/multiboot.img
  Building optionrom/linuxboot.img
  Building optionrom/multiboot.raw
  Building optionrom/linuxboot.raw
  Building optionrom/kvmvapic.img
  CC qga/qapi-generated/qga-qapi-visit.o
  Signing optionrom/multiboot.bin
  Signing optionrom/linuxboot.bin
  CC qga/qapi-generated/qga-qmp-marshal.o
  CC qemu-img.o
  CC qmp-marshal.o
  LINK qemu-bridge-helper
install -c -m 0644 /home/talz/workspace/tools/qemu/qemu/sysconfigs/target/target-x86_64.conf "/home/talz/workspace/ /tools/qemu/install/linux_x86_64/etc/qemu"
  Building optionrom/kvmvapic.raw
  Signing optionrom/kvmvapic.bin
  lt CC trace/generated-tracers.lo
lt LINK libcacard.la
  AR libqemustub.a
  CC trace/generated-tracers.o
ar: .libs/libcacard.a: Error reading trace/generated-events.o: No such file or directory
make[2]: *** [libcacard.la] Error 1

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Stefan Hajnoczi
Sent: Monday, June 09, 2014 4:38 PM
To: Tal Zilcer
Subject: Re: [Qemu-devel] [Bug 1324112] [NEW] qemu parallel building error on libcacard.la

On Wed, Jun 04, 2014 at 03:33:34AM -0000, tal zilcer wrote:
> Version 2.0.0
>
> Isn’t the libcacard Makefile run as a recursive make?
> how do you keep the dependencies between two different makes(libcacard and trace)?

In qemu.git/master:Makefile I see:
ifeq ($(CONFIG_SMARTCARD_NSS),y)
include $(SRC_PATH)/libcacard/Makefile
endif

That doesn't look like recursive make to me, but maybe I'm missing
something.

Stefan

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1324112

Title:
  qemu parallel building error on libcacard.la

Status in QEMU:
  New

Bug description:
  hi,

  im building qemu with a large make -j value(9).
  sometimes the build fails because of an error:
  libtool: link: ar cru .libs/libcacard.a stubs/arch-query-cpu-def.o stubs/clock-warp.o stubs/cpu-get-clock.o stubs/cpu-get-icount.o stubs/dump.o stubs/fdset-add-fd.o stubs/fdset-find-fd.o stubs/fdset-get-fd.o stubs/fdset-remove-fd.o stubs/gdbstub.o stubs/get-fd.o stubs/get-vm-name.o stubs/iothread-lock.o stubs/migr-blocker.o stubs/mon-is-qmp.o stubs/mon-printf.o stubs/mon-print-filename.o stubs/mon-protocol-event.o stubs/mon-set-error.o stubs/pci-drive-ho...

[tal zilcer (talz-3)]

I did some more experiments and maybe I found the issue.
I added a two prints to rules.mak:
%.o: %.c
        $(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) $($@-cflags) -c -o $@ $<," CC $(TARGET_DIR)$@")
        @echo ooooooooooooo:$@

%.lo: %.c
        $(call quiet-command,$(LIBTOOL) --mode=compile --tag=CC $(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) -c -o $@ $<," lt CC $@")
        @echo lolololololo:$@

This two prints symbolize the end of the file build.
Here is the end of the log that I got for a failed build(using V=1):

 libtool: compile: /home/talz/workspace/mmmmm/EZdk-nps/tools/3rd-party-libs/ccache/ccache_output/bin/ccache /usr/bin/gcc -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/tcg -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/tcg/i386 -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/linux-headers -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/build/linux_x86_64/linux-headers -I. -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/include -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/libcacard -Itrace -Itrace -DPIE -m64 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-all -I/usr/include/libpng12 -I/usr/include/nss3 -I/usr/include/nspr4 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1 -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/tests -MMD -MP -MT trace/generated-tracers.lo -MF trace/generated-tracers.d -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -g -c trace/generated-tracers.c -fPIE -o trace/generated-tracers.o >/dev/null 2>&1
lolololololo:trace/generated-tracers.lo
libtool --mode=link --tag=CC c++ -fPIE -DPIE -m64 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-all -I/usr/include/libpng12 -I/usr/include/nss3 -I/usr/include/nspr4 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1 -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/tests -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -g -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g -rpath /home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/install/linux_x86_64/lib -no-undefined -export-syms /home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/libcacard/libcacard.syms -o libcacard.la stubs/arch-query-cpu-def.lo stubs/clock-warp...

[tal zilcer (talz-3)]

To summarize what I think is the problem:
Libcacard.la depends on trace/generated*.la files(ibcacard.la: $(libcacard-lobj-y))
Trace/generated*.o files depends on trace/generated*.la files($(libcacard-obj-y): | $(libcacard-lobj-y))
Also util depends on generated-*.o files (util-obj-y += generated-events.o)
This means when libcacard.la is being build generated-*.o files can be build by the util target.
I think you should change libcacard.la dependencies to include the o files and not only the la files.

Thanks,
tal

-----Original Message-----
From: Tal Zilcer
Sent: Tuesday, June 10, 2014 3:46 PM
To: 'Bug 1324112'
Subject: RE: [Qemu-devel] [Bug 1324112] [NEW] qemu parallel building error on libcacard.la

I did some more experiments and maybe I found the issue.
I added a two prints to rules.mak:
%.o: %.c
        $(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) $($@-cflags) -c -o $@ $<," CC $(TARGET_DIR)$@")
        @echo ooooooooooooo:$@

%.lo: %.c
        $(call quiet-command,$(LIBTOOL) --mode=compile --tag=CC $(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) -c -o $@ $<," lt CC $@")
        @echo lolololololo:$@

This two prints symbolize the end of the file build.
Here is the end of the log that I got for a failed build(using V=1):

 libtool: compile: /home/talz/workspace/mmmmm/EZdk-nps/tools/3rd-party-libs/ccache/ccache_output/bin/ccache /usr/bin/gcc -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/tcg -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/tcg/i386 -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/linux-headers -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/build/linux_x86_64/linux-headers -I. -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/include -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/libcacard -Itrace -Itrace -DPIE -m64 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-all -I/usr/include/libpng12 -I/usr/include/nss3 -I/usr/include/nspr4 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1 -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/tests -MMD -MP -MT trace/generated-tracers.lo -MF trace/generated-tracers.d -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -g -c trace/generated-tracers.c -fPIE -o trace/generated-tracers.o >/dev/null 2>&1
lolololololo:trace/generated-tracers.lo
libtool --mode=link --tag=CC c++ -fPIE -DPIE -m64 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignore...

[tal zilcer (talz-3)]

The following patch solved the issue for me:
--- a/qemu/libcacard/Makefile
+++ b/qemu/libcacard/Makefile
@@ -26,8 +26,8 @@ vscclient$(EXESUF): libcacard/vscclient.o libcacard.la
 libcacard.la: LDFLAGS += -rpath $(libdir) -no-undefined \
        -export-syms $(SRC_PATH)/libcacard/libcacard.syms
 libcacard.la: LIBS = $(libcacard_libs)
-libcacard.la: $(libcacard-lobj-y)
- $(call LINK,$^)
+libcacard.la: $(libcacard-obj-y)
+ $(call LINK,$(libcacard-lobj-y))

 libcacard.pc: $(SRC_PATH)/libcacard/libcacard.pc.in
        $(call quiet-command,sed -e 's|@LIBDIR@|$(libdir)|' \

-----Original Message-----
From: Tal Zilcer
Sent: Tuesday, June 10, 2014 6:09 PM
To: 'Bug 1324112'
Subject: RE: [Qemu-devel] [Bug 1324112] [NEW] qemu parallel building error on libcacard.la

To summarize what I think is the problem:
Libcacard.la depends on trace/generated*.la files(ibcacard.la: $(libcacard-lobj-y))
Trace/generated*.o files depends on trace/generated*.la files($(libcacard-obj-y): | $(libcacard-lobj-y))
Also util depends on generated-*.o files (util-obj-y += generated-events.o)
This means when libcacard.la is being build generated-*.o files can be build by the util target.
I think you should change libcacard.la dependencies to include the o files and not only the la files.

Thanks,
tal

-----Original Message-----
From: Tal Zilcer
Sent: Tuesday, June 10, 2014 3:46 PM
To: 'Bug 1324112'
Subject: RE: [Qemu-devel] [Bug 1324112] [NEW] qemu parallel building error on libcacard.la

I did some more experiments and maybe I found the issue.
I added a two prints to rules.mak:
%.o: %.c
        $(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) $($@-cflags) -c -o $@ $<," CC $(TARGET_DIR)$@")
        @echo ooooooooooooo:$@

%.lo: %.c
        $(call quiet-command,$(LIBTOOL) --mode=compile --tag=CC $(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) -c -o $@ $<," lt CC $@")
        @echo lolololololo:$@

This two prints symbolize the end of the file build.
Here is the end of the log that I got for a failed build(using V=1):

 libtool: compile: /home/talz/workspace/mmmmm/EZdk-nps/tools/3rd-party-libs/ccache/ccache_output/bin/ccache /usr/bin/gcc -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/tcg -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/tcg/i386 -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/linux-headers -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/build/linux_x86_64/linux-headers -I. -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/include -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/libcacard -Itrace -Itrace -DPIE -m64 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-all -I/usr/include/libpng12 -I/usr/include/nss3 -I/usr/include/nsp...

[Stefan Hajnoczi (stefanha)]

On Tue, Jun 10, 2014 at 04:35:36PM -0000, tal zilcer wrote:
> The following patch solved the issue for me:
> --- a/qemu/libcacard/Makefile
> +++ b/qemu/libcacard/Makefile
> @@ -26,8 +26,8 @@ vscclient$(EXESUF): libcacard/vscclient.o libcacard.la
> libcacard.la: LDFLAGS += -rpath $(libdir) -no-undefined \
> -export-syms $(SRC_PATH)/libcacard/libcacard.syms
> libcacard.la: LIBS = $(libcacard_libs)
> -libcacard.la: $(libcacard-lobj-y)
> - $(call LINK,$^)
> +libcacard.la: $(libcacard-obj-y)
> + $(call LINK,$(libcacard-lobj-y))
>
> libcacard.pc: $(SRC_PATH)/libcacard/libcacard.pc.in
> $(call quiet-command,sed -e 's|@LIBDIR@|$(libdir)|' \

Peter, any thoughts on this Makefile issue?

Stefan

> -----Original Message-----
> From: Tal Zilcer
> Sent: Tuesday, June 10, 2014 6:09 PM
> To: 'Bug 1324112'
> Subject: RE: [Qemu-devel] [Bug 1324112] [NEW] qemu parallel building error on libcacard.la
>
> To summarize what I think is the problem:
> Libcacard.la depends on trace/generated*.la files(ibcacard.la: $(libcacard-lobj-y))
> Trace/generated*.o files depends on trace/generated*.la files($(libcacard-obj-y): | $(libcacard-lobj-y))
> Also util depends on generated-*.o files (util-obj-y += generated-events.o)
> This means when libcacard.la is being build generated-*.o files can be build by the util target.
> I think you should change libcacard.la dependencies to include the o files and not only the la files.
>
> Thanks,
> tal
>
> -----Original Message-----
> From: Tal Zilcer
> Sent: Tuesday, June 10, 2014 3:46 PM
> To: 'Bug 1324112'
> Subject: RE: [Qemu-devel] [Bug 1324112] [NEW] qemu parallel building error on libcacard.la
>
> I did some more experiments and maybe I found the issue.
> I added a two prints to rules.mak:
> %.o: %.c
> $(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) $($@-cflags) -c -o $@ $<," CC $(TARGET_DIR)$@")
> @echo ooooooooooooo:$@
>
> %.lo: %.c
> $(call quiet-command,$(LIBTOOL) --mode=compile --tag=CC $(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) -c -o $@ $<," lt CC $@")
> @echo lolololololo:$@
>
> This two prints symbolize the end of the file build.
> Here is the end of the log that I got for a failed build(using V=1):
>
> libtool: compile: /home/talz/workspace/mmmmm/EZdk-nps/tools/3rd-party-libs/ccache/ccache_output/bin/ccache /usr/bin/gcc -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/tcg -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/tcg/i386 -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/linux-headers -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/build/linux_x86_64/linux-headers -I. -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/include -I/home/talz/workspace/mmmmm/EZdk-nps/tools/qemu/qemu/libcacard -Itrace -Itrace -DPIE -m64 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wi...

[Peter Maydell (pmaydell)]

On 9 July 2014 08:49, Stefan Hajnoczi <email address hidden> wrote:
> On Tue, Jun 10, 2014 at 04:35:36PM -0000, tal zilcer wrote:
>> The following patch solved the issue for me:
>> --- a/qemu/libcacard/Makefile
>> +++ b/qemu/libcacard/Makefile
>> @@ -26,8 +26,8 @@ vscclient$(EXESUF): libcacard/vscclient.o libcacard.la
>> libcacard.la: LDFLAGS += -rpath $(libdir) -no-undefined \
>> -export-syms $(SRC_PATH)/libcacard/libcacard.syms
>> libcacard.la: LIBS = $(libcacard_libs)
>> -libcacard.la: $(libcacard-lobj-y)
>> - $(call LINK,$^)
>> +libcacard.la: $(libcacard-obj-y)
>> + $(call LINK,$(libcacard-lobj-y))
>>
>> libcacard.pc: $(SRC_PATH)/libcacard/libcacard.pc.in
>> $(call quiet-command,sed -e 's|@LIBDIR@|$(libdir)|' \
>
> Peter, any thoughts on this Makefile issue?

Not particularly; I've never looked at the libtool stuff because
I have a strong dislike of libtool... It seems a bug worth
fixing for 2.1 if you think the fix is correct.

-- PMM

[Fam Zheng (famz)]

Could be because of this rule:

> # libtool will build the .o files, too
> $(libcacard-obj-y): | $(libcacard-lobj-y)

Does removing the "|" (order deps) solve the issue?

Fam

[Stefan Hajnoczi (stefanha)]

On Thu, Jul 10, 2014 at 7:28 AM, Fam Zheng <email address hidden> wrote:
>
> Could be because of this rule:
>
> > # libtool will build the .o files, too
> > $(libcacard-obj-y): | $(libcacard-lobj-y)
>
> Does removing the "|" (order deps) solve the issue?

I don't think so:
libcacard.la: $(libcacard-lobj-y)
        $(call LINK,$^)

The problem is the libcacard-obj-y target is not required by anything.

Try this on qemu.git/master:

$ make distclean
$ ./configure
$ make libcacard/vscclient
libcacard/vscclient.o: In function `do_command':
/home/stefanha/qemu/libcacard/vscclient.c:500: undefined reference to
`vreader_get_reader_by_id'
/home/stefanha/qemu/libcacard/vscclient.c:502: undefined reference to
`vcard_emul_force_card_insert'
/home/stefanha/qemu/libcacard/vscclient.c:503: undefined reference to
`vreader_get_name'
...

The Makefile is broken. I suspect that putting proper dependencies in
place with fix this issue.

Stefan

[Stefan Hajnoczi (stefanha)]

On Thu, Jul 10, 2014 at 9:32 AM, Stefan Hajnoczi <email address hidden> wrote:
> Try this on qemu.git/master:
>
> $ make distclean
> $ ./configure
> $ make libcacard/vscclient
> libcacard/vscclient.o: In function `do_command':
> /home/stefanha/qemu/libcacard/vscclient.c:500: undefined reference to
> `vreader_get_reader_by_id'
> /home/stefanha/qemu/libcacard/vscclient.c:502: undefined reference to
> `vcard_emul_force_card_insert'
> /home/stefanha/qemu/libcacard/vscclient.c:503: undefined reference to
> `vreader_get_name'
> ...
>
> The Makefile is broken. I suspect that putting proper dependencies in
> place with fix this issue.

Please ignore, the libcacard Makefile is actually supposed to be
invoked as "make vscclient" from the QEMU root directory. These
errors were just caused by a %.o: %.c default make target.

Stefan

[Stefan Hajnoczi (stefanha)]

Since the following commit, libcacard and vscclient no longer link
against QEMU common code:

commit fd25c0e6dd1ed2aa932fa7ef814b32457bf270fd
Author: Michael Tokarev <email address hidden>
Date: Thu May 8 12:30:48 2014 +0400

    libcacard: replace qemu thread primitives with glib ones

Therefore this bug no longer exists in qemu.git/master and can be closed.

Stefan