Hi all,

Now, The functional tests have restrictions: flaky
https://salsa.debian.org/multimedia-team/gsequencer/blob/master/debian/tests/control

bests,
Joël

On Sun, Jan 27, 2019 at 9:35 AM Matthias Klose <email address hidden> wrote:
>
> Public bug reported:
>
> gsequencer (1.4.34-1 to 2.1.34-1)
> Maintainer: Debian Multimedia Maintainers
> Section: universe/misc
> 15 days old
> autopkgtest for gsequencer/2.1.34-1: amd64: Pass, arm64: Regression ♻ , armhf: Regression ♻ , i386: Regression ♻ , ppc64el: Regression ♻ , s390x: Regression ♻
>
> The Debian maintainer provided this analysis:
> The qemu image has only 1 GB RAM assigned. This is not enough, I was able to
> reproduce the failure with 1 GB RAM, as wanting to obtain a stack-trace with GDB
> it complained, that it can't allocate memory.
>
> After assigning 8 GB of RAM the functional tests all passed on i386.
>
> By the way I would love to mark the functional tests as flaky (see DEP-8)
> https://dep-team.pages.debian.net/deps/dep8/
>
> If we would package a more recent version of GSequencer we would be able
> to run the unit-tests during autopkgtest.
>
> Note, there 2 different types of tests unit-tests and functional
> integration-tests. The
> later are not mandatory to pass at my opinion. Especially if the VM
> doesn't get enough
> computing power. I have seen VMs taking more than 2 seconds to show a GtkMenu,
> if the delay is too long (not enough computing power) the tests aren't
> reliable anymore.
>
> To target this issue, I had to adjust the timeouts and delays. Yes,
> the tests could run much
> faster. But I decided to throttle them, to target environments will
> lesser CPU resources than
> I have on my workstation.
>
> Here is the reaction time specified:
>
> http://git.savannah.nongnu.org/cgit/gsequencer.git/tree/ags/test/X/ags_functional_test_util.c?h=2.1.x#n35
>
> Here are the unit-tests to run against the installation:
>
> http://git.savannah.nongnu.org/cgit/gsequencer.git/tree/unit-system-
> tests.mk.am?h=2.1.x
>
> This is new for autopkgtest and not yet available:
>
> https://salsa.debian.org/multimedia-
> team/gsequencer/blob/master/debian/tests/ags-integration-unit-test
>
> ** Affects: gsequencer (Ubuntu)
>      Importance: Undecided
>          Status: New
>
> --
> You received this bug notification because you are subscribed to
> gsequencer in Ubuntu.
> Matching subscriptions: gsequencer, ubuntu-launchpad-gsequencer
> https://bugs.launchpad.net/bugs/1813456
>
> Title:
>   gsequencer fails autopkg tests on anything except amd64
>
> Status in gsequencer package in Ubuntu:
>   New
>
> Bug description:
>   gsequencer (1.4.34-1 to 2.1.34-1)
>   Maintainer: Debian Multimedia Maintainers
>   Section: universe/misc
>   15 days old
>   autopkgtest for gsequencer/2.1.34-1: amd64: Pass, arm64: Regression ♻ , armhf: Regression ♻ , i386: Regression ♻ , ppc64el: Regression ♻ , s390x: Regression ♻
>
>   The Debian maintainer provided this analysis:
>   The qemu image has only 1 GB RAM assigned. This is not enough, I was able to
>   reproduce the failure with 1 GB RAM, as wanting to obtain a stack-trace with GDB
>   it complained, that it can't allocate memory.
>
>   After assigning 8 GB of RAM the functional tests all passed on i386.
>
>   By the way I would love to mark the functional tests as flaky (see DEP-8)
>   https://dep-team.pages.debian.net/deps/dep8/
>
>   If we would package a more recent version of GSequencer we would be able
>   to run the unit-tests during autopkgtest.
>
>   Note, there 2 different types of tests unit-tests and functional
>   integration-tests. The
>   later are not mandatory to pass at my opinion. Especially if the VM
>   doesn't get enough
>   computing power. I have seen VMs taking more than 2 seconds to show a GtkMenu,
>   if the delay is too long (not enough computing power) the tests aren't
>   reliable anymore.
>
>   To target this issue, I had to adjust the timeouts and delays. Yes,
>   the tests could run much
>   faster. But I decided to throttle them, to target environments will
>   lesser CPU resources than
>   I have on my workstation.
>
>   Here is the reaction time specified:
>
>   http://git.savannah.nongnu.org/cgit/gsequencer.git/tree/ags/test/X/ags_functional_test_util.c?h=2.1.x#n35
>
>   Here are the unit-tests to run against the installation:
>
>   http://git.savannah.nongnu.org/cgit/gsequencer.git/tree/unit-system-
>   tests.mk.am?h=2.1.x
>
>   This is new for autopkgtest and not yet available:
>
>   https://salsa.debian.org/multimedia-
>   team/gsequencer/blob/master/debian/tests/ags-integration-unit-test
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/gsequencer/+bug/1813456/+subscriptions