$HOME is not set in --shell-fail prompt

[Robie Basak]

Steps to reproduce:

pull-lp-source hello
cd hello-*
# Stick "exit 1" into debian/tests/upstream-tests so it will fail
dpkg-buildpackage -us -uc -S -sd -nc -d
cd ..
autopkgtest -s -B *dsc -- lxd ubuntu-daily:lunar
echo $HOME

# Actual behaviour: $HOME is unset; expected behaviour: $HOME is set

$ dpkg-query -W autopkgtest
autopkgtest 5.25ubuntu4

I tried this on sid with the null runner, and $HOME was set correctly there. This might be a consequence of the null runner not resetting the environment (as might be expected), or point to a bug specifically in the lxd runner; I'm not sure.

I did find that $HOME is set in the actual test run. But then my test-in-development started failing in other ways because $HOME was not set in the --shell-fail environment. What I expect is that the --shell-fail environment is identical; otherwise it's difficult for debugging.

[Paride Legovini (paride)]

Hello Robie,

Let me try to recap to see if I got this right. $HOME is correctly set during the actual test run (_regardless_ of --shell-fail), but in the shell you are brought to by --shell-fail $HOME is not set. You were trying to use this environment to further develop the test, but the absence of $HOME got in the way. Is this correct?

Assuming it is:

It would be nice to make --shell and --shell-fail bring to the exact environment where the test run. However at the moment autopkgtest makes no promise in this regard, and in general I don't think it can make a "hard" promise as the test run itself may have altered the system in a way that makes it impossible to recreate the same environment. (One random example: the test run alters the system hostname. How is autopkgtest going to "undo" that to recreate the environment before the test?)

As I see it --shell and --shell-fail are mostly meant to inspect the testbed system. Most of the time it is possible to re-run tests from that environment, but that's not guaranteed.

autopkgtest uses different ways to get a shell in the testbed system, depending on the virt server in use. This is likely why noticed a difference when using the null virt server.

[Paride Legovini (paride)]

I tried following your steps to reproduce adding an "echo $HOME" before the "exit 1", and $HOME is there.

Note that --shell-fail doesn't stop execution on failure and put you in the very environment where the test was executing, but opens a brand new shell to the testbed system.

[Robie Basak (racb)]

Hi Paride! Thank you for replying so quickly. FWIW, working around this
for $HOME at least was trivial now that I know about it, so there's no
urgency on this bug. I created it to help improve things over time, and
also have somewhere to link to from my workaround :)

On Wed, Mar 22, 2023 at 02:07:11PM -0000, Paride Legovini wrote:
> Let me try to recap to see if I got this right. $HOME is correctly set
> during the actual test run (_regardless_ of --shell-fail), but in the
> shell you are brought to by --shell-fail $HOME is not set. You were
> trying to use this environment to further develop the test, but the
> absence of $HOME got in the way. Is this correct?

Right. My test didn't work initially. I wanted to know why, so I used
--shell-fail (actually the shortcut -s). I designed my test to be
idempotent, so I expected to be able to reproduce the failure and
further develop the test from that environment. But $HOME is set during
the test run but wasn't set in my --shell-fail environment, so that led
to the test failing unexpectedly earlier for those reasons, which I
found confusing.

> It would be nice to make --shell and --shell-fail bring to the exact
> environment where the test run. However at the moment autopkgtest makes
> no promise in this regard...

This might be a feature request then :-)

> ...in general I don't think it can make a
> "hard" promise as the test run itself may have altered the system in a
> way that makes it impossible to recreate the same environment. (One
> random example: the test run alters the system hostname. How is
> autopkgtest going to "undo" that to recreate the environment before the
> test?)

I'd consider that my responsibility, not autopkgtest. I understand that
anything *my* test does will not be undone. But I'd like the environment
to otherwise be the same.

[...]

> autopkgtest uses different ways to get a shell in the testbed system,
> depending on the virt server in use. This is likely why noticed a
> difference when using the null virt server.

Probably just that the null virt server doesn't reset environment
variables? That seems reasonable, anyway.

On Wed, Mar 22, 2023 at 02:13:23PM -0000, Paride Legovini wrote:
> I tried following your steps to reproduce adding an "echo $HOME" before
> the "exit 1", and $HOME is there.

Right - $HOME is there during a regular test run (that's part of the
spec, in fact!), but not during --shell-fail (with the lxd runner at
least).

> Note that --shell-fail doesn't stop execution on failure and put you in
> the very environment where the test was executing, but opens a brand new
> shell to the testbed system.

I think that's OK, but I'd like for the new environment to be
initialized in the same way as the test run was.

[Paride Legovini (paride)]

I triaged this as a wishlist item, looks like we agree it is. This would be nice to have but is not trivial to implement as all the virt server would need to be covered, each with its own way of getting a shell on the testbed system.