dbconfig-common: writing config to /etc/dbconfig-common/maas-region-controller.conf stat: cannot stat '/etc/dbconfig-common/maas-region-controller.conf': No such file or directory

[Andres Rodriguez]

In MAAS, we make use of dbconfig common. We have been using it exactly the same way for awhile now and we have just starting seeing the following on install or upgrade:

dbconfig-common: writing config to /etc/dbconfig-common/maas-region-controller.conf
stat: cannot stat '/etc/dbconfig-common/maas-region-controller.conf': No such file or directory

Also, before this it always asks whether to use the local version of the file or install a new one in case of an upgrade.

[Paul Gevers (paul-climbing)]

Hi Andres,

On 06-03-16 17:42, Andres Rodriguez wrote:
> In MAAS, we make use of dbconfig common. We have been using it exactly
> the same way for awhile now and we have just starting seeing the
> following on install or upgrade:
>
> dbconfig-common: writing config to /etc/dbconfig-common/maas-region-controller.conf

dbconfig-common has always done that. So nothing new.

> stat: cannot stat '/etc/dbconfig-common/maas-region-controller.conf': No such file or directory

Any idea why this happens? What is special about your setup? How did you
configure dbconfig-common? Is the /etc/dbconfig-common directory
missing, or is the maas-region-controller.conf file missing? Is this on
BOTH install and upgrade or only during install or only during upgrade?

> Also, before this it always asks whether to use the local version of the
> file or install a new one in case of an upgrade.

I am not 100% sure if I understand what you mean. The behavior that you
describe seems to be the question raised by ucf. Are you now saying that
that is not happening anymore? That would indeed be a serious bug, but
nothing on that front has changed. The only thing that I can think of on
this front is that I removed the date of installation, such that ucf
will not ask the question if the content of the file has not changed.
Can you confirm that this is what you mean?

Furthermore, there is important information missing: about which version
of dbconfig-common are you talking? With which version did you
experience the alleged change?

Paul

[Andres Rodriguez (andreserl)]

Hi Paul

Sorry for not being clear enough. The actual issue/concern I have is:

stat: cannot stat '/etc/dbconfig-common/maas-region-controller.conf': No such file or directory

So, I've only been able to reproduce in my system so far, but I have not being able to reproduce it in any cleanly installed system with the latest version of dbconfig-common in Xenial. (2.0.3). At first I thought this was just my system being broken, but I now have not only been the one coming across this issue. See https://bugs.launchpad.net/maas/+bug/1553648. That being said, I must say that I've been using Xenial since very early, and I just started seen this recently, probably the last couple weeks. Obviously, I've been upgrading pretty much daily, but it would match the latest sync from Debian.

Now, the way I'm reproducing this issue right now is an install. I have remove/purged all the packages, include dbconfig common packages and made sure that /etc/dbconfig-common doesn't exist :

roaksoax@unleashed:~/Desktop/project/packaging-1.10⟫ dpkg -l | grep dbconfig
1 roaksoax@unleashed:~/Desktop/project/packaging-1.10⟫ sudo apt-get install maas
Reading package lists... Done
[...]
The following additional packages will be installed:
  dbconfig-common dbconfig-pgsql isc-dhcp-server libecap2v5 maas-cli maas-common maas-dhcp maas-dns maas-proxy maas-rack-controller
[...]
Do you want to continue? [Y/n] y

The installation resumes, and It asks me the question to: http://pastebin.ubuntu.com/15319155/ .Right after answering the questions (no matter if I say install the package maintainer's version or keep the local version currently installed) , it is always the same result:

Setting up maas-region-controller (2.0.0~alpha1+bzr4736-0ubuntu1~xenial2) ...
dbconfig-common: writing config to /etc/dbconfig-common/maas-region-controller.conf
stat: cannot stat '/etc/dbconfig-common/maas-region-controller.conf': No such file or directory
creating postgres user maas: success.
verifying creation of user: success.
[....]
roaksoax@unleashed:~/Desktop/project/packaging-1.10⟫ ls -l /etc/dbconfig-common/
total 4
-rw------- 1 root root 484 Mar 6 23:52 config

As you can see, the file doesn't get created. Now, what I'm really wondering if is there is only an issue in systems where dbconfig-common was first upgraded to the latest version (2.0.3), and not on a freshly installed system where dbconfig-common is latest, as I've not been able to reproduce this on a freshly installed system yet.

PS:
Now, to answer your last question, the question raised by UCF was never there because we preseed the following:

set_question maas-region-controller/dbconfig-install true

Or maybe, the preseeding doesn't really matter and we just have never gotten into a similar situation, hence it never really has been shown before.

[Paul Gevers (paul-climbing)]

Hi Andres,

On 07-03-16 06:07, Andres Rodriguez wrote:
> Sorry for not being clear enough.

NP, as long as you follow up on questions that is ok.

> The actual issue/concern I have is:
>
> stat: cannot stat '/etc/dbconfig-common/maas-region-controller.conf': No
> such file or directory

Ack. I have a belly feeling that this may be related to the
implementation in ucf of Debian wish list bug 812321¹. Can you try once
again after you "touch /etc/dbconfig-common/maas-region-controller.conf".

> Now, the way I'm reproducing this issue right now is an install. I have
> remove/purged all the packages, include dbconfig common packages and
> made sure that /etc/dbconfig-common doesn't exist :

This looks like dbconfig-common isn't cleaning up properly on purge and
keeps the file registered with ucf. Weird, I'll try to investigate, but
I do purging a lot with all kind of example packages in my test suite²
and have never seen this before.

> The installation resumes, and It asks me the question to: http://pastebin.ubuntu.com/15319155/

Right. As expected, these are questions from ucf. So it may be that the
issue lies there.

> Right after answering the questions (no matter if I say install the
> package maintainer's version or keep the local version currently
> installed) , it is always the same result:
>
> Setting up maas-region-controller (2.0.0~alpha1+bzr4736-0ubuntu1~xenial2) ...
> dbconfig-common: writing config to /etc/dbconfig-common/maas-region-controller.conf
> stat: cannot stat '/etc/dbconfig-common/maas-region-controller.conf': No such file or directory

That at least is a bug in ucf. When you answer to use the package
maintainers version, it must create the file as requested. So I guess we
could at least file an additional bug against ucf. But let's wait until
we find out if there isn't just one issue here.

> As you can see, the file doesn't get created. Now, what I'm really
> wondering if is there is only an issue in systems where dbconfig-common
> was first upgraded to the latest version (2.0.3), and not on a freshly
> installed system where dbconfig-common is latest, as I've not been able
> to reproduce this on a freshly installed system yet.

As said, I suspect this may be because the file is still registered with
ucf. During install (after purge or on a clean system) ucf should not
need to ask the question. It only should do that when there once was
already a file (either still is or ucf knows that the admin has deleted
the file).

> Now, to answer your last question, the question raised by UCF was never there because we preseed the following:
>
> set_question maas-region-controller/dbconfig-install true

Doesn't look related. But how ugly is this, anyways. This means that you
are preventing an admin from defering the setup to later, e.g. because
she doesn't have the admin rights on the database he wants to use. By
the way, you do know that when an admin installs dbconfig-no-thanks,
dbconfig-common will not run, irrespective of your preseeding.

> Or maybe, the preseeding doesn't really matter and we just have never
> gotten into a similar situation, hence it never really has been shown
> before.

As said, this may be related to...

[Paul Gevers (paul-climbing)]

Hi Andres,

Maybe unrelated and I may not understand your intentions, but I see the
following in your maas-region-controller.config:
set_question() {
    if ! db_fget "$1" seen; then
        db_register dbconfig-common/dbconfig-install "$1"
        db_subst "$1" ID "$1"
        db_fget "$1" seen
    fi

Which gets called by:
    set_question maas-region-controller/dbconfig-install true
    set_question maas-region-controller/pgsql/app-pass ""
    set_question maas-region-controller/remote/host "localhost"

But isn't the question from the second set, the $1 in the first
function? So shouldn't that "db_register" line also register the other
questions and use the value provided as argument?

Paul

[Paul Gevers (paul-climbing)]

Andres,

This (in maas-region-controller.postrm) prevents dbconfig-common from
unregistring the file with ucf (so the fact that you see the ucf prompt
after purging and installing is at least caused by maas itself):
if [ -f /etc/dbconfig-common/maas.conf ]; then
    if [ -f /usr/share/dbconfig-common/dpkg/postrm ]; then
        . /usr/share/dbconfig-common/dpkg/postrm.pgsql
        dbc_go maas-region-controller $@
    fi
fi

Paul

[Andres Rodriguez (andreserl)]

So the reason we did this is because we wanted MAAS to be installed and we
didn't want any question to be asked, so we decided to do the preseeding.
The registration piece is something that I found in another package, but
allowed MAAS to register the preseed options, otherwise this would have
failed!

On Mon, Mar 7, 2016 at 3:59 PM, Paul Gevers <email address hidden>
wrote:

> Hi Andres,
>
> Maybe unrelated and I may not understand your intentions, but I see the
> following in your maas-region-controller.config:
> set_question() {
> if ! db_fget "$1" seen; then
> db_register dbconfig-common/dbconfig-install "$1"
> db_subst "$1" ID "$1"
> db_fget "$1" seen
> fi
>
>
> Which gets called by:
> set_question maas-region-controller/dbconfig-install true
> set_question maas-region-controller/pgsql/app-pass ""
> set_question maas-region-controller/remote/host "localhost"
>
> But isn't the question from the second set, the $1 in the first
> function? So shouldn't that "db_register" line also register the other
> questions and use the value provided as argument?
>
> Paul
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1553757
>
> Title:
> dbconfig-common: writing config to /etc/dbconfig-common/maas-region-
> controller.conf stat: cannot stat '/etc/dbconfig-common/maas-region-
> controller.conf': No such file or directory
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/dbconfig-common/+bug/1553757/+subscriptions
>

--
Andres Rodriguez (RoAkSoAx)
Ubuntu Server Developer
MSc. Telecom & Networking
Systems Engineer

[Andres Rodriguez (andreserl)]

Hi Paul

Sorry for the delay and thanks for looking into it. Good catch on the
postrm. At least I know that's my fault. That being said, I wonder if
unregistering the file actually explains why it doesn't get re-created. The
funny thing here is that this is the first time it actually does something
similar (not creating the file), and the packaging hasn;'t really change in
that area.

I'm gonna try to reproduce what you suggest with touching the file and see
whether things get resolved with the fix to postrm..

Thanks!

On Mon, Mar 7, 2016 at 4:02 PM, Paul Gevers <email address hidden>
wrote:

> Andres,
>
> This (in maas-region-controller.postrm) prevents dbconfig-common from
> unregistring the file with ucf (so the fact that you see the ucf prompt
> after purging and installing is at least caused by maas itself):
> if [ -f /etc/dbconfig-common/maas.conf ]; then
> if [ -f /usr/share/dbconfig-common/dpkg/postrm ]; then
> . /usr/share/dbconfig-common/dpkg/postrm.pgsql
> dbc_go maas-region-controller $@
> fi
> fi
>
> Paul
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1553757
>
> Title:
> dbconfig-common: writing config to /etc/dbconfig-common/maas-region-
> controller.conf stat: cannot stat '/etc/dbconfig-common/maas-region-
> controller.conf': No such file or directory
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/dbconfig-common/+bug/1553757/+subscriptions
>

--
Andres Rodriguez (RoAkSoAx)
Ubuntu Server Developer
MSc. Telecom & Networking
Systems Engineer

[Paul Gevers (paul-climbing)]

Hi Andres,

On 08-03-16 04:38, Andres Rodriguez wrote:
> So the reason we did this is because we wanted MAAS to be installed and we
> didn't want any question to be asked, so we decided to do the preseeding.

I recommend that you try to use dbc_prio_high=medium instead of your
current code. By default only questions with prio=high are asked, so I
think that the default behavior will be that dbconfig-common is run as
you desire, without removing the possibility by the admin to configure
otherwise. Also the default "remote-host" is localhost. You would also
need way less code to "work around" dbconfig-common "limitations".

> The registration piece is something that I found in another package, but
> allowed MAAS to register the preseed options, otherwise this would have
> failed!

My point is that is looks like you are registering the wrong question
for your template, aren't you? Or am I mistaken here/

Paul

[Paul Gevers (paul-climbing)]

Hi Andres,

On 08-03-16 04:41, Andres Rodriguez wrote:
> Sorry for the delay and thanks for looking into it. Good catch on the
> postrm. At least I know that's my fault. That being said, I wonder if
> unregistering the file actually explains why it doesn't get re-created.

Hmm, I was a bit to certain in my phrasing. I should have said: it COULD
be caused by the admin and the way maas handles purging. By no means did
I mean that it is certain here.

> The
> funny thing here is that this is the first time it actually does something
> similar (not creating the file), and the packaging hasn;'t really change in
> that area.

Neither has dbconfig-common.

> I'm gonna try to reproduce what you suggest with touching the file and see
> whether things get resolved with the fix to postrm..

Good.

By the way, I tried removing the configuration file while working with
one of my example packages (included in the source of dbconfig-common)
and the behavior of ucf is as expected.

Unfortunately, on my Debian system I can't easily check installing maas
with apt-get, as that is not available in Debian.

Paul

[Andres Rodriguez (andreserl)]

Hi Paul,

So this is what I did:

1. Modified the postrm to not check for
/etc/dbconfig-common/maas-region-controller.conf
2. purged maas
3. re-installed maas and the result was:
dbconfig-common: writing config to
/etc/dbconfig-common/maas-region-controller.conf
Creating config file /etc/dbconfig-common/maas-region-controller.conf with
new version

The, I did:
1. Install maas with buggy postrm.
2. purged maas. / dbconfig*, /etc/dbconfig-common was left with
maas-region-controller.postinst, so I removed it.
3. Installed maas again, and the result:
dbconfig-common: writing config to
/etc/dbconfig-common/maas-region-controller.conf
stat: cannot stat '/etc/dbconfig-common/maas-region-controller.conf': No
such file or directory
4. purged maas/dbconfig, but manually fixed maas' postrm.

Finally I did:
1. Installed maas, result:
dbconfig-common: writing config to
/etc/dbconfig-common/maas-region-controller.conf
Creating config file /etc/dbconfig-common/maas-region-controller.conf with
new version
2. purged maas/dbconfig* with fixed postrm. /etc/dbconfig-common got
removed.
4. Installed maas again, and things went well.

So this confirms that because of the postrm bug, we would see the install
issues. So fixing this is MAAS definitely fixes the error I was having. I'm
gonna mark the bug invalid, however, I wonder if dbconfig-common should do
something to not to write a new config ?

On Tue, Mar 8, 2016 at 2:32 PM, Paul Gevers <email address hidden>
wrote:

> Hi Andres,
>
> On 08-03-16 04:41, Andres Rodriguez wrote:
> > Sorry for the delay and thanks for looking into it. Good catch on the
> > postrm. At least I know that's my fault. That being said, I wonder if
> > unregistering the file actually explains why it doesn't get re-created.
>
> Hmm, I was a bit to certain in my phrasing. I should have said: it COULD
> be caused by the admin and the way maas handles purging. By no means did
> I mean that it is certain here.
>
> > The
> > funny thing here is that this is the first time it actually does
> something
> > similar (not creating the file), and the packaging hasn;'t really change
> in
> > that area.
>
> Neither has dbconfig-common.
>
> > I'm gonna try to reproduce what you suggest with touching the file and
> see
> > whether things get resolved with the fix to postrm..
>
> Good.
>
> By the way, I tried removing the configuration file while working with
> one of my example packages (included in the source of dbconfig-common)
> and the behavior of ucf is as expected.
>
> Unfortunately, on my Debian system I can't easily check installing maas
> with apt-get, as that is not available in Debian.
>
> Paul
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1553757
>
> Title:
> dbconfig-common: writing config to /etc/dbconfig-common/maas-region-
> controller.conf stat: cannot stat '/etc/dbconfig-common/maas-region-
> controller.conf': No such file or directory
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/dbconfig-common/+bug/1553757/+subscriptions
>

--
Andres Rodriguez (RoAkSoAx)
Ubuntu Server Developer
MSc. Te...

[Andres Rodriguez (andreserl)]

Thanks a lot for all the help!!

[Paul Gevers (paul-climbing)]

Hi Andres,

On 10-03-16 02:47, Andres Rodriguez wrote:
> So this confirms that because of the postrm bug, we would see the install
> issues.

Good that we nailed it down.

> however, I wonder if dbconfig-common should do
> something to not to write a new config ?

As dbconfig-common leaves this to ucf, I think than it should be done
there. Or do you have something else in mind?

Paul

[Paul Gevers (paul-climbing)]

Hi Andres,

On 10-03-16 16:57, Paul Gevers wrote:
>> however, I wonder if dbconfig-common should do
>> something to not to write a new config ?
>
> As dbconfig-common leaves this to ucf, I think than it should be done
> there. Or do you have something else in mind?

It seems that there IS a bug in ucf when the source file is not present.
At least the patch in Debian bug 817982¹ suggests as much.

Paul

¹ https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=817982