automatically removed packages includes postgresql-10 which can result in cluster dropping

Bug #1825563 reported by Michael Kuhn
88
This bug affects 10 people
Affects Status Importance Assigned to Milestone
ubuntu-release-upgrader (Ubuntu)
Fix Released
High
Brian Murray
Disco
Fix Released
Undecided
Unassigned

Bug Description

[Impact]
Users who choose to autoremove packages can have postgresql-10 removed which during its purge will drop the cluster and its databases.

[Test Case]
On a cosmic system
1) install postgresql-10 and ubuntu-release-upgrader-core
2) create a test database using psql - create database junk;
3) fill that database with junk using pgbench e.g. pgbench -in -s10 junk
4) verify you have data in /var/lib/postgresql - du -sh /var/lib/postgresql/*
5) run do-release-upgrade
6) choose to automatically remove packages (watch postgresql-10 get removed)
7) observe you have no more data in /var/lib/postgresql

With the version of the release upgrader in -proposed postgresql-10 will not be removed, postgresql-11 will not be installed and you'll still have your data.

[Regression Potential]
The regex could be too generic and prevent some other packages from being removed so look at it closely.

While upgrading to Ubuntu 19.04, the release upgrader removed postgresql-10. To make matters worse, apt/dpkg did NOT ask whether the databases should be kept and simply removed all of them.

I believe this is due to removal_blacklist.cfg containing ^postgresql-.*[0-9]\.[0-9].*, which does not match newer releases such as postgresql-10 as it expects an X.Y version scheme. Something like ^postgresql-.*[0-9].* should probably be added in addition.

Revision history for this message
Brian Murray (brian-murray) wrote :

Thanks for taking the time to report this bug and I hope the databases aren't really gone. Are the databases still in /var/lib/postgresql? Could you provide the /var/log/dist-upgrade/apt-term.log file for the upgrade so we can understand why the packages were removed?

Changed in ubuntu-release-upgrader (Ubuntu):
importance: Undecided → High
status: New → Incomplete
Revision history for this message
Brian Murray (brian-murray) wrote :

I tested an upgrade from cosmic to disco with postgresql-10 installed and received a message, attached in a screenshot, regarding the transition from postgresql-10 to postgresql-11. Did you receive that message?

Revision history for this message
Michael Kuhn (suraia) wrote :

Sadly, the databases were indeed gone. do-release-upgrade purged the postgresql-10 package, which dropped the cluster. I have a backup from last night, so I could restore a fairly recent state.

I have attached the relevant part of apt-term.log (the rest of the file contained config file diffs with passwords etc.); if you need anything else, please let me know. postgresql-10 was removed during the last step of do-release-upgrade (removing automatically installed packages).

I cannot remember a dialog such as the one shown in your screenshot. Should the prompt be recorded in apt-term.log? If so, I guess it did not pop up since I cannot find anything similar in there.

Changed in ubuntu-release-upgrader (Ubuntu):
status: Incomplete → In Progress
summary: - Upgrade to Ubuntu 19.04 removed PostgreSQL databases
+ automatically removed packages includes postgresql-10 which can result
+ in cluster dropping
Changed in ubuntu-release-upgrader (Ubuntu):
assignee: nobody → Brian Murray (brian-murray)
description: updated
tags: added: disco eoan
Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello Michael, or anyone else affected,

Accepted ubuntu-release-upgrader into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:19.04.16.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in ubuntu-release-upgrader (Ubuntu Disco):
status: New → Fix Committed
tags: added: verification-needed verification-needed-disco
Revision history for this message
Brian Murray (brian-murray) wrote :
Download full text (4.8 KiB)

I've verified this in an lxc container using the instructions provided.

root@cosmic:~# cat /tmp/ubuntu-release-upgrader-vhkwxwud/DistUpgradeVersion.py
VERSION = '19.04.16.1'
root@cosmic:~# grep "postgresql-10" /var/log/dist-upgrade/main.log
2019-04-19 23:28:49,876 DEBUG Keep at same version: bash-completion bcache-tools bsdmainutils byobu bzip2 command-not-found command-not-found-data cpio cron dosfstools eject finalrd friendly-recovery htop iproute2 less libatm1 libbind9-160 libbz2-1.0 libdns-export1102 libdns1102 libestr0 libevent-2.1-6 libexpat1 libfastjson4 libgdbm5 libgeoip1 libgpm2 libicu60 libidn2-0 libirs160 libisc-export169 libisc169 libisccc160 libisccfg160 libksba8 liblwres160 liblzo2-2 libmnl0 libmpdec2 libnetfilter-conntrack3 libnfnetlink0 libnpth0 libntfs-3g88 libpcap0.8 libpci3 libperl5.26 libpython3.6 libpython3.6-minimal libpython3.6-stdlib libreadline5 libreadline7 librtmp1 libsepol1 libsigsegv2 libslang2 libssl1.0.0 libtasn1-6 libtext-wrapi18n-perl libusb-1.0-0 libutempter0 libx11-6 libx11-data libxau6 libxdmcp6 libxext6 libxmuu1 libxslt1.1 libyaml-0-2 linux-base lshw manpages mawk mime-support mlocate mtr-tiny pastebinit patch pciutils perl-modules-5.26 pinentry-curses pollinate popularity-contest postgresql-10 postgresql-client-10 powermgmt-base python3-asn1crypto python3-automat python3-blinker python3-certifi python3-colorama python3-commandnotfound python3-configobj python3-constantly python3-hyperlink python3-idna python3-incremental python3-jinja2 python3-json-pointer python3-jsonpatch python3-pyasn1 python3-pyasn1-modules python3-requests-unixsocket python3-serial python3-service-identity python3.6 python3.6-minimal run-one screen sensible-utils shared-mime-info ssh-import-id ssl-cert sysvinit-utils tcpdump time ubuntu-keyring xauth xfsprogs xkb-data zerofree
2019-04-19 23:29:45,113 DEBUG Keep at same version: bash-completion bcache-tools bsdmainutils byobu bzip2 command-not-found command-not-found-data cpio cron dosfstools eject finalrd friendly-recovery htop iproute2 less libatm1 libbind9-160 libbz2-1.0 libc-bin libc6 libdns-export1102 libdns1102 libestr0 libevent-2.1-6 libexpat1 libfastjson4 libgdbm5 libgeoip1 libgpm2 libicu60 libidn2-0 libirs160 libisc-export169 libisc169 libisccc160 libisccfg160 libksba8 liblwres160 liblzo2-2 libmnl0 libmpdec2 libnetfilter-conntrack3 libnfnetlink0 libnpth0 libntfs-3g88 libpcap0.8 libpci3 libperl5.26 libpython3.6 libpython3.6-minimal libpython3.6-stdlib libreadline5 libreadline7 librtmp1 libsepol1 libsigsegv2 libslang2 libssl1.0.0 libtasn1-6 libtext-wrapi18n-perl libusb-1.0-0 libutempter0 libx11-6 libx11-data libxau6 libxdmcp6 libxext6 libxmuu1 libxslt1.1 libyaml-0-2 linux-base locales lshw manpages mawk mime-support mlocate mtr-tiny pastebinit patch pciutils perl-modules-5.26 pinentry-curses pollinate popularity-contest postgresql-10 postgresql-client-10 powermgmt-base python3-asn1crypto python3-automat python3-blinker python3-certifi python3-colorama python3-commandnotfound python3-configobj python3-constantly python3-hyperlink python3-idna python3-incremental python3-jinja2 python3-json-pointer python3-jsonpatch python3-pyasn1 python3-pyasn1-modules python3-requests-unixsoc...

Read more...

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-release-upgrader - 1:19.04.16.1

---------------
ubuntu-release-upgrader (1:19.04.16.1) disco; urgency=medium

  * data/removal_blacklist.cfg: add blacklist entry for postgresql versions
    with two major digits e.g. postgresql-11. (LP: #1825563)
  * Update of demotions and mirrors data.

 -- Brian Murray <email address hidden> Fri, 19 Apr 2019 15:25:07 -0700

Changed in ubuntu-release-upgrader (Ubuntu Disco):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote :

I've also updated the meta-release file so that the release-upgrader from -updates will now be used. Thanks again for reporting for this and I'm truly sorry that we missed adding this to the blacklist.

Revision history for this message
Michael Kuhn (suraia) wrote :

At least now I know that my backups work. :-) Thanks for the quick fix!

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-release-upgrader - 1:19.10.1

---------------
ubuntu-release-upgrader (1:19.10.1) eoan; urgency=medium

  * DistUpgrade/{EOL,Devel}ReleaseAnnouncement,
    DistUpgrade/window_main.ui,
    data/gtkbuilder/DistUpgrade.ui,
    data/DistUpgrade.cfg,
    pre-build.sh:
    - updated for 19.04 -> 19.10
  * data/DistUpgrade.cfg.bionic,
    data/demoted.cfg.bionic,
    pre-build.sh:
    - support upgrades from bionic (LP: #1827635)
  * data/removal_blacklist.cfg: add blacklist entry for postgresql versions
    with two major digits e.g. postgresql-11. (LP: #1825563)
  * Fix broken symbolic link of DistUpgrade/DistUpgrade which caused code from
    the installed ubuntu-release-upgrader to be used instead of code in the
    dist-upgrader tarball. (LP: #1824430)

 -- Brian Murray <email address hidden> Fri, 03 May 2019 10:24:48 -0700

Changed in ubuntu-release-upgrader (Ubuntu):
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.