Database upgrade/migration fails with nested db directories (lucid to precise)

Bug #1003854 reported by Kenny Millington
52
This bug affects 9 people
Affects Status Importance Assigned to Milestone
openldap (Debian)
Fix Released
Undecided
Unassigned
openldap (Ubuntu)
Fix Released
Medium
Unassigned
Precise
Won't Fix
Medium
Unassigned

Bug Description

Hi,

I've just performed an upgrade of our LDAP server on Ubuntu 10.04.4 LTS to Ubuntu 12.04 (I acknowledge this upgrade path is not officially supported yet).

The incompatible database upgrading process in the preinst/postinst files failed in the following scenario.

We have two suffixes/databases at the following paths:-

 * /var/lib/ldap
 * /var/lib/ldap/accesslog

The preinst database dumping part of the process worked just fine and created the appropriate LDIF files under /var/backup/slapd-2.4.21-0ubuntu5.7, however the restore failed stating:-

"""
  Loading from /var/backups/slapd-2.4.21-0ubuntu5.7:
  - directory dc=REDACTEDs,dc=co,dc=uk... failed.

Loading the database from the LDIF dump failed with the following
error while running slapadd:
    4fbdfebf olcDbDirectory: value #0: invalid path: No such file or directory
    4fbdfebf config error processing olcDatabase={2}hdb,cn=config: olcDbDirectory: value #0: invalid path: No such file or directory
    slapadd: bad configuration directory!
"""

This is because when move_incompatible_databases_away() runs it finds the main database first (/var/lib/ldap) and moves all top level entries (find -mindepth 1 -maxdepth 1 ...) into the backup directory and this includes the accesslog subdirectory which then no longer exists. When slapadd runs it checks config specifying that directory and bails with the above error given it is indeed missing.

I've tested a tentative fix and that's to patch the two find commands (one in is_empty_dir() one in move_old_database_away to also specify -type f so that the directory structure is preserved when moving the old database away (accesslog will be backed up separately when its suffx is iterated over in move_incompatible_databases_away()).

The simple and very tentative patch for this is:-

"""
# diff -u slapd.scripts-common.old slapd.scripts-common
--- slapd.scripts-common.old 2012-05-24 10:33:01.746206585 +0100
+++ slapd.scripts-common 2012-05-24 10:33:23.967902747 +0100
@@ -391,7 +391,7 @@
   echo -n " - directory $suffix... " >&2
   mkdir -p "$backupdir"
   find "$databasedir" -mindepth 1 -maxdepth 1 \
- -exec mv {} "$backupdir" \;
+ -type f -exec mv {} "$backupdir" \;
   echo done. >&2
  else
   cat >&2 <<EOF
@@ -728,7 +728,7 @@
 # (i.e., contains no files except for an optional DB_CONFIG).
 # Usage: if is_empty_dir "$dir"; then ... fi

- output=`find "$1" -mindepth 1 -maxdepth 1 \! -name DB_CONFIG 2>/dev/null`
+ output=`find "$1" -mindepth 1 -maxdepth 1 -type f \! -name DB_CONFIG 2>/dev/null`
  if [ -n "$output" ]; then
       return 1
  else
"""

Related branches

CVE References

Revision history for this message
Kenny Millington (kmdm) wrote :

(If requested I can provide a suitable debdiff for the proposed fix)

Scott Moser (smoser)
Changed in openldap (Ubuntu):
importance: Undecided → Medium
status: New → Confirmed
Changed in openldap (Ubuntu Precise):
status: New → Confirmed
importance: Undecided → Medium
Revision history for this message
Maarten Rijke (infie) wrote :

I encountered this bug today as well, when upgrading from 10.04.4 to 12.04.1.
Applying Kenny's changes to the /var/lib/dpkg/info/slapd.{pre,post}inst fixed the problem. Thanks!

Rolf Leggewie (r0lf)
tags: added: lucid2precise
tags: added: patch
Revision history for this message
Rolf Leggewie (r0lf) wrote :

Lucid to precise is very much a supported update path. Can you try to get your patch included in Debian? We should then backport the patch to raring and precise.

Changed in openldap (Ubuntu Precise):
assignee: nobody → Rolf Leggewie (r0lf)
Revision history for this message
GaryProsser (gary-prosser) wrote :

We just upgraded from 10.04 to 12.04 and subsequently our nightly backup of ldap database reports the same error as this bug report.

Our backup script run from root's crontab is

/usr/sbin/slapcat -F /etc/ldap/slapd.d -l /root/system/ldap.ldif

the error reported is

51256a8a olcDbDirectory: value #0: invalid path: No such file or directory
51256a8a config error processing olcDatabase={2}hdb,cn=config: olcDbDirectory: value #0: invalid path: No such file or directory
slapcat: bad configuration directory!

the database structure is

/var/lib/ldap
/var/lib/ldap/accesslog

meanwhile

#slaptest -u
config file testing succeeded

Rolf Leggewie (r0lf)
Changed in openldap (Ubuntu Precise):
assignee: Rolf Leggewie (r0lf) → nobody
Revision history for this message
Paul Bergene (paulbergene) wrote :

This problem seems to still exist.
I encountered it this week when upgrading from 10.4 to 12.04.3.

The changes above work, but since I had a half-installed slapd package changing /var/lib/dpkg/info/slapd.config was what did the trick.

The scripts that I got from apt were still the old broken version.

description: updated
Revision history for this message
Johan Smits (johan-smits) wrote :

It also affects upgrading from precise to trusty!
Suggested changes work.

Revision history for this message
Ryan Tandy (rtandy) wrote :

Did anyone ever forward this to Debian?

The patch seems reasonable to me. I don't know of any situation where bdb/hdb/mdb create subdirectories under /var/lib/ldap, only files.

I've committed this to the pkg-openldap Git repository. Thanks for the patch!

Changed in openldap (Debian):
status: New → Fix Committed
Revision history for this message
Ryan Tandy (rtandy) wrote :

Fixed in Debian in 2.4.40-1.

Changed in openldap (Debian):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (10.3 KiB)

This bug was fixed in the package openldap - 2.4.40+dfsg-1ubuntu1

---------------
openldap (2.4.40+dfsg-1ubuntu1) wily; urgency=low

  * Merge from Debian testing (LP: #1395098, LP: #1316124). Remaining changes:
    - Enable AppArmor support:
      - d/apparmor-profile: add AppArmor profile
      - d/rules: use dh_apparmor
      - d/control: Build-Depends on dh-apparmor
      - d/slapd.README.Debian: add note about AppArmor
    - Enable GSSAPI support:
      - d/patches/gssapi.diff, thanks to Jerry Carter (Likewise):
        - Add --with-gssapi support
        - Make guess_service_principal() more robust when determining
          principal
      - d/configure.options: Configure with --with-gssapi
      - d/control: Added heimdal-dev as a build depend
    - Enable ufw support:
      - d/control: suggest ufw.
      - d/rules: install ufw profile.
      - d/slapd.ufw.profile: add ufw profile.
    - Enable nss overlay:
      - d/{patches/nssov-build,rules}: Apply, build and package the
        nss overlay.
    - d/{rules,slapd.py}: Add apport hook.
    - d/slapd.init.ldif: don't set olcRootDN since it's not defined in
      either the default DIT nor via an Authn mapping.
    - d/slapd.scripts-common:
      - add slapcat_opts to local variables.
      - Remove unused variable new_conf.
      - Fix backup directory naming for multiple reconfiguration.
    - d/{slapd.default,slapd.README.Debian}: use the new configuration style.
    - d/rules: Enable -DLDAP_CONNECTIONLESS to build CLDAP (UDP) support
      in the openldap library, as required by Likewise-Open
    - Show distribution in version:
      - d/control: added lsb-release
      - d/patches/fix-ldap-distribution.patch: show distribution in version
  * Drop patches included upstream:
    - d/patches/0001-ITS-7430-GnuTLS-Avoid-use-of-deprecated-function.patch
    - d/patches/bdb-deadlock.patch
    - d/patches/its-7354-fix-delta-sync-mmr.diff
  * Drop hardening-wrapper as Debian now sets PIE and bindnow flags.
  * debian/patches/nssov-build: Adjust for upstream changes.
  * debian/apparmor-profile:
    - Change 'r' to 'rw' for ldapi and nslcd sockets, required for apparmor
      kernel ABI v7 (utopic and later). (LP: #1392018)
    - Reduce permissions on /run/nslcd to just the nslcd socket.
  * Enable the mdb backend again on ppc64el, fixed upstream in ITS#7713.
    (LP: #1293250)

openldap (2.4.40+dfsg-1) unstable; urgency=medium

  * Remove inetorgperson.schema from the upstream source. Replace it with a
    copy stripped of RFC text. (Closes: #780283)
  * Adjust debian/watch for +dfsg versioning.
  * debian/patches/ITS7975-fix-mdb-onelevel-search.patch: Import upstream
    patch to fix scope=onelevel searches wrongly including the search base in
    results under the MDB backend. (ITS#7975) (Closes: #782212)

openldap (2.4.40-4) unstable; urgency=medium

  * debian/patches/ITS8027-deref-reject-empty-attr-list.patch: Import upstream
    patch to fix a crash when a search includes the Deref control with an
    empty attribute list. (ITS#8027) (CVE-2015-1545, Closes: #776988)
  * debian/patches/ITS8046-fix-vrFilter_free-crash.patch: Import upstream
    patch to fix a double free triggered by...

Changed in openldap (Ubuntu):
status: Confirmed → Fix Released
Revision history for this message
Nick Peelman (peelman) wrote :

Encountered a similar error when updating from 12.04 to 14.04.3. The patch (at least the patch alone) did not fix it.

What did finally make it succeed was `mkdir /var/lib/ldap/accesslog`

via https://techhelplist.com/tech-tutorials/34-openldap/332-cloning-my-openldap-data-between-servers

Revision history for this message
Kenny Millington (kmdm) wrote :

Did you re-export the backup after applying the patch?

IIRC it fixes the backup/export phase not the restore/import phase so unless you re-created the dumps it wouldn't have actually done anything.

Revision history for this message
Emi Bcn (emi) wrote :

@Kenny: how do you re-export the backup? I just do `dpkg --configure -a`.

Revision history for this message
Dave Jones (waveform) wrote :

Also encountered this when upgrading a server from 12.04 to 14.04 (for reference, wound up filing duplicate bug #1579566). Managed to get slapd restored happily by creating the accesslog directory (then slapadd to restore).

Unfortunately as I restored the database, rather than apt handling it, the slapd package is now in "iF" state (according to dpkg -l). Attempting to fix this with "apt-get -f install" leads to the original situation: the package attempts to backup and restore the database unsuccessfully presumably because the slapd package it's attempting to update to is 2.4.31 (the current version in trusty) while the fixed version is the later 2.4.40.

Any hints on how to resolve the situation (e.g. by simply marking the package "installed"? slapd does appear to be working happily with the manually restored database)

Revision history for this message
Dave Jones (waveform) wrote :

Ah, no worries - finally figured it out. Needed to apply the patch to /var/lib/dpkg/info/slapd.postinst (rather than slapd.config - was confused over package's half-configured state) and restore the old-version backup prior to retrying "dpkg --configure -a". All good now!

Revision history for this message
Andre Becker (andrebecker) wrote :

Hi Dave,i have exact the same problem here.
Could you describe me step by step how you successfully upgraded?
Did you anything before the 'do-release-upgrade' or everything after
the the issue occurred?
thx!

Revision history for this message
Dave Jones (waveform) wrote :

Hi Andre - I did take a backup of the LDAP directory prior to "do-release-upgrade" (actually I had nightly backups running so I just grabbed a copy of the last one of those). That's what I was referring to when I stated "restore the old-version backup...". It's been a few months now but I vaguely recall that the upgrade procedure needs the state of the database prior to the (failed) upgrade in order to succeed after patching.

Hence, if my recollection is accurate, my procedure was:

1. Patch the slapd.postinst script
2. Restore the database to its state prior to the (failed) upgrade
3. Run "dpkg --configure -a" to cause the slapd postinst script to upgrade the database successfully

Revision history for this message
Andre Becker (andrebecker) wrote :

Hi Dave, thx for your reply!
for testing purposes i took a machine with an ubuntu 12.04 and installed the slapd
exactly like in my productive system. Then i made a do-release-upgrade and the same error occurred.
I done the slapcat from the productive server for all 3 LDAP-DBs and putted all 3 *.ldifs into the testmachine with slapadd and everything looks fine after starting the slapd.
I checked the state of the slapd package with dpkg -l and it looked normal ("ii")

So, is it necessary to do the patch in that case?
Where do i find the patch (if necessary at all)?

Did you took the whole LDAP-directories (/var/lib/ldap and /etc/ldap) instead of the slapcat / slapadd process?

Revision history for this message
Dave Jones (waveform) wrote :

My backup was indeed slapcat / slapadd (for both the main directory and the cn=config stuff). I'd *guess* that if the package state is "ii" you may not need the patch (incidentally the patch is in bug description at the top), particularly if the server's starting fine and everything queries happily. Mine didn't start after the upgrade so I definitely needed the patch, but I wouldn't like to guess about other situations (given how variable LDAP setups can be).

Revision history for this message
Andre Becker (andrebecker) wrote :

thx again ... as i see the patch only manage some stuff for dealing with the directories while upgrading. So i think i don´t need it at all.
tonight i will perform the upgrade on my productive system
;)

Revision history for this message
Steve Langasek (vorlon) wrote :

The Precise Pangolin has reached end of life, so this bug will not be fixed for that release

Changed in openldap (Ubuntu Precise):
status: Confirmed → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.