Unable to migrate Intrepid configuration to Jaunty

[Lionel Dricot]

I'm upgrading my server from Intrepid to Jaunty. Straight upgrade, nothing fancy. It should just work.

Note : etckeeper was misconfigured and I ctrl+c the first upgrade attempt. It's the only bad thing I did (and I should be allowed to do so).

Then, the whole upgrade failed on slapd because :

Setting up slapd (2.4.15-1ubuntu3) ...
  Backing up /etc/ldap/slapd.conf in /var/backups/slapd-2.4.11-0ubuntu6.1... done.
  Moving old database directories to /var/backups:

  Backup path /var/backups/dc=ploum,dc=net-2.4.11-0ubuntu6.1.ldapdb exists. Giving up...
dpkg: error processing slapd (--configure):

Ok, I removed the backup (and I have to remove it at each attempt ! This is an incridibly stupid upgrade script)

Setting up slapd (2.4.15-1ubuntu3) ...
  Backing up /etc/ldap/slapd.conf in /var/backups/slapd-2.4.11-0ubuntu6.1... done.
  Moving old database directories to /var/backups:
  - directory dc=ploum,dc=net... done.
  Loading from /var/backups/slapd-2.4.11-0ubuntu6.1:
  Directory /var/lib/ldap for dc=ploum,dc=net not empty, aborting.
dpkg: error processing slapd (--configure):

WTF ? I then emptied the folder by hand (I of course keep a backup)

Setting up slapd (2.4.15-1ubuntu3) ...
  Backing up /etc/ldap/slapd.conf in /var/backups/slapd-2.4.11-0ubuntu6.1... done.
  Moving old database directories to /var/backups:
  - directory dc=ploum,dc=net... done.
  Loading from /var/backups/slapd-2.4.11-0ubuntu6.1:
  - directory dc=ploum,dc=net... failed.

Loading the database from the LDIF dump failed with the following
error while running slapadd:
    ldif_read_file: Not a directory for "/etc/ldap/slapd.conf/cn=config.ldif"
    slapadd: bad configuration directory!
dpkg: error processing slapd (--configure):

Ok, Now I'm there, stuck in the middle of the upgrade not understanding what to do and, more importantly, what that script want to do.

As I have a relatively simple ldap setup, I think the upgrade could be crazy for more complex stuffs.

[Lionel Dricot (ploum-deactivatedaccount)]

I think it has to do with a config format change that happened a while back. I was proposed to change my config format during an upgrade. I refused and now everything seems to be broken.

[Lionel Dricot (ploum-deactivatedaccount)]

The workaround for now is to keep the following packages from intrepid :

ldap-utils
libldap-2.4-2
slapd

Any attempt to upgrade those packages will break the configuration.

[Mathias Gug (mathiaz)]

As you've mentioned you're using slapd.conf instead of the dynamic cn=config backend. Only the latter is supported by the package. It seems that the upgrade script should fail with a more explicit error - however slapd.conf isn't supported anymore.

[Lionel Dricot (ploum-deactivatedaccount)]

I think that it's a ratter important issue as, AFAIK, slapd.conf was the default in Hardy.

Anyway, I migrated my configuration to slapd.d using slaptest. I'm using intrepid packages without slapd.conf at all and it's working.

But upgrading still fails :

Stopping OpenLDAP: slapd.
  Dumping to /var/backups/slapd-2.4.11-0ubuntu6.1:
  - directory dc=ploum,dc=net... <= str2entry: str2ad(olcLdapSyntaxes): attribute type undefined
=> ldif_enum_tree: failed to read entry for /etc/ldap/slapd.d//cn=config/cn=schema.ldif
slapcat: bad configuration directory!
failed.

[Salik Rafiq (chameeyass)]

I was upgrading from 8.10 to 9.04 and so not using slapd.conf. I get this error..

Setting up slapd (2.4.15-1ubuntu3) ...
  Backing up /etc/ldap/slapd.d/ in /var/backups/slapd-2.4.11-0ubuntu6.1... done.
  Moving old database directories to /var/backups:
  - directory dc=cham,dc=local... done.
  Loading from /var/backups/slapd-2.4.11-0ubuntu6.1:
  Directory /var/lib/ldap/DB1 for dc=cham,dc=local not empty, aborting.
dpkg: error processing slapd (--configure):
 subprocess post-installation script returned error exit status 1
Errors were encountered while processing:
 slapd

I keep my database in a custom location. DB1 is a soft link to a partition on my LVM RAID.

The software seems to work okay..apt just thinks is half-configured. I wouldn't mind a way to make it marked 'configured'. Unless there is some database change..but can't think why that would be the case...

[Salik Rafiq (chameeyass)]

I correct myself. my slapd is not running I have the following errors in syslog:

Apr 28 13:16:10 chamraid02 slapd[3507]: bdb(dc=cham,dc=local): Program version 4.7 doesn't match environment version 0.99
Apr 28 13:16:10 chamraid02 slapd[3507]: hdb_db_open: database "dc=cham,dc=local" cannot be opened, err -30971. Restore from backup!
Apr 28 13:16:10 chamraid02 slapd[3507]: bdb(dc=cham,dc=local): txn_checkpoint interface requires an environment configured for the transaction subsystem
Apr 28 13:16:10 chamraid02 slapd[3507]: bdb_db_close: database "dc=cham,dc=local": txn_checkpoint failed: Invalid argument (22).
Apr 28 13:16:10 chamraid02 slapd[3507]: backend_startup_one: bi_db_open failed! (-30971)
Apr 28 13:16:10 chamraid02 slapd[3507]: bdb_db_close: database "dc=cham,dc=local": alock_close failed
Apr 28 13:16:10 chamraid02 slapd[3507]: slapd stopped.

Not sure how to solve this. except to revert the slapd somehow.

[Adam Sommer (asommer)]

On Tue, Apr 28, 2009 at 8:25 AM, Salik Rafiq <email address hidden> wrote:

> I correct myself. my slapd is not running I have the following errors in
> syslog:
>
> Apr 28 13:16:10 chamraid02 slapd[3507]: bdb(dc=cham,dc=local): Program
> version 4.7 doesn't match environment version 0.99
> Apr 28 13:16:10 chamraid02 slapd[3507]: hdb_db_open: database
> "dc=cham,dc=local" cannot be opened, err -30971. Restore from backup!
> Apr 28 13:16:10 chamraid02 slapd[3507]: bdb(dc=cham,dc=local):
> txn_checkpoint interface requires an environment configured for the
> transaction subsystem
> Apr 28 13:16:10 chamraid02 slapd[3507]: bdb_db_close: database
> "dc=cham,dc=local": txn_checkpoint failed: Invalid argument (22).
> Apr 28 13:16:10 chamraid02 slapd[3507]: backend_startup_one: bi_db_open
> failed! (-30971)
> Apr 28 13:16:10 chamraid02 slapd[3507]: bdb_db_close: database
> "dc=cham,dc=local": alock_close failed
> Apr 28 13:16:10 chamraid02 slapd[3507]: slapd stopped.
>
> Not sure how to solve this. except to revert the slapd somehow.
>
> --
>

You mentioned that you keep your db in a custom location? You probably want
to make sure that location is configured in the AppArmor profile. The
profile is located in /etc/apparmor.d/usr.sbin.slapd.

Thanks.

--
Party On,
Adam

[Greek Ordono (grexk)]

This also happens when using ldap backend. I'm suspecting that the slapd.preinst is the culprit to this bug:

dump_databases() { # {{{
# If the user wants us to dump the databases they are dumped to the
# configured directory.

        local db suffix file dir failed slapcat_opts

        database_dumping_enabled || return 0

        dir=`database_dumping_destdir`
        echo >&2 " Dumping to $dir: "
        for suffix in `get_suffix`; do
                file="$dir/$suffix.ldif"
                echo -n " - directory $suffix... " >&2
                # Need to support slapd.d migration from preinst
                if [ -f "${SLAPD_CONF}" ]; then
                        slapcat_opts="-f ${SLAPD_CONF}"
                else
                        slapcat_opts="-F ${SLAPD_CONF}"
                fi
                slapcat ${slapcat_opts} -b "$suffix" > "$file" || failed=1
                if [ "$failed" ]; then
                        rm -f "$file"
                        echo failed. >&2
                        exit 1
                fi
                echo "done." >&2
        done
}

[Martin Eriksson (martin-essist)]

I have this (or similar problem)

I had to migrate slapd.conf to slapd.d. But upgrade still fails, similar to above (Salik Rafiq). Sorry fot lack of detail, but it is a "production" server in a virual environment. I have to rollback snapshot first sign of trouble.

I can try and perform this upgrade again to add some additional details for this comment. What logs do you guys need?

Note: I'm running gosa/samba on this server, schemas for this is included in ldap setup. Could it be some of those that breaks upgrade? (I had to change name of the gosa+samba3.schema to perform slapd.conf migration)

[Lionel Dricot (ploum-deactivatedaccount)]

Changing the title as Salik seems to report that it's not related to slapd.conf at all.

Is there any news on this issue or anything we should test ? It will be time to migrate to Karmic soon and we are still using intrepid packages.

[Mathias Gug (mathiaz)]

Please don't assign the ubuntu-server team to bugs as it doesn't help track things. Thank you.

[Lionel Dricot (ploum-deactivatedaccount)]

I've been able to eventually migrate to Jaunty slapd. As I was expecting, it was not configuration related because slaptest -u would told me that my config fil was fine. Here are the step I followed :

1) Stop slapd. Backup and remove /etc/ldap and /var/lib/ldap. With them, upgrade is impossible for reasons I don't understand.

2) upgrade packages with apt

3) stop slapd.

4) restore /etc/ldap and /var/lib/ldap

5) remove /var/lib/ldap/log.000000001 (no need to backup it)

6) cd /var/lib/ldap then run db4.6_restore -h . -v

At this point, it will tell you that 4.7 is the environement version.

7) Run once again db4.6_restore -h . -v You have to run it at least twice, else it doesn't work.

8) chown openldap:openldap log.000000001

9) Start ldap, your migration is now over.

The bugs are the following :

A) The install script of the slapd package seems completely crazy when dealing with an existing installation. It backup everything in strange ways and fails miserably if not everything is like expected. It should be more defensive.

B) slapd should be able to restore by itself a database when the log file is missing. It was the case with the slapd in intrepid, I've no idea why it's not working anymore in jaunty