Custom files collide with upstream files not packaged as config

[Don Penney]

Brief Description
-----------------
In StarlingX, we provide some customized versions of files provided by upstream packages. Generally, these are configuration files, and are packaged in the spec with %config(noreplace), so when the package is updated (ie. via system patching), the custom files are maintained. However, some of these files are not packaged as %config(noreplace), and the customizations are lost when the upstream package is patched.

Running rpm-audit can show us a list of non-config files that are modified from the original installation:

controller-0:~$ sudo rpm-audit --skip-links --skip-multi
Mismatch: /etc/inittab (initscripts-9.49.46-1.el7.tis.19.x86_64)
Mismatch: /var/log/wtmp (initscripts-9.49.46-1.el7.tis.19.x86_64)
Mismatch: /var/run/utmp (initscripts-9.49.46-1.el7.tis.19.x86_64)
Mismatch: /boot/initramfs-3.10.0-957.21.3.el7.2.tis.x86_64.img (kernel-3.10.0-957.21.3.el7.2.tis.x86_64)
Mismatch: /usr/local/etc/ldapscripts/ldapscripts.conf (ldapscripts-2.0.8-0.tis.2.x86_64)
Mismatch: /etc/rc.d/init.d/lighttpd (lighttpd-1.4.54-1.el7.tis.7.x86_64)
Mismatch: /etc/default/lldpd (lldpd-0.9.0-0.tis.4.x86_64)
Missing: /var/lib/ntp/drift (ntp-4.2.6p5-28.el7.centos.tis.1.x86_64)
Missing: /etc/openldap/slapd.conf (openldap-config-1.0-0.tis.noarch)
Mismatch: /usr/lib/systemd/system/slapd.service (openldap-servers-2.4.44-20.el7.tis.9.x86_64)
Mismatch: /var/log/lastlog (setup-2.8.71-10.el7.tis.16.noarch)
Mismatch: /etc/udev/hwdb.bin (systemd-219-62.el7_6.5.tis.11.x86_64)
Mismatch: /usr/lib/systemd/system/tmp.mount (systemd-219-62.el7_6.5.tis.11.x86_64)
Mismatch: /usr/lib/tmpfiles.d/systemd.conf (systemd-219-62.el7_6.5.tis.11.x86_64)
Mismatch: /usr/lib/tmpfiles.d/tmp.conf (systemd-219-62.el7_6.5.tis.11.x86_64)
Mismatch: /usr/lib/udev/rules.d/60-persistent-storage.rules (systemd-219-62.el7_6.5.tis.11.x86_64)
Mismatch: /var/lib/systemd/catalog/database (systemd-219-62.el7_6.5.tis.11.x86_64)
Missing: /var/lib/systemd/clock (systemd-219-62.el7_6.5.tis.11.x86_64)
Mismatch: /var/lib/systemd/random-seed (systemd-219-62.el7_6.5.tis.11.x86_64)
Mismatch: /var/log/wtmp (systemd-219-62.el7_6.5.tis.11.x86_64)
Mismatch: /var/run/utmp (systemd-219-62.el7_6.5.tis.11.x86_64)

If we update systemd on a running system, for example, we end up with critical failures on a reboot, with services like memcached and ntpd unable to start up, due to the loss of some of these customizations.

Some of these can be addressed by updating installation paths of the customized files. The systemd files, for example, can be installed under /etc, and the custom files will override the installed files from systemd. Others can be addressed by updating the packaging to use %config(noreplace).

Severity
--------
Major

Expected Behavior
------------------
Updates to packages should not overwrite customized files.

Reproducibility
---------------
Reproducible

System Configuration
--------------------
All configurations

Branch/Pull Time/Commit
-----------------------
master, as of Oct 30, 2019

Test Activity
-------------
Regression Testing

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/692217

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/692218

[Ghada Khalil (gkhalil)]

stx.3.0 / medium priority - this would impact updates (aka patching) of the impacted rpms

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/692217
Committed: https://git.openstack.org/cgit/starlingx/integ/commit/?id=b00d7ff68c4e3af9e1d3862c2e12bb90b8f3bca3
Submitter: Zuul
Branch: master

commit b00d7ff68c4e3af9e1d3862c2e12bb90b8f3bca3
Author: Don Penney <email address hidden>
Date: Wed Oct 30 16:22:55 2019 -0400

    Package /etc/default/lldpd as a config file

    The lldpd package currently does not package the /etc/default/lldpd
    file as a config file, but it is modified at runtime by a puppet
    manifest. As a result, if the lldpd package is updated on a system, it
    would overwrite the modified file with the version from the package.

    This update adds the %config(noreplace) to lldpd.spec for this file.

    Change-Id: I82e62bdcac9ea07a3eaea0dfca5b1037b4b392d6
    Partial-Bug: 1850695
    Signed-off-by: Don Penney <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/692218
Committed: https://git.openstack.org/cgit/starlingx/config-files/commit/?id=fedff3f124c975a4c343c5bad52bde2cd0795d9f
Submitter: Zuul
Branch: master

commit fedff3f124c975a4c343c5bad52bde2cd0795d9f
Author: Don Penney <email address hidden>
Date: Wed Oct 30 16:09:07 2019 -0400

    Resolve custom file packaging conflicts

    StarlingX provides custom versions of certain files that are provided
    by upstream packages. If the files are not packaged upstream as config
    files, using the %config(noreplace) spec file directive, this can
    cause a problem if the original package is updated on a running
    system, as the custom file will be overwritten.

    For certain systemd files, we can instead install these files to
    directories under /etc, which will override the original installed
    files at runtime without needing to modify the original files. This
    handles most of the conflicts addressed by this update.

    For example, instead of replacing /usr/lib/tmpfiles.d/systemd.conf in
    a postinstall scriptlet with the customized file, we can install the
    file to /etc/tmpfiles.d/systemd.conf to the same end effect.

    In the case of the customization to /etc/rc.d/init.d/lighttpd,
    however, this update addresses it by ensuring lighttpd-config copies
    the customized file whenever lighttpd-config is updated. If lighttpd
    is ever updated, updating lighttpd-config at the same time will ensure
    the customized version of the file is maintained. Other options here
    would be to update the lighttpd package to mark this script as though
    it is a config file, or update how it is called by StarlingX so that
    the custom file can be given a unique filename.

    Change-Id: Iab1e916430c1921fd8f5399166dbfd950ce1a74c
    Partial-Bug: 1850695
    Signed-off-by: Don Penney <email address hidden>