IncludeOptional fails when a directory does not exist

[Dmitrii Shcherbakov]

[Impact]

An instruction in apache config such as the following will make apache2 fail to start if the target path does not exist:

IncludeOptional /etc/apache2/mellon/sp-location*.conf

https://paste.ubuntu.com/p/f2Tb9rgrBB/ (full file: /etc/apache2/sites-enabled/openstack_https_frontend.conf)

Apr 23 06:58:31 maas-xenial1 apache2[28269]: * The apache2 configtest failed.
Apr 23 06:58:31 maas-xenial1 apache2[28269]: Output of config test was:
Apr 23 06:58:31 maas-xenial1 apache2[28269]: apache2: Syntax error on line 219 of /etc/apache2/apache2.conf: Syntax error on line 17 of /etc/apache2/sites-enabled/openstack_https_frontend.conf: Could not o
Apr 23 06:58:31 maas-xenial1 apache2[28269]: Action 'configtest' failed.
Apr 23 06:58:31 maas-xenial1 apache2[28269]: The Apache error log may have more information.
Apr 23 06:58:31 maas-xenial1 systemd[1]: apache2.service: Control process exited, code=exited status=1
Apr 23 06:58:31 maas-xenial1 systemd[1]: Failed to start LSB: Apache2 web server.

A fix is available upstream:

https://bz.apache.org/bugzilla/show_bug.cgi?id=57585
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=878920

https://github.com/apache/httpd/commit/a17ce7dd5e6277867ca48659f70c4bb8a11add56#diff-18a4bdb1520687ad43a0b4dd2b51d957R1954

[Test Case]

 * add IncludeOptional /etc/apache2/mellon/sp-location*.conf to an apache config file and try to restart the apache2 service

[Regression Potential]

 * minimal, the patch eases up requirements without rewriting or removing the existing code;
 * the patch author says that it's been used for ~ 3 years in clearlinux https://bz.apache.org/bugzilla/show_bug.cgi?id=57585#c4
 * applied in debian https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=878920#22

[Dmitrii Shcherbakov (dmitriis)]

Tried 2.4.33 from Debian sid with the fix included and a dummy path ("/etc/apache2/whatever" dir does not exist):

IncludeOptional /etc/apache2/whatever/*

stat /etc/apache2/whatever
stat: cannot stat '/etc/apache2/whatever': No such file or directory

apt policy apache2
apache2:
  Installed: 2.4.33-2
  Candidate: 2.4.33-2
  Version table:
 *** 2.4.33-2 500
        500 http://ftp.de.debian.org/debian sid/main amd64 Packages
        100 /var/lib/dpkg/status

It works properly as opposed to the version without the fix which fails with:

Apr 25 22:19:27 maas-xenial1 apache2[18520]: apache2: Syntax error on line 219 of /etc/apache2/apache2.conf: Syntax error on line 16 of /etc/apache2/sites-enabled/openstack_https_frontend.conf: Could not o

[Dmitrii Shcherbakov (dmitriis)]

At the same time including a wildcard at the end of the non-existent directory name with 2.4.18-2ubuntu3.8 makes apache2 happy:

IncludeOptional /etc/apache2/whatever*/*

apt policy apache2
apache2:
  Installed: 2.4.18-2ubuntu3.8
  Candidate: 2.4.18-2ubuntu3.8

[Andreas Hasenack (ahasenack)]

PPA with test packages: ppa:ahasenack/apache-includeoptional-1766186 (https://launchpad.net/~ahasenack/+archive/ubuntu/apache-includeoptional-1766186/)

[Andreas Hasenack (ahasenack)]

Packages uploaded to proposed unapproved.

[Robie Basak (racb)]

I suggest that the negative case also be checked during SRU verification: please also verify that if a file is included by IncludeOptional with this syntax that it is actually being included rather than being silently ignored.

[Robie Basak (racb)]

Hello Dmitrii, or anyone else affected,

Accepted apache2 into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apache2/2.4.29-1ubuntu4.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Robie Basak (racb)]

Hello Dmitrii, or anyone else affected,

Accepted apache2 into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apache2/2.4.27-2ubuntu4.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Robie Basak (racb)]

Hello Dmitrii, or anyone else affected,

Accepted apache2 into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apache2/2.4.18-2ubuntu3.9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Dmitrii Shcherbakov (dmitriis)]

Checked with xenial-proposed:

root@maas-vhost2:/etc/apache2# apt policy apache2
apache2:
  Installed: 2.4.18-2ubuntu3.9
  Candidate: 2.4.18-2ubuntu3.9
  Version table:
 *** 2.4.18-2ubuntu3.9 500
        500 http://archive.ubuntu.com/ubuntu xenial-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     2.4.18-2ubuntu3.8 500
        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        500 http://archive.ubuntu.com/ubuntu xenial-security/main amd64 Packages
     2.4.18-2ubuntu3 500
        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages

grep IncludeOptional sites-enabled/openstack_https_frontend.conf
    IncludeOptional /etc/apache2/mellon/sp-location*.conf
    IncludeOptional /etc/apache2/mellon/sp-location*.conf

stat mellon/sp-location.keystone-saml-mellon.conf
  File: 'mellon/sp-location.keystone-saml-mellon.conf'
# ...

systemctl restart apache2 && sleep 10 && systemctl status apache2 | grep Active
   Active: active (running) since Fri 2018-06-22 22:17:04 UTC; 10s ago

mv mellon/sp-location.keystone-saml-mellon.conf /tmp/

systemctl restart apache2 && sleep 5 && systemctl status apache2 | grep Active
   Active: active (running) since Fri 2018-06-22 22:19:02 UTC; 5s ago

mv /tmp/sp-location.keystone-saml-mellon.conf mellon/

echo deadbeef >> mellon/sp-location.keystone-saml-mellon.conf

# looks like it actually gets imported which is what we are after
systemctl restart apache2 && sleep 5 && systemctl status apache2 | grep Active
Job for apache2.service failed because the control process exited with error code. See "systemctl status apache2.service" and "journalctl -xe" for details.

[Dmitrii Shcherbakov (dmitriis)]

artful-proposed:

apt policy apache2
apache2:
  Installed: 2.4.27-2ubuntu4.2
  Candidate: 2.4.27-2ubuntu4.2
  Version table:
 *** 2.4.27-2ubuntu4.2 500
        500 http://archive.ubuntu.com/ubuntu artful-proposed/main amd64 Packages
        100 /var/lib/dpkg/status

grep IncludeOptional sites-enabled/openstack_https_frontend.conf
    IncludeOptional /etc/apache2/mellon/sp-location*.conf
    IncludeOptional /etc/apache2/mellon/sp-location*.conf

stat mellon/sp-location.keystone-saml-mellon.conf
  File: 'mellon/sp-location.keystone-saml-mellon.conf'
# ...

systemctl restart apache2 && sleep 10 && systemctl status apache2 | grep Active
   Active: active (running) since Fri 2018-06-22 22:26:15 UTC; 10s ago

mv mellon/sp-location.keystone-saml-mellon.conf /tmp/

systemctl restart apache2 && sleep 5 && systemctl status apache2 | grep Active
   Active: active (running) since Fri 2018-06-22 22:27:59 UTC; 5s ago

systemctl restart apache2 && sleep 5 && systemctl status apache2 | grep Active
Job for apache2.service failed because the control process exited with error code. See "systemctl status apache2.service" and "journalctl -xe" for details.

[Dmitrii Shcherbakov (dmitriis)]

artful-proposed (added missing statements):

apt policy apache2
apache2:
  Installed: 2.4.27-2ubuntu4.2
  Candidate: 2.4.27-2ubuntu4.2
  Version table:
 *** 2.4.27-2ubuntu4.2 500
        500 http://archive.ubuntu.com/ubuntu artful-proposed/main amd64 Packages
        100 /var/lib/dpkg/status

grep IncludeOptional sites-enabled/openstack_https_frontend.conf
    IncludeOptional /etc/apache2/mellon/sp-location*.conf
    IncludeOptional /etc/apache2/mellon/sp-location*.conf

stat mellon/sp-location.keystone-saml-mellon.conf
  File: 'mellon/sp-location.keystone-saml-mellon.conf'
# ...

systemctl restart apache2 && sleep 10 && systemctl status apache2 | grep Active
   Active: active (running) since Fri 2018-06-22 22:26:15 UTC; 10s ago

mv mellon/sp-location.keystone-saml-mellon.conf /tmp/

systemctl restart apache2 && sleep 5 && systemctl status apache2 | grep Active
   Active: active (running) since Fri 2018-06-22 22:27:59 UTC; 5s ago

mv /tmp/sp-location.keystone-saml-mellon.conf mellon/

echo deadbeef >> mellon/sp-location.keystone-saml-mellon.conf

systemctl restart apache2 && sleep 5 && systemctl status apache2 | grep Active
Job for apache2.service failed because the control process exited with error code. See "systemctl status apache2.service" and "journalctl -xe" for details.

[Dmitrii Shcherbakov (dmitriis)]

bionic-proposed:

apt policy apache2
apache2:
  Installed: 2.4.29-1ubuntu4.2
  Candidate: 2.4.29-1ubuntu4.2
  Version table:
 *** 2.4.29-1ubuntu4.2 500
        500 http://archive.ubuntu.com/ubuntu bionic-proposed/main amd64 Packages
        100 /var/lib/dpkg/status

grep IncludeOptional sites-enabled/openstack_https_frontend.conf
    IncludeOptional /etc/apache2/mellon/sp-location*.conf
    IncludeOptional /etc/apache2/mellon/sp-location*.conf

stat mellon/sp-location.keystone-saml-mellon.conf
  File: 'mellon/sp-location.keystone-saml-mellon.conf'

systemctl restart apache2 && sleep 10 && systemctl status apache2 | grep Active
   Active: active (running) since Fri 2018-06-22 22:32:18 UTC; 10s ago

mv mellon/sp-location.keystone-saml-mellon.conf /tmp/

systemctl restart apache2 && sleep 5 && systemctl status apache2 | grep Active
   Active: active (running) since Fri 2018-06-22 22:32:51 UTC; 5s ago

mv /tmp/sp-location.keystone-saml-mellon.conf mellon/

echo deadbeef >> mellon/sp-location.keystone-saml-mellon.conf

systemctl restart apache2 && sleep 5 && systemctl status apache2 | grep Active
Job for apache2.service failed because the control process exited with error code. See "systemctl status apache2.service" and "journalctl -xe" for details.

[Andreas Hasenack (ahasenack)]

I'm investigating the DEP8 errors.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apache2 - 2.4.29-1ubuntu4.2

---------------
apache2 (2.4.29-1ubuntu4.2) bionic; urgency=medium

  * debian/patches/includeoptional-ignore-non-existent.patch: silently
    ignore a not existent file path with IncludeOptional . Closes LP:
    #1766186.

 -- Andreas Hasenack <email address hidden> Thu, 07 Jun 2018 18:10:10 -0300

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for apache2 has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Andreas Hasenack (ahasenack)]

Artful:
- the cacti regressions are due to a mariadb regression: https://bugs.launchpad.net/ubuntu/+source/mariadb-10.1/+bug/1757107
After being installed, mariadb isn't running, and thus the cacti postinst can't configure a database and cacti fails to work.
- resource-agents on s390x: will investigate next

[Andreas Hasenack (ahasenack)]

Artful:
- the resource-agents failures (not just in s390) were fixed in https://bugs.launchpad.net/ubuntu/+source/resource-agents/+bug/1734148 for bionic. Since Artful is EOL soon, probably not worth an SRU.

[Andreas Hasenack (ahasenack)]

TL;DR artful:
- cacti regressions are due to mariadb-server regression bug #1757107. After being installed, mariadb isn't running. I applied this crude workaround (https://pastebin.ubuntu.com/p/TwJBtJq96J/) and ran the tests locally, and they passed: http://people.ubuntu.com/~ahasenack/artful-dep8-cacti-workaround/
Note that the workaround doesn't work if what you want to test is a new upload of cacti itself, since I purge and reinstall it. But this SRU is about apache, so it should be enough.
- resource-agents failures were fixed for bionic in bug #1734148 and are probably not worth an SRU since artful is EOL in a few days.

There are no other artful failures.

Therefore, I ask to let this SRU continue for Artful.

I'll now investigate the xenial DEP8 failures.

[Andreas Hasenack (ahasenack)]

Xenial:

a) libapache2-mod-perl2
two test errors: t/apache/read.t and t/filter/in_bbs_inject_header.t. Both are fixed in 2.0.10-2 in debian:
libapache2-mod-perl2 (2.0.10-2) unstable; urgency=medium

  * Patch the test suite for Apache 2.4.24 compatibility.
    Thanks to Stefan Fritsch. (Closes: #849082)

 -- Niko Tyni <email address hidden> Sun, 25 Dec 2016 11:51:10 +0200

This is the diff:
https://git.launchpad.net/ubuntu/+source/libapache2-mod-perl2/commit/?h=debian/buster&id=1a0e6c046960f816f0089d8d164f7fa9b077719f

I filed a bug and attached a branch to it: https://bugs.launchpad.net/ubuntu/+source/libapache2-mod-perl2/+bug/1779400

Here is the MP: https://code.launchpad.net/~ahasenack/ubuntu/+source/libapache2-mod-perl2/+git/libapache2-mod-perl2/+merge/348780

PPA with test packages: https://launchpad.net/~ahasenack/+archive/ubuntu/libapache2-mod-perl2-dep8/+packages

DEP8 run with the PPA packages: http://people.ubuntu.com/~ahasenack/deb8-modperl2-xenial-ppa/

b) gvfs
This is an "old friend". The gvfs tests are very bad, just see the history in http://autopkgtest.ubuntu.com/packages/g/gvfs/xenial/s390x. I'll give it a few retries.

[Andreas Hasenack (ahasenack)]

Xenial tldr:

a) libapache2-mod-perl2 failing across the board. Addressed in bug #1779400 (SRU), which is in review. That bug also has a sample dep8 run with the fixed packages, showing they pass with this apache from proposed.

b) gvfs
flaky tests, with a big tendency to fail, finally passed.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apache2 - 2.4.18-2ubuntu3.9

---------------
apache2 (2.4.18-2ubuntu3.9) xenial; urgency=medium

  * debian/patches/includeoptional-ignore-non-existent.patch: silently
    ignore a not existent file path with IncludeOptional . Closes LP:
    #1766186.

 -- Andreas Hasenack <email address hidden> Thu, 07 Jun 2018 16:43:03 -0300

[Launchpad Janitor (janitor)]

This bug was fixed in the package apache2 - 2.4.27-2ubuntu4.2

---------------
apache2 (2.4.27-2ubuntu4.2) artful; urgency=medium

  * debian/patches/includeoptional-ignore-non-existent.patch: silently
    ignore a not existent file path with IncludeOptional . Closes LP:
    #1766186.

 -- Andreas Hasenack <email address hidden> Thu, 07 Jun 2018 17:53:23 -0300