[1.8rc1] Too many open files, after upgrade to rc1

[Andres Rodriguez]

When upgrading a MAAS 1.8 beta version against the RC, the following error has been seen in the logs.

un 2 06:29:07 maas maas.lease_upload_service: [ERROR] Failed to upload leases: [Errno 24] Too many open files: '/tmp/tmpM0jQ8Y'
Jun 2 06:29:07 maas maas.service_monitor: [ERROR] While monitoring service 'maas-dhcpd' an error was encountered: [Errno 24] Too many open files
Jun 2 06:29:07 maas maas.service_monitor: [ERROR] While monitoring service 'maas-dhcpd6' an error was encountered: [Errno 24] Too many open files
Jun 2 06:29:07 maas maas.service_monitor: [ERROR] While monitoring service 'tgt' an error was encountered: [Errno 24] Too many open files
Jun 2 06:29:07 maas maas.dhcp.probe: [ERROR] Failed to probe sockets; did you configure authbind as per HACKING.txt?
Jun 2 06:29:08 maas maas.import-images: [INFO] Started importing boot images.
Jun 2 06:29:08 maas maas.boot_image_download_service: [ERROR] Failed to download images: [Errno 24] Too many open files: u'/tmp/maas-BcxaR6keyrings/192.168.9.2-MAAS-images-stream-streams-v1-index.json.gpg'
Jun 2 06:29:08 maas maas.power: [ERROR] maas-1-01.cluster.mallards: Failed to refresh power state: [Errno 24] Too many open files: u'/etc/maas/templates/power/ipmi.template'
@

[Andres Rodriguez (andreserl)]

13:42 < sabdfl> i did the upgrade
13:42 < sabdfl> and noticed that it was saying it had no boot image source
13:42 < sabdfl> so i went to settings
13:42 < sabdfl> there was the same usual text there
13:42 < sabdfl> i just pressed save
13:42 < sabdfl> and then it had the usual map of releases / architectures on the images page
13:42 < sabdfl> and downloaded images just fine

[Scott Moser (smoser)]

I think this can be fixed without the monkey patching with:
--- src/maasserver/bootresources.py 2015-06-02 08:46:21 +0000
+++ src/maasserver/bootresources.py 2015-06-27 01:17:41 +0000
@@ -666,6 +666,7 @@
                 cksummer.update(buf)
                 if len(buf) != self.read_size:
                     break
+ reader.close()

         if not cksummer.check():
             # Calculated sha256 hash from the data does not match, what

[Scott Moser (smoser)]

I'd like to see some test / result.
I did verify that simplestreams 'insert_item' does not call reader.close() for you.
the thing that is strange is that this reader has an 'open()' but you don't have to (and don't) close it.
so its kind of odd to expect you to close it.

Perhaps it looks more conventional like this:
--- src/maasserver/bootresources.py 2015-06-02 08:46:21 +0000
+++ src/maasserver/bootresources.py 2015-06-27 01:20:47 +0000
@@ -660,12 +660,14 @@
         # Write the contents into the database, while calculating the sha256
         # hash for the read data.
         with rfile.largefile.content.open('wb') as stream:
+ reader.open()
             while True:
                 buf = reader.read(self.read_size)
                 stream.write(buf)
                 cksummer.update(buf)
                 if len(buf) != self.read_size:
                     break
+ reader.close()

         if not cksummer.check():
             # Calculated sha256 hash from the data does not match, what

[Andres Rodriguez (andreserl)]

http://paste.ubuntu.com/11781385/