| 2016-09-08 21:09:37 |
AtesComp |
bug |
|
|
added bug |
| 2016-09-08 21:27:58 |
AtesComp |
description |
A persistent bug exists in the update notifier that affects the ttf-mscorefonts-installer and may also affect other packages. On boot, a user is presented with an update notification indicating that the ttf-mscorefonts-installer failed to install and needs installation:
---snip---
The following packages requested additional data downloads after package
installation, but the data could not be downloaded or could not be
processed.
ttf-mscorefonts-installer
The download will be attempted again later, or you can try the download
again now. Running this command requires an active Internet connection.
---snip---
Running the command fails. Manual installation of the ttf-mscorefonts-installer does not correct the notification. The issue was traced to the /usr/lib/update-notifier/package-data-downloader script.
There are several issues in the package-data-downloader script:
1. The function process_download_requests() checks the hook date against the stamp date for a installation package, such as
Hook File: /usr/share/package-data-downloads/ttf-mscorefonts-installer
Stamp File: /var/lib/update-notifier/package-data-downloads/ttf-mscorefonts-installer
with the code:
---snip---
try:
hook_date = os.stat(file).st_mtime
stamp_date = os.stat(stampfile).st_mtime
if hook_date < stamp_date:
continue
---snip---
The stamp file exist if there was a previous installation that did not produce a permanent failure. However, a normal failure sets up the condition to run this script. The failure condition is not cleared on success. The code related to determining success and failure is:
---snip---
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
---snip---
Therefore, in the event of a non-permanent failure, the stamp file is created or updated which prevents an followup download to correct the non-permanent failure.
2. The function process_download_requests() also tests for keywords 'script', 'should-download', 'url', and 'sha256' in the para list from the Hook file to determine action:
---snip---
for para in hook.iter_paragraphs(open(file)):
* if 'script' in para:
if not files:
record_failure(relfile)
break
* command = [para['script']]
* if 'should-download' in para:
db = debconf.DebconfCommunicator('update-notifier')
try:
* should = db.get(para['should-download'])
if should == "false":
# Do nothing with this file.
break
except:
pass
finally:
db.shutdown()
# Download each file and verify the sum
try:
downloaded = set()
for i in range(len(files)):
print("%s: downloading %s" % (relfile, files[i]))
dest_file = download_file(files[i], sums[i])
if dest_file:
command.append(dest_file)
downloaded.add(dest_file)
else:
record_failure(relfile)
break
if relfile in failures + permanent_failures:
break
sys.stdout.flush()
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
# cleanup
for f in downloaded:
os.remove(f)
break
except Exception:
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
# The 'script' is always the last stanza
break
# Not in a 'script' stanza, so we should have some urls
try:
* files.append(para['url'])
* sums.append(para['sha256'])
except Exception as e:
if not isinstance(e, KeyError):
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
break
---snip---
where * note the lines affected. These keywords begin with an uppercase in the related files:
---snip---
Url: http://downloads.sourceforge.net/corefonts/webdin32.exe
Sha256: 64595b5abc1080fba8610c5c34fab5863408e806aafe84653ca8575bed17d75a
Script: /usr/lib/msttcorefonts/update-ms-fonts
Should-Download: msttcorefonts/accepted-mscorefonts-eula
---snip---
3. The function process_download_requests() tracks so called "failures" and "permanent_failures". When a failure occurs, the following code sets up the condition for reprocessing later using the function trigger_update_notifier():
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
---snip---
SOLUTION:
====================
The existing code set up several conditions based on the existence of the following files:
stamp file
notifier file
notifier permanent file
The conditions are:
EXISTS ( stamp file )
EXISTS ( stamp file AND notifier file )
EXISTS ( notifier file AND notifier permanent file )
EXISTS ( notifier permanent file )
When no permanent failures are found (no notifier permanent file), the stamp file is set even when normal failures exist (notifier file). This causes the next script execution to skip to the next install item. In the event of success, failure conditions are cleared, but the notifier files are never cleared causing persistent requests to update that never get fulfilled.
For part 1, adding code handle the failure condition when a stamp file exists would be prudent:
---snip---
try:
> if not os.path.exists(NOTIFIER_FILE) and not os.path.exists(NOTIFIER_PERMANENT_FILE):
>>>> hook_date = os.stat(file).st_mtime
>>>> stamp_date = os.stat(stampfile).st_mtime
>>>> if hook_date < stamp_date:
>>>> continue
> elif os.path.exists(stampfile):
> os.unlink(stampfile)
---snip---
where > indicated the added/modded code. This corrects the problem by allowing any perceived failures, permanent or not, to reinstall the downloads. It always removes the stampfile so that if a permanent failure develops, it is handled correctly later.
For part 2, the paragraph keyword lookups may or may not care about case, but update the keywords:
'script', 'should-download', 'url', 'sha256'
to:
'Script', 'Should-Download', 'Url', 'Sha256'
so that they match the file.
For part 3, the comment suggests that the so called "update-notifier file" will be clobbered, but it never is. On a non-permament failure, the failed and permanent-failure marker files are removed when function create_or_update_stampfile() is called. The code should be changed to handle cleared failure conditions (i.e., empty failure lists) as follows:
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
> if not previous_failures and os.path.exists(NOTIFIER_PERMANENT_FILE):
> os.unlink(NOTIFIER_PERMANENT_FILE)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
> elif os.path.exists(NOTIFIER_FILE):
> os.unlink(NOTIFIER_FILE)
---snip---
where > indicated the added code.
In total, when no failure is found, the script will create the stamp file and clear the notifier files. For permanent failures, the stamp file is not created and any notifier files will persist until the next attempted update. For normal failures, the stamp and non-permanent notifier file persists until the next attempted update. On any update, if notifier files exist, the script will attempt to correct the install. If just the stamp file exists, then that install is skipped. |
A persistent bug exists in the update notifier that affects the ttf-mscorefonts-installer and may also affect other packages. On boot, a user is presented with an update notification indicating that the ttf-mscorefonts-installer failed to install and needs installation:
---snip---
The following packages requested additional data downloads after package
installation, but the data could not be downloaded or could not be
processed.
ttf-mscorefonts-installer
The download will be attempted again later, or you can try the download
again now. Running this command requires an active Internet connection.
---snip---
Running the command fails. Manual installation of the ttf-mscorefonts-installer does not correct the notification. The issue was traced to the /usr/lib/update-notifier/package-data-downloader script.
There are several issues in the package-data-downloader script:
1. The function process_download_requests() checks the hook date against the stamp date for a installation package, such as
Hook File: /usr/share/package-data-downloads/ttf-mscorefonts-installer
Stamp File: /var/lib/update-notifier/package-data-downloads/ttf-mscorefonts-installer
with the code:
---snip---
try:
hook_date = os.stat(file).st_mtime
stamp_date = os.stat(stampfile).st_mtime
if hook_date < stamp_date:
continue
---snip---
The stamp file exist if there was a previous installation that did not produce a permanent failure. However, a normal failure sets up the condition to run this script. The failure condition is not cleared on success. The code related to determining success and failure is:
---snip---
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
---snip---
Therefore, in the event of a non-permanent failure, the stamp file is created or updated which prevents an followup download to correct the non-permanent failure.
2. The function process_download_requests() also tests for keywords 'script', 'should-download', 'url', and 'sha256' in the para list from the Hook file to determine action:
<code>
for para in hook.iter_paragraphs(open(file)):
* if 'script' in para:
if not files:
record_failure(relfile)
break
* command = [para['script']]
* if 'should-download' in para:
db = debconf.DebconfCommunicator('update-notifier')
try:
* should = db.get(para['should-download'])
if should == "false":
# Do nothing with this file.
break
except:
pass
finally:
db.shutdown()
# Download each file and verify the sum
try:
downloaded = set()
for i in range(len(files)):
print("%s: downloading %s" % (relfile, files[i]))
dest_file = download_file(files[i], sums[i])
if dest_file:
command.append(dest_file)
downloaded.add(dest_file)
else:
record_failure(relfile)
break
if relfile in failures + permanent_failures:
break
sys.stdout.flush()
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
# cleanup
for f in downloaded:
os.remove(f)
break
except Exception:
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
# The 'script' is always the last stanza
break
# Not in a 'script' stanza, so we should have some urls
try:
* files.append(para['url'])
* sums.append(para['sha256'])
except Exception as e:
if not isinstance(e, KeyError):
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
break
</code>
where * note the lines affected. These keywords begin with an uppercase in the related files:
---snip---
Url: http://downloads.sourceforge.net/corefonts/webdin32.exe
Sha256: 64595b5abc1080fba8610c5c34fab5863408e806aafe84653ca8575bed17d75a
Script: /usr/lib/msttcorefonts/update-ms-fonts
Should-Download: msttcorefonts/accepted-mscorefonts-eula
---snip---
3. The function process_download_requests() tracks so called "failures" and "permanent_failures". When a failure occurs, the following code sets up the condition for reprocessing later using the function trigger_update_notifier():
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
---snip---
SOLUTION:
====================
The existing code set up several conditions based on the existence of the following files:
stamp file
notifier file
notifier permanent file
The conditions are:
EXISTS ( stamp file )
EXISTS ( stamp file AND notifier file )
EXISTS ( notifier file AND notifier permanent file )
EXISTS ( notifier permanent file )
When no permanent failures are found (no notifier permanent file), the stamp file is set even when normal failures exist (notifier file). This causes the next script execution to skip to the next install item. In the event of success, failure conditions are cleared, but the notifier files are never cleared causing persistent requests to update that never get fulfilled.
For part 1, adding code handle the failure condition when a stamp file exists would be prudent:
---snip---
try:
> if not os.path.exists(NOTIFIER_FILE) and not os.path.exists(NOTIFIER_PERMANENT_FILE):
>>>> hook_date = os.stat(file).st_mtime
>>>> stamp_date = os.stat(stampfile).st_mtime
>>>> if hook_date < stamp_date:
>>>> continue
> elif os.path.exists(stampfile):
> os.unlink(stampfile)
---snip---
where > indicated the added/modded code. This corrects the problem by allowing any perceived failures, permanent or not, to reinstall the downloads. It always removes the stampfile so that if a permanent failure develops, it is handled correctly later.
For part 2, the paragraph keyword lookups may or may not care about case, but update the keywords:
'script', 'should-download', 'url', 'sha256'
to:
'Script', 'Should-Download', 'Url', 'Sha256'
so that they match the file.
For part 3, the comment suggests that the so called "update-notifier file" will be clobbered, but it never is. On a non-permament failure, the failed and permanent-failure marker files are removed when function create_or_update_stampfile() is called. The code should be changed to handle cleared failure conditions (i.e., empty failure lists) as follows:
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
> if not previous_failures and os.path.exists(NOTIFIER_PERMANENT_FILE):
> os.unlink(NOTIFIER_PERMANENT_FILE)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
> elif os.path.exists(NOTIFIER_FILE):
> os.unlink(NOTIFIER_FILE)
---snip---
where > indicated the added code.
In total, when no failure is found, the script will create the stamp file and clear the notifier files. For permanent failures, the stamp file is not created and any notifier files will persist until the next attempted update. For normal failures, the stamp and non-permanent notifier file persists until the next attempted update. On any update, if notifier files exist, the script will attempt to correct the install. If just the stamp file exists, then that install is skipped. |
|
| 2016-09-08 22:20:17 |
AtesComp |
description |
A persistent bug exists in the update notifier that affects the ttf-mscorefonts-installer and may also affect other packages. On boot, a user is presented with an update notification indicating that the ttf-mscorefonts-installer failed to install and needs installation:
---snip---
The following packages requested additional data downloads after package
installation, but the data could not be downloaded or could not be
processed.
ttf-mscorefonts-installer
The download will be attempted again later, or you can try the download
again now. Running this command requires an active Internet connection.
---snip---
Running the command fails. Manual installation of the ttf-mscorefonts-installer does not correct the notification. The issue was traced to the /usr/lib/update-notifier/package-data-downloader script.
There are several issues in the package-data-downloader script:
1. The function process_download_requests() checks the hook date against the stamp date for a installation package, such as
Hook File: /usr/share/package-data-downloads/ttf-mscorefonts-installer
Stamp File: /var/lib/update-notifier/package-data-downloads/ttf-mscorefonts-installer
with the code:
---snip---
try:
hook_date = os.stat(file).st_mtime
stamp_date = os.stat(stampfile).st_mtime
if hook_date < stamp_date:
continue
---snip---
The stamp file exist if there was a previous installation that did not produce a permanent failure. However, a normal failure sets up the condition to run this script. The failure condition is not cleared on success. The code related to determining success and failure is:
---snip---
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
---snip---
Therefore, in the event of a non-permanent failure, the stamp file is created or updated which prevents an followup download to correct the non-permanent failure.
2. The function process_download_requests() also tests for keywords 'script', 'should-download', 'url', and 'sha256' in the para list from the Hook file to determine action:
<code>
for para in hook.iter_paragraphs(open(file)):
* if 'script' in para:
if not files:
record_failure(relfile)
break
* command = [para['script']]
* if 'should-download' in para:
db = debconf.DebconfCommunicator('update-notifier')
try:
* should = db.get(para['should-download'])
if should == "false":
# Do nothing with this file.
break
except:
pass
finally:
db.shutdown()
# Download each file and verify the sum
try:
downloaded = set()
for i in range(len(files)):
print("%s: downloading %s" % (relfile, files[i]))
dest_file = download_file(files[i], sums[i])
if dest_file:
command.append(dest_file)
downloaded.add(dest_file)
else:
record_failure(relfile)
break
if relfile in failures + permanent_failures:
break
sys.stdout.flush()
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
# cleanup
for f in downloaded:
os.remove(f)
break
except Exception:
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
# The 'script' is always the last stanza
break
# Not in a 'script' stanza, so we should have some urls
try:
* files.append(para['url'])
* sums.append(para['sha256'])
except Exception as e:
if not isinstance(e, KeyError):
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
break
</code>
where * note the lines affected. These keywords begin with an uppercase in the related files:
---snip---
Url: http://downloads.sourceforge.net/corefonts/webdin32.exe
Sha256: 64595b5abc1080fba8610c5c34fab5863408e806aafe84653ca8575bed17d75a
Script: /usr/lib/msttcorefonts/update-ms-fonts
Should-Download: msttcorefonts/accepted-mscorefonts-eula
---snip---
3. The function process_download_requests() tracks so called "failures" and "permanent_failures". When a failure occurs, the following code sets up the condition for reprocessing later using the function trigger_update_notifier():
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
---snip---
SOLUTION:
====================
The existing code set up several conditions based on the existence of the following files:
stamp file
notifier file
notifier permanent file
The conditions are:
EXISTS ( stamp file )
EXISTS ( stamp file AND notifier file )
EXISTS ( notifier file AND notifier permanent file )
EXISTS ( notifier permanent file )
When no permanent failures are found (no notifier permanent file), the stamp file is set even when normal failures exist (notifier file). This causes the next script execution to skip to the next install item. In the event of success, failure conditions are cleared, but the notifier files are never cleared causing persistent requests to update that never get fulfilled.
For part 1, adding code handle the failure condition when a stamp file exists would be prudent:
---snip---
try:
> if not os.path.exists(NOTIFIER_FILE) and not os.path.exists(NOTIFIER_PERMANENT_FILE):
>>>> hook_date = os.stat(file).st_mtime
>>>> stamp_date = os.stat(stampfile).st_mtime
>>>> if hook_date < stamp_date:
>>>> continue
> elif os.path.exists(stampfile):
> os.unlink(stampfile)
---snip---
where > indicated the added/modded code. This corrects the problem by allowing any perceived failures, permanent or not, to reinstall the downloads. It always removes the stampfile so that if a permanent failure develops, it is handled correctly later.
For part 2, the paragraph keyword lookups may or may not care about case, but update the keywords:
'script', 'should-download', 'url', 'sha256'
to:
'Script', 'Should-Download', 'Url', 'Sha256'
so that they match the file.
For part 3, the comment suggests that the so called "update-notifier file" will be clobbered, but it never is. On a non-permament failure, the failed and permanent-failure marker files are removed when function create_or_update_stampfile() is called. The code should be changed to handle cleared failure conditions (i.e., empty failure lists) as follows:
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
> if not previous_failures and os.path.exists(NOTIFIER_PERMANENT_FILE):
> os.unlink(NOTIFIER_PERMANENT_FILE)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
> elif os.path.exists(NOTIFIER_FILE):
> os.unlink(NOTIFIER_FILE)
---snip---
where > indicated the added code.
In total, when no failure is found, the script will create the stamp file and clear the notifier files. For permanent failures, the stamp file is not created and any notifier files will persist until the next attempted update. For normal failures, the stamp and non-permanent notifier file persists until the next attempted update. On any update, if notifier files exist, the script will attempt to correct the install. If just the stamp file exists, then that install is skipped. |
A persistent bug exists in the update notifier that affects the ttf-mscorefonts-installer and may also affect other packages. On boot, a user is presented with an update notification indicating that the ttf-mscorefonts-installer failed to install and needs installation:
---snip---
The following packages requested additional data downloads after package
installation, but the data could not be downloaded or could not be
processed.
ttf-mscorefonts-installer
The download will be attempted again later, or you can try the download
again now. Running this command requires an active Internet connection.
---snip---
Running the command fails. Manual installation of the ttf-mscorefonts-installer does not correct the notification. The issue was traced to the /usr/lib/update-notifier/package-data-downloader script.
There are several issues in the package-data-downloader script:
1. The function process_download_requests() checks the hook date against the stamp date for a installation package, such as
Hook File: /usr/share/package-data-downloads/ttf-mscorefonts-installer
Stamp File: /var/lib/update-notifier/package-data-downloads/ttf-mscorefonts-installer
with the code:
---snip---
try:
hook_date = os.stat(file).st_mtime
stamp_date = os.stat(stampfile).st_mtime
if hook_date < stamp_date:
continue
---snip---
The stamp file exist if there was a previous installation that did not produce a permanent failure. However, a normal failure sets up the condition to run this script. The failure condition is not cleared on success. The code related to determining success and failure is:
---snip---
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
---snip---
Therefore, in the event of a non-permanent failure, the stamp file is created or updated which prevents an followup download to correct the non-permanent failure.
2. The function process_download_requests() also tests for keywords 'script', 'should-download', 'url', and 'sha256' in the para list from the Hook file to determine action:
<code>
for para in hook.iter_paragraphs(open(file)):
*...........if 'script' in para:
if not files:
record_failure(relfile)
break
* command = [para['script']]
* if 'should-download' in para:
db = debconf.DebconfCommunicator('update-notifier')
try:
* should = db.get(para['should-download'])
if should == "false":
# Do nothing with this file.
break
except:
pass
finally:
db.shutdown()
# Download each file and verify the sum
try:
downloaded = set()
for i in range(len(files)):
print("%s: downloading %s" % (relfile, files[i]))
dest_file = download_file(files[i], sums[i])
if dest_file:
command.append(dest_file)
downloaded.add(dest_file)
else:
record_failure(relfile)
break
if relfile in failures + permanent_failures:
break
sys.stdout.flush()
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
# cleanup
for f in downloaded:
os.remove(f)
break
except Exception:
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
# The 'script' is always the last stanza
break
# Not in a 'script' stanza, so we should have some urls
try:
* files.append(para['url'])
* sums.append(para['sha256'])
except Exception as e:
if not isinstance(e, KeyError):
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
break
<code>
where * note the lines affected. These keywords begin with an uppercase in the related files:
---snip---
Url: http://downloads.sourceforge.net/corefonts/webdin32.exe
Sha256: 64595b5abc1080fba8610c5c34fab5863408e806aafe84653ca8575bed17d75a
Script: /usr/lib/msttcorefonts/update-ms-fonts
Should-Download: msttcorefonts/accepted-mscorefonts-eula
---snip---
3. The function process_download_requests() tracks so called "failures" and "permanent_failures". When a failure occurs, the following code sets up the condition for reprocessing later using the function trigger_update_notifier():
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
---snip---
SOLUTION:
====================
The existing code set up several conditions based on the existence of the following files:
stamp file
notifier file
notifier permanent file
The conditions are:
EXISTS ( stamp file )
EXISTS ( stamp file AND notifier file )
EXISTS ( notifier file AND notifier permanent file )
EXISTS ( notifier permanent file )
When no permanent failures are found (no notifier permanent file), the stamp file is set even when normal failures exist (notifier file). This causes the next script execution to skip to the next install item. In the event of success, failure conditions are cleared, but the notifier files are never cleared causing persistent requests to update that never get fulfilled.
For part 1, adding code handle the failure condition when a stamp file exists would be prudent:
---snip---
try:
> if not os.path.exists(NOTIFIER_FILE) and not os.path.exists(NOTIFIER_PERMANENT_FILE):
>>>> hook_date = os.stat(file).st_mtime
>>>> stamp_date = os.stat(stampfile).st_mtime
>>>> if hook_date < stamp_date:
>>>> continue
> elif os.path.exists(stampfile):
> os.unlink(stampfile)
---snip---
where > indicated the added/modded code. This corrects the problem by allowing any perceived failures, permanent or not, to reinstall the downloads. It always removes the stampfile so that if a permanent failure develops, it is handled correctly later.
For part 2, the paragraph keyword lookups may or may not care about case, but update the keywords:
'script', 'should-download', 'url', 'sha256'
to:
'Script', 'Should-Download', 'Url', 'Sha256'
so that they match the file.
For part 3, the comment suggests that the so called "update-notifier file" will be clobbered, but it never is. On a non-permament failure, the failed and permanent-failure marker files are removed when function create_or_update_stampfile() is called. The code should be changed to handle cleared failure conditions (i.e., empty failure lists) as follows:
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
> if not previous_failures and os.path.exists(NOTIFIER_PERMANENT_FILE):
> os.unlink(NOTIFIER_PERMANENT_FILE)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
> elif os.path.exists(NOTIFIER_FILE):
> os.unlink(NOTIFIER_FILE)
---snip---
where > indicated the added code.
In total, when no failure is found, the script will create the stamp file and clear the notifier files. For permanent failures, the stamp file is not created and any notifier files will persist until the next attempted update. For normal failures, the stamp and non-permanent notifier file persists until the next attempted update. On any update, if notifier files exist, the script will attempt to correct the install. If just the stamp file exists, then that install is skipped. |
|
| 2016-09-08 22:20:52 |
AtesComp |
description |
A persistent bug exists in the update notifier that affects the ttf-mscorefonts-installer and may also affect other packages. On boot, a user is presented with an update notification indicating that the ttf-mscorefonts-installer failed to install and needs installation:
---snip---
The following packages requested additional data downloads after package
installation, but the data could not be downloaded or could not be
processed.
ttf-mscorefonts-installer
The download will be attempted again later, or you can try the download
again now. Running this command requires an active Internet connection.
---snip---
Running the command fails. Manual installation of the ttf-mscorefonts-installer does not correct the notification. The issue was traced to the /usr/lib/update-notifier/package-data-downloader script.
There are several issues in the package-data-downloader script:
1. The function process_download_requests() checks the hook date against the stamp date for a installation package, such as
Hook File: /usr/share/package-data-downloads/ttf-mscorefonts-installer
Stamp File: /var/lib/update-notifier/package-data-downloads/ttf-mscorefonts-installer
with the code:
---snip---
try:
hook_date = os.stat(file).st_mtime
stamp_date = os.stat(stampfile).st_mtime
if hook_date < stamp_date:
continue
---snip---
The stamp file exist if there was a previous installation that did not produce a permanent failure. However, a normal failure sets up the condition to run this script. The failure condition is not cleared on success. The code related to determining success and failure is:
---snip---
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
---snip---
Therefore, in the event of a non-permanent failure, the stamp file is created or updated which prevents an followup download to correct the non-permanent failure.
2. The function process_download_requests() also tests for keywords 'script', 'should-download', 'url', and 'sha256' in the para list from the Hook file to determine action:
<code>
for para in hook.iter_paragraphs(open(file)):
*...........if 'script' in para:
if not files:
record_failure(relfile)
break
* command = [para['script']]
* if 'should-download' in para:
db = debconf.DebconfCommunicator('update-notifier')
try:
* should = db.get(para['should-download'])
if should == "false":
# Do nothing with this file.
break
except:
pass
finally:
db.shutdown()
# Download each file and verify the sum
try:
downloaded = set()
for i in range(len(files)):
print("%s: downloading %s" % (relfile, files[i]))
dest_file = download_file(files[i], sums[i])
if dest_file:
command.append(dest_file)
downloaded.add(dest_file)
else:
record_failure(relfile)
break
if relfile in failures + permanent_failures:
break
sys.stdout.flush()
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
# cleanup
for f in downloaded:
os.remove(f)
break
except Exception:
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
# The 'script' is always the last stanza
break
# Not in a 'script' stanza, so we should have some urls
try:
* files.append(para['url'])
* sums.append(para['sha256'])
except Exception as e:
if not isinstance(e, KeyError):
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
break
<code>
where * note the lines affected. These keywords begin with an uppercase in the related files:
---snip---
Url: http://downloads.sourceforge.net/corefonts/webdin32.exe
Sha256: 64595b5abc1080fba8610c5c34fab5863408e806aafe84653ca8575bed17d75a
Script: /usr/lib/msttcorefonts/update-ms-fonts
Should-Download: msttcorefonts/accepted-mscorefonts-eula
---snip---
3. The function process_download_requests() tracks so called "failures" and "permanent_failures". When a failure occurs, the following code sets up the condition for reprocessing later using the function trigger_update_notifier():
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
---snip---
SOLUTION:
====================
The existing code set up several conditions based on the existence of the following files:
stamp file
notifier file
notifier permanent file
The conditions are:
EXISTS ( stamp file )
EXISTS ( stamp file AND notifier file )
EXISTS ( notifier file AND notifier permanent file )
EXISTS ( notifier permanent file )
When no permanent failures are found (no notifier permanent file), the stamp file is set even when normal failures exist (notifier file). This causes the next script execution to skip to the next install item. In the event of success, failure conditions are cleared, but the notifier files are never cleared causing persistent requests to update that never get fulfilled.
For part 1, adding code handle the failure condition when a stamp file exists would be prudent:
---snip---
try:
> if not os.path.exists(NOTIFIER_FILE) and not os.path.exists(NOTIFIER_PERMANENT_FILE):
>>>> hook_date = os.stat(file).st_mtime
>>>> stamp_date = os.stat(stampfile).st_mtime
>>>> if hook_date < stamp_date:
>>>> continue
> elif os.path.exists(stampfile):
> os.unlink(stampfile)
---snip---
where > indicated the added/modded code. This corrects the problem by allowing any perceived failures, permanent or not, to reinstall the downloads. It always removes the stampfile so that if a permanent failure develops, it is handled correctly later.
For part 2, the paragraph keyword lookups may or may not care about case, but update the keywords:
'script', 'should-download', 'url', 'sha256'
to:
'Script', 'Should-Download', 'Url', 'Sha256'
so that they match the file.
For part 3, the comment suggests that the so called "update-notifier file" will be clobbered, but it never is. On a non-permament failure, the failed and permanent-failure marker files are removed when function create_or_update_stampfile() is called. The code should be changed to handle cleared failure conditions (i.e., empty failure lists) as follows:
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
> if not previous_failures and os.path.exists(NOTIFIER_PERMANENT_FILE):
> os.unlink(NOTIFIER_PERMANENT_FILE)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
> elif os.path.exists(NOTIFIER_FILE):
> os.unlink(NOTIFIER_FILE)
---snip---
where > indicated the added code.
In total, when no failure is found, the script will create the stamp file and clear the notifier files. For permanent failures, the stamp file is not created and any notifier files will persist until the next attempted update. For normal failures, the stamp and non-permanent notifier file persists until the next attempted update. On any update, if notifier files exist, the script will attempt to correct the install. If just the stamp file exists, then that install is skipped. |
A persistent bug exists in the update notifier that affects the ttf-mscorefonts-installer and may also affect other packages. On boot, a user is presented with an update notification indicating that the ttf-mscorefonts-installer failed to install and needs installation:
---snip---
The following packages requested additional data downloads after package
installation, but the data could not be downloaded or could not be
processed.
ttf-mscorefonts-installer
The download will be attempted again later, or you can try the download
again now. Running this command requires an active Internet connection.
---snip---
Running the command fails. Manual installation of the ttf-mscorefonts-installer does not correct the notification. The issue was traced to the /usr/lib/update-notifier/package-data-downloader script.
There are several issues in the package-data-downloader script:
1. The function process_download_requests() checks the hook date against the stamp date for a installation package, such as
Hook File: /usr/share/package-data-downloads/ttf-mscorefonts-installer
Stamp File: /var/lib/update-notifier/package-data-downloads/ttf-mscorefonts-installer
with the code:
---snip---
try:
hook_date = os.stat(file).st_mtime
stamp_date = os.stat(stampfile).st_mtime
if hook_date < stamp_date:
continue
---snip---
The stamp file exist if there was a previous installation that did not produce a permanent failure. However, a normal failure sets up the condition to run this script. The failure condition is not cleared on success. The code related to determining success and failure is:
---snip---
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
---snip---
Therefore, in the event of a non-permanent failure, the stamp file is created or updated which prevents an followup download to correct the non-permanent failure.
2. The function process_download_requests() also tests for keywords 'script', 'should-download', 'url', and 'sha256' in the para list from the Hook file to determine action:
<code>
for para in hook.iter_paragraphs(open(file)):
* if 'script' in para:
if not files:
record_failure(relfile)
break
* command = [para['script']]
* if 'should-download' in para:
db = debconf.DebconfCommunicator('update-notifier')
try:
* should = db.get(para['should-download'])
if should == "false":
# Do nothing with this file.
break
except:
pass
finally:
db.shutdown()
# Download each file and verify the sum
try:
downloaded = set()
for i in range(len(files)):
print("%s: downloading %s" % (relfile, files[i]))
dest_file = download_file(files[i], sums[i])
if dest_file:
command.append(dest_file)
downloaded.add(dest_file)
else:
record_failure(relfile)
break
if relfile in failures + permanent_failures:
break
sys.stdout.flush()
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
# cleanup
for f in downloaded:
os.remove(f)
break
except Exception:
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
# The 'script' is always the last stanza
break
# Not in a 'script' stanza, so we should have some urls
try:
* files.append(para['url'])
* sums.append(para['sha256'])
except Exception as e:
if not isinstance(e, KeyError):
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
break
<code>
where * note the lines affected. These keywords begin with an uppercase in the related files:
---snip---
Url: http://downloads.sourceforge.net/corefonts/webdin32.exe
Sha256: 64595b5abc1080fba8610c5c34fab5863408e806aafe84653ca8575bed17d75a
Script: /usr/lib/msttcorefonts/update-ms-fonts
Should-Download: msttcorefonts/accepted-mscorefonts-eula
---snip---
3. The function process_download_requests() tracks so called "failures" and "permanent_failures". When a failure occurs, the following code sets up the condition for reprocessing later using the function trigger_update_notifier():
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
---snip---
SOLUTION:
====================
The existing code set up several conditions based on the existence of the following files:
stamp file
notifier file
notifier permanent file
The conditions are:
EXISTS ( stamp file )
EXISTS ( stamp file AND notifier file )
EXISTS ( notifier file AND notifier permanent file )
EXISTS ( notifier permanent file )
When no permanent failures are found (no notifier permanent file), the stamp file is set even when normal failures exist (notifier file). This causes the next script execution to skip to the next install item. In the event of success, failure conditions are cleared, but the notifier files are never cleared causing persistent requests to update that never get fulfilled.
For part 1, adding code handle the failure condition when a stamp file exists would be prudent:
---snip---
try:
> if not os.path.exists(NOTIFIER_FILE) and not os.path.exists(NOTIFIER_PERMANENT_FILE):
>>>> hook_date = os.stat(file).st_mtime
>>>> stamp_date = os.stat(stampfile).st_mtime
>>>> if hook_date < stamp_date:
>>>> continue
> elif os.path.exists(stampfile):
> os.unlink(stampfile)
---snip---
where > indicated the added/modded code. This corrects the problem by allowing any perceived failures, permanent or not, to reinstall the downloads. It always removes the stampfile so that if a permanent failure develops, it is handled correctly later.
For part 2, the paragraph keyword lookups may or may not care about case, but update the keywords:
'script', 'should-download', 'url', 'sha256'
to:
'Script', 'Should-Download', 'Url', 'Sha256'
so that they match the file.
For part 3, the comment suggests that the so called "update-notifier file" will be clobbered, but it never is. On a non-permament failure, the failed and permanent-failure marker files are removed when function create_or_update_stampfile() is called. The code should be changed to handle cleared failure conditions (i.e., empty failure lists) as follows:
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
> if not previous_failures and os.path.exists(NOTIFIER_PERMANENT_FILE):
> os.unlink(NOTIFIER_PERMANENT_FILE)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
> elif os.path.exists(NOTIFIER_FILE):
> os.unlink(NOTIFIER_FILE)
---snip---
where > indicated the added code.
In total, when no failure is found, the script will create the stamp file and clear the notifier files. For permanent failures, the stamp file is not created and any notifier files will persist until the next attempted update. For normal failures, the stamp and non-permanent notifier file persists until the next attempted update. On any update, if notifier files exist, the script will attempt to correct the install. If just the stamp file exists, then that install is skipped. |
|
| 2016-09-08 22:21:46 |
AtesComp |
description |
A persistent bug exists in the update notifier that affects the ttf-mscorefonts-installer and may also affect other packages. On boot, a user is presented with an update notification indicating that the ttf-mscorefonts-installer failed to install and needs installation:
---snip---
The following packages requested additional data downloads after package
installation, but the data could not be downloaded or could not be
processed.
ttf-mscorefonts-installer
The download will be attempted again later, or you can try the download
again now. Running this command requires an active Internet connection.
---snip---
Running the command fails. Manual installation of the ttf-mscorefonts-installer does not correct the notification. The issue was traced to the /usr/lib/update-notifier/package-data-downloader script.
There are several issues in the package-data-downloader script:
1. The function process_download_requests() checks the hook date against the stamp date for a installation package, such as
Hook File: /usr/share/package-data-downloads/ttf-mscorefonts-installer
Stamp File: /var/lib/update-notifier/package-data-downloads/ttf-mscorefonts-installer
with the code:
---snip---
try:
hook_date = os.stat(file).st_mtime
stamp_date = os.stat(stampfile).st_mtime
if hook_date < stamp_date:
continue
---snip---
The stamp file exist if there was a previous installation that did not produce a permanent failure. However, a normal failure sets up the condition to run this script. The failure condition is not cleared on success. The code related to determining success and failure is:
---snip---
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
---snip---
Therefore, in the event of a non-permanent failure, the stamp file is created or updated which prevents an followup download to correct the non-permanent failure.
2. The function process_download_requests() also tests for keywords 'script', 'should-download', 'url', and 'sha256' in the para list from the Hook file to determine action:
<code>
for para in hook.iter_paragraphs(open(file)):
* if 'script' in para:
if not files:
record_failure(relfile)
break
* command = [para['script']]
* if 'should-download' in para:
db = debconf.DebconfCommunicator('update-notifier')
try:
* should = db.get(para['should-download'])
if should == "false":
# Do nothing with this file.
break
except:
pass
finally:
db.shutdown()
# Download each file and verify the sum
try:
downloaded = set()
for i in range(len(files)):
print("%s: downloading %s" % (relfile, files[i]))
dest_file = download_file(files[i], sums[i])
if dest_file:
command.append(dest_file)
downloaded.add(dest_file)
else:
record_failure(relfile)
break
if relfile in failures + permanent_failures:
break
sys.stdout.flush()
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
# cleanup
for f in downloaded:
os.remove(f)
break
except Exception:
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
# The 'script' is always the last stanza
break
# Not in a 'script' stanza, so we should have some urls
try:
* files.append(para['url'])
* sums.append(para['sha256'])
except Exception as e:
if not isinstance(e, KeyError):
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
break
<code>
where * note the lines affected. These keywords begin with an uppercase in the related files:
---snip---
Url: http://downloads.sourceforge.net/corefonts/webdin32.exe
Sha256: 64595b5abc1080fba8610c5c34fab5863408e806aafe84653ca8575bed17d75a
Script: /usr/lib/msttcorefonts/update-ms-fonts
Should-Download: msttcorefonts/accepted-mscorefonts-eula
---snip---
3. The function process_download_requests() tracks so called "failures" and "permanent_failures". When a failure occurs, the following code sets up the condition for reprocessing later using the function trigger_update_notifier():
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
---snip---
SOLUTION:
====================
The existing code set up several conditions based on the existence of the following files:
stamp file
notifier file
notifier permanent file
The conditions are:
EXISTS ( stamp file )
EXISTS ( stamp file AND notifier file )
EXISTS ( notifier file AND notifier permanent file )
EXISTS ( notifier permanent file )
When no permanent failures are found (no notifier permanent file), the stamp file is set even when normal failures exist (notifier file). This causes the next script execution to skip to the next install item. In the event of success, failure conditions are cleared, but the notifier files are never cleared causing persistent requests to update that never get fulfilled.
For part 1, adding code handle the failure condition when a stamp file exists would be prudent:
---snip---
try:
> if not os.path.exists(NOTIFIER_FILE) and not os.path.exists(NOTIFIER_PERMANENT_FILE):
>>>> hook_date = os.stat(file).st_mtime
>>>> stamp_date = os.stat(stampfile).st_mtime
>>>> if hook_date < stamp_date:
>>>> continue
> elif os.path.exists(stampfile):
> os.unlink(stampfile)
---snip---
where > indicated the added/modded code. This corrects the problem by allowing any perceived failures, permanent or not, to reinstall the downloads. It always removes the stampfile so that if a permanent failure develops, it is handled correctly later.
For part 2, the paragraph keyword lookups may or may not care about case, but update the keywords:
'script', 'should-download', 'url', 'sha256'
to:
'Script', 'Should-Download', 'Url', 'Sha256'
so that they match the file.
For part 3, the comment suggests that the so called "update-notifier file" will be clobbered, but it never is. On a non-permament failure, the failed and permanent-failure marker files are removed when function create_or_update_stampfile() is called. The code should be changed to handle cleared failure conditions (i.e., empty failure lists) as follows:
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
> if not previous_failures and os.path.exists(NOTIFIER_PERMANENT_FILE):
> os.unlink(NOTIFIER_PERMANENT_FILE)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
> elif os.path.exists(NOTIFIER_FILE):
> os.unlink(NOTIFIER_FILE)
---snip---
where > indicated the added code.
In total, when no failure is found, the script will create the stamp file and clear the notifier files. For permanent failures, the stamp file is not created and any notifier files will persist until the next attempted update. For normal failures, the stamp and non-permanent notifier file persists until the next attempted update. On any update, if notifier files exist, the script will attempt to correct the install. If just the stamp file exists, then that install is skipped. |
A persistent bug exists in the update notifier that affects the ttf-mscorefonts-installer and may also affect other packages. On boot, a user is presented with an update notification indicating that the ttf-mscorefonts-installer failed to install and needs installation:
---snip---
The following packages requested additional data downloads after package
installation, but the data could not be downloaded or could not be
processed.
ttf-mscorefonts-installer
The download will be attempted again later, or you can try the download
again now. Running this command requires an active Internet connection.
---snip---
Running the command fails. Manual installation of the ttf-mscorefonts-installer does not correct the notification. The issue was traced to the /usr/lib/update-notifier/package-data-downloader script.
There are several issues in the package-data-downloader script:
1. The function process_download_requests() checks the hook date against the stamp date for a installation package, such as
Hook File: /usr/share/package-data-downloads/ttf-mscorefonts-installer
Stamp File: /var/lib/update-notifier/package-data-downloads/ttf-mscorefonts-installer
with the code:
---snip---
try:
hook_date = os.stat(file).st_mtime
stamp_date = os.stat(stampfile).st_mtime
if hook_date < stamp_date:
continue
---snip---
The stamp file exist if there was a previous installation that did not produce a permanent failure. However, a normal failure sets up the condition to run this script. The failure condition is not cleared on success. The code related to determining success and failure is:
---snip---
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
---snip---
Therefore, in the event of a non-permanent failure, the stamp file is created or updated which prevents an followup download to correct the non-permanent failure.
2. The function process_download_requests() also tests for keywords 'script', 'should-download', 'url', and 'sha256' in the para list from the Hook file to determine action:
<code>
for para in hook.iter_paragraphs(open(file)):
* if 'script' in para:
if not files:
record_failure(relfile)
break
* command = [para['script']]
* if 'should-download' in para:
db = debconf.DebconfCommunicator('update-notifier')
try:
* should = db.get(para['should-download'])
if should == "false":
# Do nothing with this file.
break
except:
pass
finally:
db.shutdown()
# Download each file and verify the sum
try:
downloaded = set()
for i in range(len(files)):
print("%s: downloading %s" % (relfile, files[i]))
dest_file = download_file(files[i], sums[i])
if dest_file:
command.append(dest_file)
downloaded.add(dest_file)
else:
record_failure(relfile)
break
if relfile in failures + permanent_failures:
break
sys.stdout.flush()
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
# cleanup
for f in downloaded:
os.remove(f)
break
except Exception:
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
# The 'script' is always the last stanza
break
# Not in a 'script' stanza, so we should have some urls
try:
* files.append(para['url'])
* sums.append(para['sha256'])
except Exception as e:
if not isinstance(e, KeyError):
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
break
<code>
where * note the lines affected. These keywords begin with an uppercase in the related files:
---snip---
Url: http://downloads.sourceforge.net/corefonts/webdin32.exe
Sha256: 64595b5abc1080fba8610c5c34fab5863408e806aafe84653ca8575bed17d75a
Script: /usr/lib/msttcorefonts/update-ms-fonts
Should-Download: msttcorefonts/accepted-mscorefonts-eula
---snip---
3. The function process_download_requests() tracks so called "failures" and "permanent_failures". When a failure occurs, the following code sets up the condition for reprocessing later using the function trigger_update_notifier():
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
---snip---
SOLUTION:
====================
The existing code set up several conditions based on the existence of the following files:
stamp file
notifier file
notifier permanent file
The conditions are:
EXISTS ( stamp file )
EXISTS ( stamp file AND notifier file )
EXISTS ( notifier file AND notifier permanent file )
EXISTS ( notifier permanent file )
When no permanent failures are found (no notifier permanent file), the stamp file is set even when normal failures exist (notifier file). This causes the next script execution to skip to the next install item. In the event of success, failure conditions are cleared, but the notifier files are never cleared causing persistent requests to update that never get fulfilled.
For part 1, adding code handle the failure condition when a stamp file exists would be prudent:
---snip---
try:
> if not os.path.exists(NOTIFIER_FILE) and not os.path.exists(NOTIFIER_PERMANENT_FILE):
>>>> hook_date = os.stat(file).st_mtime
>>>> stamp_date = os.stat(stampfile).st_mtime
>>>> if hook_date < stamp_date:
>>>> continue
> elif os.path.exists(stampfile):
> os.unlink(stampfile)
---snip---
where > indicated the added/modded code. This corrects the problem by allowing any perceived failures, permanent or not, to reinstall the downloads. It always removes the stampfile so that if a permanent failure develops, it is handled correctly later.
For part 2, the paragraph keyword lookups may or may not care about case, but update the keywords:
'script', 'should-download', 'url', 'sha256'
to:
'Script', 'Should-Download', 'Url', 'Sha256'
so that they match the file.
For part 3, the comment suggests that the so called "update-notifier file" will be clobbered, but it never is. On a non-permament failure, the failed and permanent-failure marker files are removed when function create_or_update_stampfile() is called. The code should be changed to handle cleared failure conditions (i.e., empty failure lists) as follows:
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
> if not previous_failures and os.path.exists(NOTIFIER_PERMANENT_FILE):
> os.unlink(NOTIFIER_PERMANENT_FILE)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
> elif os.path.exists(NOTIFIER_FILE):
> os.unlink(NOTIFIER_FILE)
---snip---
where > indicated the added code.
In total, when no failure is found, the script will create the stamp file and clear the notifier files. For permanent failures, the stamp file is not created and any notifier files will persist until the next attempted update. For normal failures, the stamp and non-permanent notifier file persists until the next attempted update. On any update, if notifier files exist, the script will attempt to correct the install. If just the stamp file exists, then that install is skipped. |
|
| 2016-09-08 22:22:26 |
AtesComp |
description |
A persistent bug exists in the update notifier that affects the ttf-mscorefonts-installer and may also affect other packages. On boot, a user is presented with an update notification indicating that the ttf-mscorefonts-installer failed to install and needs installation:
---snip---
The following packages requested additional data downloads after package
installation, but the data could not be downloaded or could not be
processed.
ttf-mscorefonts-installer
The download will be attempted again later, or you can try the download
again now. Running this command requires an active Internet connection.
---snip---
Running the command fails. Manual installation of the ttf-mscorefonts-installer does not correct the notification. The issue was traced to the /usr/lib/update-notifier/package-data-downloader script.
There are several issues in the package-data-downloader script:
1. The function process_download_requests() checks the hook date against the stamp date for a installation package, such as
Hook File: /usr/share/package-data-downloads/ttf-mscorefonts-installer
Stamp File: /var/lib/update-notifier/package-data-downloads/ttf-mscorefonts-installer
with the code:
---snip---
try:
hook_date = os.stat(file).st_mtime
stamp_date = os.stat(stampfile).st_mtime
if hook_date < stamp_date:
continue
---snip---
The stamp file exist if there was a previous installation that did not produce a permanent failure. However, a normal failure sets up the condition to run this script. The failure condition is not cleared on success. The code related to determining success and failure is:
---snip---
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
---snip---
Therefore, in the event of a non-permanent failure, the stamp file is created or updated which prevents an followup download to correct the non-permanent failure.
2. The function process_download_requests() also tests for keywords 'script', 'should-download', 'url', and 'sha256' in the para list from the Hook file to determine action:
<code>
for para in hook.iter_paragraphs(open(file)):
* if 'script' in para:
if not files:
record_failure(relfile)
break
* command = [para['script']]
* if 'should-download' in para:
db = debconf.DebconfCommunicator('update-notifier')
try:
* should = db.get(para['should-download'])
if should == "false":
# Do nothing with this file.
break
except:
pass
finally:
db.shutdown()
# Download each file and verify the sum
try:
downloaded = set()
for i in range(len(files)):
print("%s: downloading %s" % (relfile, files[i]))
dest_file = download_file(files[i], sums[i])
if dest_file:
command.append(dest_file)
downloaded.add(dest_file)
else:
record_failure(relfile)
break
if relfile in failures + permanent_failures:
break
sys.stdout.flush()
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
# cleanup
for f in downloaded:
os.remove(f)
break
except Exception:
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
# The 'script' is always the last stanza
break
# Not in a 'script' stanza, so we should have some urls
try:
* files.append(para['url'])
* sums.append(para['sha256'])
except Exception as e:
if not isinstance(e, KeyError):
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
break
<code>
where * note the lines affected. These keywords begin with an uppercase in the related files:
---snip---
Url: http://downloads.sourceforge.net/corefonts/webdin32.exe
Sha256: 64595b5abc1080fba8610c5c34fab5863408e806aafe84653ca8575bed17d75a
Script: /usr/lib/msttcorefonts/update-ms-fonts
Should-Download: msttcorefonts/accepted-mscorefonts-eula
---snip---
3. The function process_download_requests() tracks so called "failures" and "permanent_failures". When a failure occurs, the following code sets up the condition for reprocessing later using the function trigger_update_notifier():
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
---snip---
SOLUTION:
====================
The existing code set up several conditions based on the existence of the following files:
stamp file
notifier file
notifier permanent file
The conditions are:
EXISTS ( stamp file )
EXISTS ( stamp file AND notifier file )
EXISTS ( notifier file AND notifier permanent file )
EXISTS ( notifier permanent file )
When no permanent failures are found (no notifier permanent file), the stamp file is set even when normal failures exist (notifier file). This causes the next script execution to skip to the next install item. In the event of success, failure conditions are cleared, but the notifier files are never cleared causing persistent requests to update that never get fulfilled.
For part 1, adding code handle the failure condition when a stamp file exists would be prudent:
---snip---
try:
> if not os.path.exists(NOTIFIER_FILE) and not os.path.exists(NOTIFIER_PERMANENT_FILE):
>>>> hook_date = os.stat(file).st_mtime
>>>> stamp_date = os.stat(stampfile).st_mtime
>>>> if hook_date < stamp_date:
>>>> continue
> elif os.path.exists(stampfile):
> os.unlink(stampfile)
---snip---
where > indicated the added/modded code. This corrects the problem by allowing any perceived failures, permanent or not, to reinstall the downloads. It always removes the stampfile so that if a permanent failure develops, it is handled correctly later.
For part 2, the paragraph keyword lookups may or may not care about case, but update the keywords:
'script', 'should-download', 'url', 'sha256'
to:
'Script', 'Should-Download', 'Url', 'Sha256'
so that they match the file.
For part 3, the comment suggests that the so called "update-notifier file" will be clobbered, but it never is. On a non-permament failure, the failed and permanent-failure marker files are removed when function create_or_update_stampfile() is called. The code should be changed to handle cleared failure conditions (i.e., empty failure lists) as follows:
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
> if not previous_failures and os.path.exists(NOTIFIER_PERMANENT_FILE):
> os.unlink(NOTIFIER_PERMANENT_FILE)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
> elif os.path.exists(NOTIFIER_FILE):
> os.unlink(NOTIFIER_FILE)
---snip---
where > indicated the added code.
In total, when no failure is found, the script will create the stamp file and clear the notifier files. For permanent failures, the stamp file is not created and any notifier files will persist until the next attempted update. For normal failures, the stamp and non-permanent notifier file persists until the next attempted update. On any update, if notifier files exist, the script will attempt to correct the install. If just the stamp file exists, then that install is skipped. |
A persistent bug exists in the update notifier that affects the ttf-mscorefonts-installer and may also affect other packages. On boot, a user is presented with an update notification indicating that the ttf-mscorefonts-installer failed to install and needs installation:
---snip---
The following packages requested additional data downloads after package
installation, but the data could not be downloaded or could not be
processed.
ttf-mscorefonts-installer
The download will be attempted again later, or you can try the download
again now. Running this command requires an active Internet connection.
---snip---
Running the command fails. Manual installation of the ttf-mscorefonts-installer does not correct the notification. The issue was traced to the /usr/lib/update-notifier/package-data-downloader script.
There are several issues in the package-data-downloader script:
1. The function process_download_requests() checks the hook date against the stamp date for a installation package, such as
Hook File: /usr/share/package-data-downloads/ttf-mscorefonts-installer
Stamp File: /var/lib/update-notifier/package-data-downloads/ttf-mscorefonts-installer
with the code:
---snip---
try:
hook_date = os.stat(file).st_mtime
stamp_date = os.stat(stampfile).st_mtime
if hook_date < stamp_date:
continue
---snip---
The stamp file exist if there was a previous installation that did not produce a permanent failure. However, a normal failure sets up the condition to run this script. The failure condition is not cleared on success. The code related to determining success and failure is:
---snip---
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
---snip---
Therefore, in the event of a non-permanent failure, the stamp file is created or updated which prevents an followup download to correct the non-permanent failure.
2. The function process_download_requests() also tests for keywords 'script', 'should-download', 'url', and 'sha256' in the para list from the Hook file to determine action:
"
for para in hook.iter_paragraphs(open(file)):
* if 'script' in para:
if not files:
record_failure(relfile)
break
* command = [para['script']]
* if 'should-download' in para:
db = debconf.DebconfCommunicator('update-notifier')
try:
* should = db.get(para['should-download'])
if should == "false":
# Do nothing with this file.
break
except:
pass
finally:
db.shutdown()
# Download each file and verify the sum
try:
downloaded = set()
for i in range(len(files)):
print("%s: downloading %s" % (relfile, files[i]))
dest_file = download_file(files[i], sums[i])
if dest_file:
command.append(dest_file)
downloaded.add(dest_file)
else:
record_failure(relfile)
break
if relfile in failures + permanent_failures:
break
sys.stdout.flush()
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
# cleanup
for f in downloaded:
os.remove(f)
break
except Exception:
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
# The 'script' is always the last stanza
break
# Not in a 'script' stanza, so we should have some urls
try:
* files.append(para['url'])
* sums.append(para['sha256'])
except Exception as e:
if not isinstance(e, KeyError):
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
break
"
where * note the lines affected. These keywords begin with an uppercase in the related files:
---snip---
Url: http://downloads.sourceforge.net/corefonts/webdin32.exe
Sha256: 64595b5abc1080fba8610c5c34fab5863408e806aafe84653ca8575bed17d75a
Script: /usr/lib/msttcorefonts/update-ms-fonts
Should-Download: msttcorefonts/accepted-mscorefonts-eula
---snip---
3. The function process_download_requests() tracks so called "failures" and "permanent_failures". When a failure occurs, the following code sets up the condition for reprocessing later using the function trigger_update_notifier():
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
---snip---
SOLUTION:
====================
The existing code set up several conditions based on the existence of the following files:
stamp file
notifier file
notifier permanent file
The conditions are:
EXISTS ( stamp file )
EXISTS ( stamp file AND notifier file )
EXISTS ( notifier file AND notifier permanent file )
EXISTS ( notifier permanent file )
When no permanent failures are found (no notifier permanent file), the stamp file is set even when normal failures exist (notifier file). This causes the next script execution to skip to the next install item. In the event of success, failure conditions are cleared, but the notifier files are never cleared causing persistent requests to update that never get fulfilled.
For part 1, adding code handle the failure condition when a stamp file exists would be prudent:
---snip---
try:
> if not os.path.exists(NOTIFIER_FILE) and not os.path.exists(NOTIFIER_PERMANENT_FILE):
>>>> hook_date = os.stat(file).st_mtime
>>>> stamp_date = os.stat(stampfile).st_mtime
>>>> if hook_date < stamp_date:
>>>> continue
> elif os.path.exists(stampfile):
> os.unlink(stampfile)
---snip---
where > indicated the added/modded code. This corrects the problem by allowing any perceived failures, permanent or not, to reinstall the downloads. It always removes the stampfile so that if a permanent failure develops, it is handled correctly later.
For part 2, the paragraph keyword lookups may or may not care about case, but update the keywords:
'script', 'should-download', 'url', 'sha256'
to:
'Script', 'Should-Download', 'Url', 'Sha256'
so that they match the file.
For part 3, the comment suggests that the so called "update-notifier file" will be clobbered, but it never is. On a non-permament failure, the failed and permanent-failure marker files are removed when function create_or_update_stampfile() is called. The code should be changed to handle cleared failure conditions (i.e., empty failure lists) as follows:
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
> if not previous_failures and os.path.exists(NOTIFIER_PERMANENT_FILE):
> os.unlink(NOTIFIER_PERMANENT_FILE)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
> elif os.path.exists(NOTIFIER_FILE):
> os.unlink(NOTIFIER_FILE)
---snip---
where > indicated the added code.
In total, when no failure is found, the script will create the stamp file and clear the notifier files. For permanent failures, the stamp file is not created and any notifier files will persist until the next attempted update. For normal failures, the stamp and non-permanent notifier file persists until the next attempted update. On any update, if notifier files exist, the script will attempt to correct the install. If just the stamp file exists, then that install is skipped. |
|
| 2016-09-08 22:28:17 |
AtesComp |
description |
A persistent bug exists in the update notifier that affects the ttf-mscorefonts-installer and may also affect other packages. On boot, a user is presented with an update notification indicating that the ttf-mscorefonts-installer failed to install and needs installation:
---snip---
The following packages requested additional data downloads after package
installation, but the data could not be downloaded or could not be
processed.
ttf-mscorefonts-installer
The download will be attempted again later, or you can try the download
again now. Running this command requires an active Internet connection.
---snip---
Running the command fails. Manual installation of the ttf-mscorefonts-installer does not correct the notification. The issue was traced to the /usr/lib/update-notifier/package-data-downloader script.
There are several issues in the package-data-downloader script:
1. The function process_download_requests() checks the hook date against the stamp date for a installation package, such as
Hook File: /usr/share/package-data-downloads/ttf-mscorefonts-installer
Stamp File: /var/lib/update-notifier/package-data-downloads/ttf-mscorefonts-installer
with the code:
---snip---
try:
hook_date = os.stat(file).st_mtime
stamp_date = os.stat(stampfile).st_mtime
if hook_date < stamp_date:
continue
---snip---
The stamp file exist if there was a previous installation that did not produce a permanent failure. However, a normal failure sets up the condition to run this script. The failure condition is not cleared on success. The code related to determining success and failure is:
---snip---
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
---snip---
Therefore, in the event of a non-permanent failure, the stamp file is created or updated which prevents an followup download to correct the non-permanent failure.
2. The function process_download_requests() also tests for keywords 'script', 'should-download', 'url', and 'sha256' in the para list from the Hook file to determine action:
"
for para in hook.iter_paragraphs(open(file)):
* if 'script' in para:
if not files:
record_failure(relfile)
break
* command = [para['script']]
* if 'should-download' in para:
db = debconf.DebconfCommunicator('update-notifier')
try:
* should = db.get(para['should-download'])
if should == "false":
# Do nothing with this file.
break
except:
pass
finally:
db.shutdown()
# Download each file and verify the sum
try:
downloaded = set()
for i in range(len(files)):
print("%s: downloading %s" % (relfile, files[i]))
dest_file = download_file(files[i], sums[i])
if dest_file:
command.append(dest_file)
downloaded.add(dest_file)
else:
record_failure(relfile)
break
if relfile in failures + permanent_failures:
break
sys.stdout.flush()
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
# cleanup
for f in downloaded:
os.remove(f)
break
except Exception:
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
# The 'script' is always the last stanza
break
# Not in a 'script' stanza, so we should have some urls
try:
* files.append(para['url'])
* sums.append(para['sha256'])
except Exception as e:
if not isinstance(e, KeyError):
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
break
"
where * note the lines affected. These keywords begin with an uppercase in the related files:
---snip---
Url: http://downloads.sourceforge.net/corefonts/webdin32.exe
Sha256: 64595b5abc1080fba8610c5c34fab5863408e806aafe84653ca8575bed17d75a
Script: /usr/lib/msttcorefonts/update-ms-fonts
Should-Download: msttcorefonts/accepted-mscorefonts-eula
---snip---
3. The function process_download_requests() tracks so called "failures" and "permanent_failures". When a failure occurs, the following code sets up the condition for reprocessing later using the function trigger_update_notifier():
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
---snip---
SOLUTION:
====================
The existing code set up several conditions based on the existence of the following files:
stamp file
notifier file
notifier permanent file
The conditions are:
EXISTS ( stamp file )
EXISTS ( stamp file AND notifier file )
EXISTS ( notifier file AND notifier permanent file )
EXISTS ( notifier permanent file )
When no permanent failures are found (no notifier permanent file), the stamp file is set even when normal failures exist (notifier file). This causes the next script execution to skip to the next install item. In the event of success, failure conditions are cleared, but the notifier files are never cleared causing persistent requests to update that never get fulfilled.
For part 1, adding code handle the failure condition when a stamp file exists would be prudent:
---snip---
try:
> if not os.path.exists(NOTIFIER_FILE) and not os.path.exists(NOTIFIER_PERMANENT_FILE):
>>>> hook_date = os.stat(file).st_mtime
>>>> stamp_date = os.stat(stampfile).st_mtime
>>>> if hook_date < stamp_date:
>>>> continue
> elif os.path.exists(stampfile):
> os.unlink(stampfile)
---snip---
where > indicated the added/modded code. This corrects the problem by allowing any perceived failures, permanent or not, to reinstall the downloads. It always removes the stampfile so that if a permanent failure develops, it is handled correctly later.
For part 2, the paragraph keyword lookups may or may not care about case, but update the keywords:
'script', 'should-download', 'url', 'sha256'
to:
'Script', 'Should-Download', 'Url', 'Sha256'
so that they match the file.
For part 3, the comment suggests that the so called "update-notifier file" will be clobbered, but it never is. On a non-permament failure, the failed and permanent-failure marker files are removed when function create_or_update_stampfile() is called. The code should be changed to handle cleared failure conditions (i.e., empty failure lists) as follows:
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
> if not previous_failures and os.path.exists(NOTIFIER_PERMANENT_FILE):
> os.unlink(NOTIFIER_PERMANENT_FILE)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
> elif os.path.exists(NOTIFIER_FILE):
> os.unlink(NOTIFIER_FILE)
---snip---
where > indicated the added code.
In total, when no failure is found, the script will create the stamp file and clear the notifier files. For permanent failures, the stamp file is not created and any notifier files will persist until the next attempted update. For normal failures, the stamp and non-permanent notifier file persists until the next attempted update. On any update, if notifier files exist, the script will attempt to correct the install. If just the stamp file exists, then that install is skipped. |
A persistent bug exists in the update notifier that affects the ttf-mscorefonts-installer and may also affect other packages. On boot, a user is presented with an update notification indicating that the ttf-mscorefonts-installer failed to install and needs installation:
---snip---
The following packages requested additional data downloads after package
installation, but the data could not be downloaded or could not be
processed.
ttf-mscorefonts-installer
The download will be attempted again later, or you can try the download
again now. Running this command requires an active Internet connection.
---endsnip---
Running the command fails. Manual installation of the ttf-mscorefonts-installer does not correct the notification. The issue was traced to the /usr/lib/update-notifier/package-data-downloader script.
There are several issues in the package-data-downloader script:
1. The function process_download_requests() checks the hook date against the stamp date for a installation package, such as
Hook File: /usr/share/package-data-downloads/ttf-mscorefonts-installer
Stamp File: /var/lib/update-notifier/package-data-downloads/ttf-mscorefonts-installer
with the code:
---snip---
try:
hook_date = os.stat(file).st_mtime
stamp_date = os.stat(stampfile).st_mtime
if hook_date < stamp_date:
continue
---endsnip---
The stamp file exist if there was a previous installation that did not produce a permanent failure. However, a normal failure sets up the condition to run this script. The failure condition is not cleared on success. The code related to determining success and failure is:
---snip---
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
---endsnip---
Therefore, in the event of a non-permanent failure, the stamp file is created or updated which prevents an followup download to correct the non-permanent failure.
2. The function process_download_requests() also tests for keywords 'script', 'should-download', 'url', and 'sha256' in the para list from the Hook file to determine action:
---snip---
for para in hook.iter_paragraphs(open(file)):
*********** if 'script' in para:
if not files:
record_failure(relfile)
break
*************** command = [para['script']]
*************** if 'should-download' in para:
db = debconf.DebconfCommunicator('update-notifier')
try:
*********************** should = db.get(para['should-download'])
if should == "false":
# Do nothing with this file.
break
except:
pass
finally:
db.shutdown()
# Download each file and verify the sum
try:
downloaded = set()
for i in range(len(files)):
print("%s: downloading %s" % (relfile, files[i]))
dest_file = download_file(files[i], sums[i])
if dest_file:
command.append(dest_file)
downloaded.add(dest_file)
else:
record_failure(relfile)
break
if relfile in failures + permanent_failures:
break
sys.stdout.flush()
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
# cleanup
for f in downloaded:
os.remove(f)
break
except Exception:
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
# The 'script' is always the last stanza
break
# Not in a 'script' stanza, so we should have some urls
try:
*************** files.append(para['url'])
*************** sums.append(para['sha256'])
except Exception as e:
if not isinstance(e, KeyError):
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
break
---endsnip---
where * note the lines affected. These keywords begin with an uppercase in the related files:
---snip---
Url: http://downloads.sourceforge.net/corefonts/webdin32.exe
Sha256: 64595b5abc1080fba8610c5c34fab5863408e806aafe84653ca8575bed17d75a
Script: /usr/lib/msttcorefonts/update-ms-fonts
Should-Download: msttcorefonts/accepted-mscorefonts-eula
---endsnip---
3. The function process_download_requests() tracks so called "failures" and "permanent_failures". When a failure occurs, the following code sets up the condition for reprocessing later using the function trigger_update_notifier():
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
---endsnip---
SOLUTION:
====================
The existing code set up several conditions based on the existence of the following files:
stamp file
notifier file
notifier permanent file
The conditions are:
EXISTS ( stamp file )
EXISTS ( stamp file AND notifier file )
EXISTS ( notifier file AND notifier permanent file )
EXISTS ( notifier permanent file )
When no permanent failures are found (no notifier permanent file), the stamp file is set even when normal failures exist (notifier file). This causes the next script execution to skip to the next install item. In the event of success, failure conditions are cleared, but the notifier files are never cleared causing persistent requests to update that never get fulfilled.
For part 1, adding code handle the failure condition when a stamp file exists would be prudent:
---snip---
try:
>>>>>>>>>>> if not os.path.exists(NOTIFIER_FILE) and not os.path.exists(NOTIFIER_PERMANENT_FILE):
>>>>>>>>>>>>>>> hook_date = os.stat(file).st_mtime
>>>>>>>>>>>>>>> stamp_date = os.stat(stampfile).st_mtime
>>>>>>>>>>>>>>> if hook_date < stamp_date:
>>>>>>>>>>>>>>>>>>> continue
>>>>>>>>>>> elif os.path.exists(stampfile):
>>>>>>>>>>>>>>> os.unlink(stampfile)
---endsnip---
where > indicated the added/modded code. This corrects the problem by allowing any perceived failures, permanent or not, to reinstall the downloads. It always removes the stampfile so that if a permanent failure develops, it is handled correctly later.
For part 2, the paragraph keyword lookups may or may not care about case, but update the keywords:
'script', 'should-download', 'url', 'sha256'
to:
'Script', 'Should-Download', 'Url', 'Sha256'
so that they match the file.
For part 3, the comment suggests that the so called "update-notifier file" will be clobbered, but it never is. On a non-permament failure, the failed and permanent-failure marker files are removed when function create_or_update_stampfile() is called. The code should be changed to handle cleared failure conditions (i.e., empty failure lists) as follows:
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
>>> if not previous_failures and os.path.exists(NOTIFIER_PERMANENT_FILE):
>>>>>>> os.unlink(NOTIFIER_PERMANENT_FILE)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
>>> elif os.path.exists(NOTIFIER_FILE):
>>>>>>> os.unlink(NOTIFIER_FILE)
---endsnip---
where > indicated the added code.
In total, when no failure is found, the script will create the stamp file and clear the notifier files. For permanent failures, the stamp file is not created and any notifier files will persist until the next attempted update. For normal failures, the stamp and non-permanent notifier file persists until the next attempted update. On any update, if notifier files exist, the script will attempt to correct the install. If just the stamp file exists, then that install is skipped. |
|
| 2016-09-08 22:32:59 |
AtesComp |
description |
A persistent bug exists in the update notifier that affects the ttf-mscorefonts-installer and may also affect other packages. On boot, a user is presented with an update notification indicating that the ttf-mscorefonts-installer failed to install and needs installation:
---snip---
The following packages requested additional data downloads after package
installation, but the data could not be downloaded or could not be
processed.
ttf-mscorefonts-installer
The download will be attempted again later, or you can try the download
again now. Running this command requires an active Internet connection.
---endsnip---
Running the command fails. Manual installation of the ttf-mscorefonts-installer does not correct the notification. The issue was traced to the /usr/lib/update-notifier/package-data-downloader script.
There are several issues in the package-data-downloader script:
1. The function process_download_requests() checks the hook date against the stamp date for a installation package, such as
Hook File: /usr/share/package-data-downloads/ttf-mscorefonts-installer
Stamp File: /var/lib/update-notifier/package-data-downloads/ttf-mscorefonts-installer
with the code:
---snip---
try:
hook_date = os.stat(file).st_mtime
stamp_date = os.stat(stampfile).st_mtime
if hook_date < stamp_date:
continue
---endsnip---
The stamp file exist if there was a previous installation that did not produce a permanent failure. However, a normal failure sets up the condition to run this script. The failure condition is not cleared on success. The code related to determining success and failure is:
---snip---
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
---endsnip---
Therefore, in the event of a non-permanent failure, the stamp file is created or updated which prevents an followup download to correct the non-permanent failure.
2. The function process_download_requests() also tests for keywords 'script', 'should-download', 'url', and 'sha256' in the para list from the Hook file to determine action:
---snip---
for para in hook.iter_paragraphs(open(file)):
*********** if 'script' in para:
if not files:
record_failure(relfile)
break
*************** command = [para['script']]
*************** if 'should-download' in para:
db = debconf.DebconfCommunicator('update-notifier')
try:
*********************** should = db.get(para['should-download'])
if should == "false":
# Do nothing with this file.
break
except:
pass
finally:
db.shutdown()
# Download each file and verify the sum
try:
downloaded = set()
for i in range(len(files)):
print("%s: downloading %s" % (relfile, files[i]))
dest_file = download_file(files[i], sums[i])
if dest_file:
command.append(dest_file)
downloaded.add(dest_file)
else:
record_failure(relfile)
break
if relfile in failures + permanent_failures:
break
sys.stdout.flush()
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
# cleanup
for f in downloaded:
os.remove(f)
break
except Exception:
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
# The 'script' is always the last stanza
break
# Not in a 'script' stanza, so we should have some urls
try:
*************** files.append(para['url'])
*************** sums.append(para['sha256'])
except Exception as e:
if not isinstance(e, KeyError):
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
break
---endsnip---
where * note the lines affected. These keywords begin with an uppercase in the related files:
---snip---
Url: http://downloads.sourceforge.net/corefonts/webdin32.exe
Sha256: 64595b5abc1080fba8610c5c34fab5863408e806aafe84653ca8575bed17d75a
Script: /usr/lib/msttcorefonts/update-ms-fonts
Should-Download: msttcorefonts/accepted-mscorefonts-eula
---endsnip---
3. The function process_download_requests() tracks so called "failures" and "permanent_failures". When a failure occurs, the following code sets up the condition for reprocessing later using the function trigger_update_notifier():
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
---endsnip---
SOLUTION:
====================
The existing code set up several conditions based on the existence of the following files:
stamp file
notifier file
notifier permanent file
The conditions are:
EXISTS ( stamp file )
EXISTS ( stamp file AND notifier file )
EXISTS ( notifier file AND notifier permanent file )
EXISTS ( notifier permanent file )
When no permanent failures are found (no notifier permanent file), the stamp file is set even when normal failures exist (notifier file). This causes the next script execution to skip to the next install item. In the event of success, failure conditions are cleared, but the notifier files are never cleared causing persistent requests to update that never get fulfilled.
For part 1, adding code handle the failure condition when a stamp file exists would be prudent:
---snip---
try:
>>>>>>>>>>> if not os.path.exists(NOTIFIER_FILE) and not os.path.exists(NOTIFIER_PERMANENT_FILE):
>>>>>>>>>>>>>>> hook_date = os.stat(file).st_mtime
>>>>>>>>>>>>>>> stamp_date = os.stat(stampfile).st_mtime
>>>>>>>>>>>>>>> if hook_date < stamp_date:
>>>>>>>>>>>>>>>>>>> continue
>>>>>>>>>>> elif os.path.exists(stampfile):
>>>>>>>>>>>>>>> os.unlink(stampfile)
---endsnip---
where > indicated the added/modded code. This corrects the problem by allowing any perceived failures, permanent or not, to reinstall the downloads. It always removes the stampfile so that if a permanent failure develops, it is handled correctly later.
For part 2, the paragraph keyword lookups may or may not care about case, but update the keywords:
'script', 'should-download', 'url', 'sha256'
to:
'Script', 'Should-Download', 'Url', 'Sha256'
so that they match the file.
For part 3, the comment suggests that the so called "update-notifier file" will be clobbered, but it never is. On a non-permament failure, the failed and permanent-failure marker files are removed when function create_or_update_stampfile() is called. The code should be changed to handle cleared failure conditions (i.e., empty failure lists) as follows:
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
>>> if not previous_failures and os.path.exists(NOTIFIER_PERMANENT_FILE):
>>>>>>> os.unlink(NOTIFIER_PERMANENT_FILE)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
>>> elif os.path.exists(NOTIFIER_FILE):
>>>>>>> os.unlink(NOTIFIER_FILE)
---endsnip---
where > indicated the added code.
In total, when no failure is found, the script will create the stamp file and clear the notifier files. For permanent failures, the stamp file is not created and any notifier files will persist until the next attempted update. For normal failures, the stamp and non-permanent notifier file persists until the next attempted update. On any update, if notifier files exist, the script will attempt to correct the install. If just the stamp file exists, then that install is skipped. |
A persistent bug exists in the update notifier that affects the ttf-mscorefonts-installer and may also affect other packages. On boot, a user is presented with an update notification indicating that the ttf-mscorefonts-installer failed to install and needs installation:
---snip---
The following packages requested additional data downloads after package
installation, but the data could not be downloaded or could not be
processed.
ttf-mscorefonts-installer
The download will be attempted again later, or you can try the download
again now. Running this command requires an active Internet connection.
---endsnip---
Running the command fails. Manual installation of the ttf-mscorefonts-installer does not correct the notification. The issue was traced to the /usr/lib/update-notifier/package-data-downloader script.
There are several issues in the package-data-downloader script:
1. The function process_download_requests() checks the hook date against the stamp date for a installation package, such as
Hook File: /usr/share/package-data-downloads/ttf-mscorefonts-installer
Stamp File: /var/lib/update-notifier/package-data-downloads/ttf-mscorefonts-installer
with the code:
---snip---
try:
hook_date = os.stat(file).st_mtime
stamp_date = os.stat(stampfile).st_mtime
if hook_date < stamp_date:
continue
---endsnip---
The stamp file exist if there was a previous installation that did not produce a permanent failure. However, a normal failure sets up the condition to run this script. The failure condition is not cleared on success. The code related to determining success and failure is:
---snip---
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
---endsnip---
Therefore, in the event of a non-permanent failure, the stamp file is created or updated which prevents an followup download to correct the non-permanent failure.
2. The function process_download_requests() also tests for keywords 'script', 'should-download', 'url', and 'sha256' in the para list from the Hook file to determine action:
---snip---
for para in hook.iter_paragraphs(open(file)):
* . . . . . if 'script' in para:
if not files:
record_failure(relfile)
break
* . . . . . . . command = [para['script']]
* . . . . . . . if 'should-download' in para:
db = debconf.DebconfCommunicator('update-notifier')
try:
* . . . . . . . . . . . should = db.get(para['should-download'])
if should == "false":
# Do nothing with this file.
break
except:
pass
finally:
db.shutdown()
# Download each file and verify the sum
try:
downloaded = set()
for i in range(len(files)):
print("%s: downloading %s" % (relfile, files[i]))
dest_file = download_file(files[i], sums[i])
if dest_file:
command.append(dest_file)
downloaded.add(dest_file)
else:
record_failure(relfile)
break
if relfile in failures + permanent_failures:
break
sys.stdout.flush()
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
# cleanup
for f in downloaded:
os.remove(f)
break
except Exception:
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
# The 'script' is always the last stanza
break
# Not in a 'script' stanza, so we should have some urls
try:
* . . . . . . . files.append(para['url'])
* . . . . . . . sums.append(para['sha256'])
except Exception as e:
if not isinstance(e, KeyError):
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
break
---endsnip---
where * note the lines affected. These keywords begin with an uppercase in the related files:
---snip---
Url: http://downloads.sourceforge.net/corefonts/webdin32.exe
Sha256: 64595b5abc1080fba8610c5c34fab5863408e806aafe84653ca8575bed17d75a
Script: /usr/lib/msttcorefonts/update-ms-fonts
Should-Download: msttcorefonts/accepted-mscorefonts-eula
---endsnip---
3. The function process_download_requests() tracks so called "failures" and "permanent_failures". When a failure occurs, the following code sets up the condition for reprocessing later using the function trigger_update_notifier():
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
---endsnip---
SOLUTION:
====================
The existing code set up several conditions based on the existence of the following files:
stamp file
notifier file
notifier permanent file
The conditions are:
EXISTS ( stamp file )
EXISTS ( stamp file AND notifier file )
EXISTS ( notifier file AND notifier permanent file )
EXISTS ( notifier permanent file )
When no permanent failures are found (no notifier permanent file), the stamp file is set even when normal failures exist (notifier file). This causes the next script execution to skip to the next install item. In the event of success, failure conditions are cleared, but the notifier files are never cleared causing persistent requests to update that never get fulfilled.
For part 1, adding code handle the failure condition when a stamp file exists would be prudent:
---snip---
try:
> . . . . . if not os.path.exists(NOTIFIER_FILE) and not os.path.exists(NOTIFIER_PERMANENT_FILE):
> . . . . . . . hook_date = os.stat(file).st_mtime
> . . . . . . . stamp_date = os.stat(stampfile).st_mtime
> . . . . . . . if hook_date < stamp_date:
> . . . . . . . . . continue
> . . . . . elif os.path.exists(stampfile):
> . . . . . . . os.unlink(stampfile)
---endsnip---
where > indicated the added/modded code. This corrects the problem by allowing any perceived failures, permanent or not, to reinstall the downloads. It always removes the stampfile so that if a permanent failure develops, it is handled correctly later.
For part 2, the paragraph keyword lookups may or may not care about case, but update the keywords:
'script', 'should-download', 'url', 'sha256'
to:
'Script', 'Should-Download', 'Url', 'Sha256'
so that they match the file.
For part 3, the comment suggests that the so called "update-notifier file" will be clobbered, but it never is. On a non-permament failure, the failed and permanent-failure marker files are removed when function create_or_update_stampfile() is called. The code should be changed to handle cleared failure conditions (i.e., empty failure lists) as follows:
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
> . if not previous_failures and os.path.exists(NOTIFIER_PERMANENT_FILE):
> . . . os.unlink(NOTIFIER_PERMANENT_FILE)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
> . elif os.path.exists(NOTIFIER_FILE):
> . . . os.unlink(NOTIFIER_FILE)
---endsnip---
where > indicated the added code.
In total, when no failure is found, the script will create the stamp file and clear the notifier files. For permanent failures, the stamp file is not created and any notifier files will persist until the next attempted update. For normal failures, the stamp and non-permanent notifier file persists until the next attempted update. On any update, if notifier files exist, the script will attempt to correct the install. If just the stamp file exists, then that install is skipped. |
|
| 2016-09-08 22:35:45 |
AtesComp |
description |
A persistent bug exists in the update notifier that affects the ttf-mscorefonts-installer and may also affect other packages. On boot, a user is presented with an update notification indicating that the ttf-mscorefonts-installer failed to install and needs installation:
---snip---
The following packages requested additional data downloads after package
installation, but the data could not be downloaded or could not be
processed.
ttf-mscorefonts-installer
The download will be attempted again later, or you can try the download
again now. Running this command requires an active Internet connection.
---endsnip---
Running the command fails. Manual installation of the ttf-mscorefonts-installer does not correct the notification. The issue was traced to the /usr/lib/update-notifier/package-data-downloader script.
There are several issues in the package-data-downloader script:
1. The function process_download_requests() checks the hook date against the stamp date for a installation package, such as
Hook File: /usr/share/package-data-downloads/ttf-mscorefonts-installer
Stamp File: /var/lib/update-notifier/package-data-downloads/ttf-mscorefonts-installer
with the code:
---snip---
try:
hook_date = os.stat(file).st_mtime
stamp_date = os.stat(stampfile).st_mtime
if hook_date < stamp_date:
continue
---endsnip---
The stamp file exist if there was a previous installation that did not produce a permanent failure. However, a normal failure sets up the condition to run this script. The failure condition is not cleared on success. The code related to determining success and failure is:
---snip---
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
---endsnip---
Therefore, in the event of a non-permanent failure, the stamp file is created or updated which prevents an followup download to correct the non-permanent failure.
2. The function process_download_requests() also tests for keywords 'script', 'should-download', 'url', and 'sha256' in the para list from the Hook file to determine action:
---snip---
for para in hook.iter_paragraphs(open(file)):
* . . . . . if 'script' in para:
if not files:
record_failure(relfile)
break
* . . . . . . . command = [para['script']]
* . . . . . . . if 'should-download' in para:
db = debconf.DebconfCommunicator('update-notifier')
try:
* . . . . . . . . . . . should = db.get(para['should-download'])
if should == "false":
# Do nothing with this file.
break
except:
pass
finally:
db.shutdown()
# Download each file and verify the sum
try:
downloaded = set()
for i in range(len(files)):
print("%s: downloading %s" % (relfile, files[i]))
dest_file = download_file(files[i], sums[i])
if dest_file:
command.append(dest_file)
downloaded.add(dest_file)
else:
record_failure(relfile)
break
if relfile in failures + permanent_failures:
break
sys.stdout.flush()
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
# cleanup
for f in downloaded:
os.remove(f)
break
except Exception:
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
# The 'script' is always the last stanza
break
# Not in a 'script' stanza, so we should have some urls
try:
* . . . . . . . files.append(para['url'])
* . . . . . . . sums.append(para['sha256'])
except Exception as e:
if not isinstance(e, KeyError):
traceback.print_exc(file=sys.stderr)
record_failure(relfile)
break
---endsnip---
where * note the lines affected. These keywords begin with an uppercase in the related files:
---snip---
Url: http://downloads.sourceforge.net/corefonts/webdin32.exe
Sha256: 64595b5abc1080fba8610c5c34fab5863408e806aafe84653ca8575bed17d75a
Script: /usr/lib/msttcorefonts/update-ms-fonts
Should-Download: msttcorefonts/accepted-mscorefonts-eula
---endsnip---
3. The function process_download_requests() tracks so called "failures" and "permanent_failures". When a failure occurs, the following code sets up the condition for reprocessing later using the function trigger_update_notifier():
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
---endsnip---
SOLUTION:
====================
The existing code set up several conditions based on the existence of the following files:
stamp file
notifier file
notifier permanent file
The conditions are:
EXISTS ( stamp file )
EXISTS ( stamp file AND notifier file )
EXISTS ( notifier file AND notifier permanent file )
EXISTS ( notifier permanent file )
When no permanent failures are found (no notifier permanent file), the stamp file is set even when normal failures exist (notifier file). This causes the next script execution to skip to the next install item. In the event of success, failure conditions are cleared, but the notifier files are never cleared causing persistent requests to update that never get fulfilled.
For part 1, adding code handle the failure condition when a stamp file exists would be prudent:
---snip---
try:
> . . . . . if not os.path.exists(NOTIFIER_FILE) and not os.path.exists(NOTIFIER_PERMANENT_FILE):
> . . . . . . . hook_date = os.stat(file).st_mtime
> . . . . . . . stamp_date = os.stat(stampfile).st_mtime
> . . . . . . . if hook_date < stamp_date:
> . . . . . . . . . continue
> . . . . . elif os.path.exists(stampfile):
> . . . . . . . os.unlink(stampfile)
---endsnip---
where > indicated the added/modded code. This corrects the problem by allowing any perceived failures, permanent or not, to reinstall the downloads. It always removes the stampfile so that if a permanent failure develops, it is handled correctly later.
For part 2, the paragraph keyword lookups may or may not care about case, but update the keywords:
'script', 'should-download', 'url', 'sha256'
to:
'Script', 'Should-Download', 'Url', 'Sha256'
so that they match the file.
For part 3, the comment suggests that the so called "update-notifier file" will be clobbered, but it never is. On a non-permament failure, the failed and permanent-failure marker files are removed when function create_or_update_stampfile() is called. The code should be changed to handle cleared failure conditions (i.e., empty failure lists) as follows:
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
> . if not previous_failures and os.path.exists(NOTIFIER_PERMANENT_FILE):
> . . . os.unlink(NOTIFIER_PERMANENT_FILE)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
> . elif os.path.exists(NOTIFIER_FILE):
> . . . os.unlink(NOTIFIER_FILE)
---endsnip---
where > indicated the added code.
In total, when no failure is found, the script will create the stamp file and clear the notifier files. For permanent failures, the stamp file is not created and any notifier files will persist until the next attempted update. For normal failures, the stamp and non-permanent notifier file persists until the next attempted update. On any update, if notifier files exist, the script will attempt to correct the install. If just the stamp file exists, then that install is skipped. |
A persistent bug exists in the update notifier that affects the ttf-mscorefonts-installer and may also affect other packages. On boot, a user is presented with an update notification indicating that the ttf-mscorefonts-installer failed to install and needs installation:
---snip---
The following packages requested additional data downloads after package
installation, but the data could not be downloaded or could not be
processed.
ttf-mscorefonts-installer
The download will be attempted again later, or you can try the download
again now. Running this command requires an active Internet connection.
---endsnip---
Running the command fails. Manual installation of the ttf-mscorefonts-installer does not correct the notification. The issue was traced to the /usr/lib/update-notifier/package-data-downloader script.
There are several issues in the package-data-downloader script:
1. The function process_download_requests() checks the hook date against the stamp date for a installation package, such as
Hook File: /usr/share/package-data-downloads/ttf-mscorefonts-installer
Stamp File: /var/lib/update-notifier/package-data-downloads/ttf-mscorefonts-installer
with the code:
---snip---
try:
hook_date = os.stat(file).st_mtime
stamp_date = os.stat(stampfile).st_mtime
if hook_date < stamp_date:
continue
---endsnip---
The stamp file exist if there was a previous installation that did not produce a permanent failure. However, a normal failure sets up the condition to run this script. The failure condition is not cleared on success. The code related to determining success and failure is:
---snip---
result = subprocess.call(command)
if result:
# There's no sense redownloading if the script fails
permanent_failures.append(relfile)
else:
create_or_update_stampfile(stampfile)
---endsnip---
Therefore, in the event of a non-permanent failure, the stamp file is created or updated which prevents an followup download to correct the non-permanent failure.
2. The function process_download_requests() also tests for keywords 'script', 'should-download', 'url', and 'sha256' in the para list from the Hook file to determine action:
---snip---
for para in hook.iter_paragraphs(open(file)):
* . . . . . if 'script' in para:
if not files:
record_failure(relfile)
break
* . . . . . . . command = [para['script']]
* . . . . . . . if 'should-download' in para:
db = debconf.DebconfCommunicator('update-notifier')
try:
* . . . . . . . . . . . should = db.get(para['should-download'])
if should == "false":
# Do nothing with this file.
break
---endsnip---
---snip---
# Not in a 'script' stanza, so we should have some urls
try:
* . . . . . . . files.append(para['url'])
* . . . . . . . sums.append(para['sha256'])
except Exception as e:
---endsnip---
where * note the lines affected. These keywords begin with an uppercase in the related files:
---snip---
Url: http://downloads.sourceforge.net/corefonts/webdin32.exe
Sha256: 64595b5abc1080fba8610c5c34fab5863408e806aafe84653ca8575bed17d75a
Script: /usr/lib/msttcorefonts/update-ms-fonts
Should-Download: msttcorefonts/accepted-mscorefonts-eula
---endsnip---
3. The function process_download_requests() tracks so called "failures" and "permanent_failures". When a failure occurs, the following code sets up the condition for reprocessing later using the function trigger_update_notifier():
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
---endsnip---
SOLUTION:
====================
The existing code set up several conditions based on the existence of the following files:
stamp file
notifier file
notifier permanent file
The conditions are:
EXISTS ( stamp file )
EXISTS ( stamp file AND notifier file )
EXISTS ( notifier file AND notifier permanent file )
EXISTS ( notifier permanent file )
When no permanent failures are found (no notifier permanent file), the stamp file is set even when normal failures exist (notifier file). This causes the next script execution to skip to the next install item. In the event of success, failure conditions are cleared, but the notifier files are never cleared causing persistent requests to update that never get fulfilled.
For part 1, adding code handle the failure condition when a stamp file exists would be prudent:
---snip---
try:
> . . . . . if not os.path.exists(NOTIFIER_FILE) and not os.path.exists(NOTIFIER_PERMANENT_FILE):
> . . . . . . . hook_date = os.stat(file).st_mtime
> . . . . . . . stamp_date = os.stat(stampfile).st_mtime
> . . . . . . . if hook_date < stamp_date:
> . . . . . . . . . continue
> . . . . . elif os.path.exists(stampfile):
> . . . . . . . os.unlink(stampfile)
---endsnip---
where > indicated the added/modded code. This corrects the problem by allowing any perceived failures, permanent or not, to reinstall the downloads. It always removes the stampfile so that if a permanent failure develops, it is handled correctly later.
For part 2, the paragraph keyword lookups may or may not care about case, but update the keywords:
'script', 'should-download', 'url', 'sha256'
to:
'Script', 'Should-Download', 'Url', 'Sha256'
so that they match the file.
For part 3, the comment suggests that the so called "update-notifier file" will be clobbered, but it never is. On a non-permament failure, the failed and permanent-failure marker files are removed when function create_or_update_stampfile() is called. The code should be changed to handle cleared failure conditions (i.e., empty failure lists) as follows:
---snip---
previous_failures = existing_permanent_failures()
# We only report about "permanent" failures when there are new ones,
# but we want the whole list of permanently-failing hooks so when
# we clobber the update-notifier file we don't lose information the
# user may not have seen yet
if permanent_failures:
new_failures = False
for failure in permanent_failures:
if failure not in previous_failures:
mark_hook_failed(failure, permanent=True)
previous_failures.append(failure)
new_failures = True
if new_failures:
trigger_update_notifier(previous_failures, permanent=True)
> . if not previous_failures and os.path.exists(NOTIFIER_PERMANENT_FILE):
> . . . os.unlink(NOTIFIER_PERMANENT_FILE)
# Filter out new failure reports for permanently-failed packages
our_failures = [x for x in failures if x not in previous_failures]
if our_failures:
for failure in our_failures:
mark_hook_failed(failure)
trigger_update_notifier(our_failures)
> . elif os.path.exists(NOTIFIER_FILE):
> . . . os.unlink(NOTIFIER_FILE)
---endsnip---
where > indicated the added code.
In total, when no failure is found, the script will create the stamp file and clear the notifier files. For permanent failures, the stamp file is not created and any notifier files will persist until the next attempted update. For normal failures, the stamp and non-permanent notifier file persists until the next attempted update. On any update, if notifier files exist, the script will attempt to correct the install. If just the stamp file exists, then that install is skipped. |
|
| 2016-09-09 00:09:06 |
AtesComp |
attachment added |
|
Solution as per initial comment https://bugs.launchpad.net/ubuntu/+source/update-notifier/+bug/1621629/+attachment/4737125/+files/package-data-downloader.diff |
|
| 2016-09-09 00:42:23 |
Ubuntu Foundations Team Bug Bot |
tags |
package-data-downloader ttf-mscorefonts-installer update-notifier |
package-data-downloader patch ttf-mscorefonts-installer update-notifier |
|
| 2016-09-09 00:42:34 |
Ubuntu Foundations Team Bug Bot |
bug |
|
|
added subscriber Ubuntu Review Team |
| 2016-09-09 07:20:59 |
Sebastien Bacher |
bug |
|
|
added subscriber Ubuntu Sponsors Team |
| 2016-09-09 17:19:24 |
Brian Murray |
tags |
package-data-downloader patch ttf-mscorefonts-installer update-notifier |
package-data-downloader patch rls-y-incoming ttf-mscorefonts-installer update-notifier |
|
| 2016-09-12 22:00:07 |
Brian Murray |
update-notifier (Ubuntu): importance |
Undecided |
High |
|
| 2016-09-12 22:00:28 |
Brian Murray |
update-notifier (Ubuntu): assignee |
|
Brian Murray (brian-murray) |
|
| 2016-09-12 22:00:32 |
Brian Murray |
removed subscriber Ubuntu Sponsors Team |
|
|
|
| 2016-09-20 22:53:57 |
Launchpad Janitor |
branch linked |
|
lp:update-notifier |
|
| 2016-09-21 20:24:53 |
Brian Murray |
update-notifier (Ubuntu): status |
New |
In Progress |
|
| 2016-09-21 20:25:00 |
Brian Murray |
removed subscriber Ubuntu Review Team |
|
|
|
| 2016-09-28 09:16:41 |
Launchpad Janitor |
update-notifier (Ubuntu): status |
In Progress |
Fix Released |
|
| 2016-09-30 23:56:53 |
Brian Murray |
nominated for series |
|
Ubuntu Xenial |
|
| 2016-09-30 23:56:53 |
Brian Murray |
bug task added |
|
update-notifier (Ubuntu Xenial) |
|
| 2016-10-05 11:23:02 |
Amr Ibrahim |
tags |
package-data-downloader patch rls-y-incoming ttf-mscorefonts-installer update-notifier |
package-data-downloader patch rls-y-incoming trusty ttf-mscorefonts-installer update-notifier xenial |
|
| 2016-10-06 19:02:15 |
Brian Murray |
update-notifier (Ubuntu Xenial): assignee |
|
Brian Murray (brian-murray) |
|
| 2016-10-06 19:02:19 |
Brian Murray |
update-notifier (Ubuntu Xenial): importance |
Undecided |
High |
|
| 2016-10-06 19:02:23 |
Brian Murray |
update-notifier (Ubuntu Xenial): status |
New |
In Progress |
|
| 2016-10-13 14:07:35 |
Martin Pitt |
update-notifier (Ubuntu Xenial): status |
In Progress |
Fix Committed |
|
| 2016-10-13 14:07:38 |
Martin Pitt |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2016-10-13 14:07:44 |
Martin Pitt |
bug |
|
|
added subscriber SRU Verification |
| 2016-10-13 14:07:46 |
Martin Pitt |
tags |
package-data-downloader patch rls-y-incoming trusty ttf-mscorefonts-installer update-notifier xenial |
package-data-downloader patch rls-y-incoming trusty ttf-mscorefonts-installer update-notifier verification-needed xenial |
|
| 2016-10-14 06:19:14 |
Amr Ibrahim |
attachment added |
|
Information-available.png https://bugs.launchpad.net/ubuntu/+source/update-notifier/+bug/1621629/+attachment/4760767/+files/Information-available.png |
|
| 2016-10-14 06:19:39 |
Amr Ibrahim |
bug |
|
|
added subscriber Amr Ibrahim |
| 2016-11-02 19:54:04 |
Brian Murray |
tags |
package-data-downloader patch rls-y-incoming trusty ttf-mscorefonts-installer update-notifier verification-needed xenial |
package-data-downloader patch rls-y-incoming trusty ttf-mscorefonts-installer update-notifier verification-done xenial |
|
| 2016-11-03 16:08:50 |
Launchpad Janitor |
update-notifier (Ubuntu Xenial): status |
Fix Committed |
Fix Released |
|
| 2016-11-03 16:08:54 |
Brian Murray |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
