FontTriplet errors when building on Jenkins when heavy load

Bug #1221721 reported by Anne Gentle
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
openstack-manuals
Fix Released
High
Davanum Srinivas (DIMS)

Bug Description

This has happened on the operations-guide and openstack-training on both 1.8.0 and 1.9.2.

http://logs.openstack.org/19/45119/3/check/gate-openstack-manuals-validate/535fd23/console.html

Tags: doc-builds
Anne Gentle (annegentle)
tags: added: doc-builds
Revision history for this message
Tom Fifield (fifieldt) wrote :

We should collect some more data perhaps?

Does David Cramer have any ideas?

Should we talk to infra>

Changed in openstack-manuals:
status: New → Incomplete
Revision history for this message
Anne Gentle (annegentle) wrote :

I just showed it to David Cramer and he doesn't see a root cause.

I think we should ask -infra about the load as it seems to come up when lots of activity is going on.

Revision history for this message
Anne Gentle (annegentle) wrote :

Also, builds for the failed book works locally, so I suspect the Jenkins slave is at fault.

Revision history for this message
Tom Fifield (fifieldt) wrote :

load sounds like a good working theory for now

Anne Gentle (annegentle)
summary: - Spurious FontTriplet errors when building on Jenkins
+ FontTriplet errors when building on Jenkins when heavy load
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to openstack-manuals (master)

Reviewed: https://review.openstack.org/45475
Committed: http://github.com/openstack/openstack-manuals/commit/f5bba0834b27f387e54443f26d6efc18732eb4c4
Submitter: Jenkins
Branch: master

commit f5bba0834b27f387e54443f26d6efc18732eb4c4
Author: Andreas Jaeger <email address hidden>
Date: Fri Sep 6 21:19:49 2013 +0200

    Handle removed files in tools/validate.py

    Previously if a file was removed, no checks were done that the file
    was not referenced anywhere.

    Now you get e.g.:
    Checking for removed files
      Removed file: doc/src/docbkx/common/section_keystone_cli_users_tenants_roles.xml
      Removed file: doc/src/docbkx/openstack-user-admin/src/section_cli_set_block_storage_quotas.xml
      Removed file: doc/src/docbkx/openstack-user-admin/src/section_cli_set_compute_quotas.xml
      Removed file: doc/src/docbkx/openstack-user-admin/src/section_keystone_cli_manage_projects_users.xml
      Removed file: doc/src/docbkx/openstack-user/src/section_cli_help.xml
      Removed file: doc/src/docbkx/openstack-user/src/section_cli_install.xml
      Removed file: doc/src/docbkx/openstack-user/src/section_cli_openrc.xml
      Removed file: doc/src/docbkx/openstack-user/src/section_cli_overview.xml
      Removed file: doc/src/docbkx/openstack-user/src/section_cli_version.xml
      File section_compute-configure-quotas.xml has an xi:include on deleted file ../openstack-user-admin/src/section_cli_set_compute_quotas.xml

    And validate.py exists with error code to mark this as failure.

    Also, limit number of parallel build jobs to 4 to reduce Jenkins load,
    hope we see less org.apache.fop.fonts.FontTriplet.
    Related-Bug: # 1221721

    Change-Id: I3e47217958840f61347f71b159414928f266d6d5

Revision history for this message
Davanum Srinivas (DIMS) (dims-v) wrote :

Here's a theory: Looks like the cache file is shared between Fop running in different jenkins jobs.
http://grepcode.com/file/repo1.maven.org/maven2/org.apache.xmlgraphics/fop/1.1/org/apache/fop/fonts/FontCache.java#FontCache.getDefaultCacheFile%28boolean%29

So it's possible that the cache file is being written by one process is being picked by another...

Revision history for this message
Tom Fifield (fifieldt) wrote :

That is a good theory indeed!

Changed in openstack-manuals:
status: Incomplete → Confirmed
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to openstack-manuals (master)

Fix proposed to branch: master
Review: https://review.openstack.org/47509

Changed in openstack-manuals:
assignee: nobody → Davanum Srinivas (DIMS) (dims-v)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to openstack-manuals (master)

Reviewed: https://review.openstack.org/47509
Committed: http://github.com/openstack/openstack-manuals/commit/dc7e0b27b1c94616b1671153668cd87844bbbab1
Submitter: Jenkins
Branch: master

commit dc7e0b27b1c94616b1671153668cd87844bbbab1
Author: Davanum Srinivas <email address hidden>
Date: Thu Sep 19 22:17:17 2013 -0400

    Possible fix for FontTriplet errors during heavy load

    Testing a theory about bad fop cache directory

    Fixes LP# 1221721

    Change-Id: Ic355f0d34b817ba52eb96b04b1ec0d2d1e7fe953

Changed in openstack-manuals:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.