ephemeral heat errors when trying to remove tirpleo deploy temp dir on failure

Bug #1947395 reported by Alex Schultz
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Triaged
High
Unassigned

Bug Description

I deployed with a bad resource_registry mapping to a file with a bad path and it errors. When trying to remove the heat tmpfs mount, it fails because it is still in use. There was a patch to do an unmount but it doesn't seem to work (as i have this patch)

https://review.opendev.org/c/openstack/python-tripleoclient/+/800021

Stack trace:

Exception: [Errno 16] Device or resource busy: '/home/stack/output/heat_launcher/tripleo_deploy-md0hq2tu'
Traceback (most recent call last):
  File "/usr/lib64/python3.6/urllib/request.py", line 1505, in open_local_file
    stats = os.stat(localfile)
FileNotFoundError: [Errno 2] No such file or directory: '/home/stack/deployment/tests/test-task-core.yaml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/heatclient/common/utils.py", line 415, in read_url_content
    content = request.urlopen(url).read()
  File "/usr/lib64/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib64/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/usr/lib64/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/usr/lib64/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib64/python3.6/urllib/request.py", line 1483, in file_open
    return self.open_local_file(req)
  File "/usr/lib64/python3.6/urllib/request.py", line 1522, in open_local_file
    raise URLError(exp)
urllib.error.URLError: <urlopen error [Errno 2] No such file or directory: '/home/stack/deployment/tests/test-task-core.yaml'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line 1943, in process_multiple_environments
    env_path=env_path, include_env_in_files=include_env_in_files)
  File "/usr/lib/python3.6/site-packages/heatclient/common/template_utils.py", line 331, in process_environment_and_files
    env_base_url)
  File "/usr/lib/python3.6/site-packages/heatclient/common/template_utils.py", line 369, in resolve_environment_urls
    is_object=is_object, object_request=object_request)
  File "/usr/lib/python3.6/site-packages/heatclient/common/template_utils.py", line 167, in get_file_contents
    file_content = utils.read_url_content(str_url)
  File "/usr/lib/python3.6/site-packages/heatclient/common/utils.py", line 417, in read_url_content
    raise exc.CommandError(_('Could not fetch contents for %s') % url)
heatclient.exc.CommandError: Could not fetch contents for file:///home/stack/deployment/tests/test-task-core.yaml

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.6/urllib/request.py", line 1505, in open_local_file
    stats = os.stat(localfile)
FileNotFoundError: [Errno 2] No such file or directory: '/home/stack/deployment/tests/test-task-core.yaml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/heatclient/common/utils.py", line 415, in read_url_content
    content = request.urlopen(url).read()
  File "/usr/lib64/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib64/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/usr/lib64/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/usr/lib64/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib64/python3.6/urllib/request.py", line 1483, in file_open
    return self.open_local_file(req)
  File "/usr/lib64/python3.6/urllib/request.py", line 1522, in open_local_file
    raise URLError(exp)
urllib.error.URLError: <urlopen error [Errno 2] No such file or directory: '/home/stack/deployment/tests/test-task-core.yaml'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1265, in _standalone_deploy
    parsed_args)
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 765, in _deploy_tripleo_heat_templates
    cleanup=parsed_args.cleanup)
  File "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line 1994, in process_multiple_environments
    env_path=f.name, include_env_in_files=include_env_in_files)
  File "/usr/lib/python3.6/site-packages/heatclient/common/template_utils.py", line 331, in process_environment_and_files
    env_base_url)
  File "/usr/lib/python3.6/site-packages/heatclient/common/template_utils.py", line 369, in resolve_environment_urls
    is_object=is_object, object_request=object_request)
  File "/usr/lib/python3.6/site-packages/heatclient/common/template_utils.py", line 167, in get_file_contents
    file_content = utils.read_url_content(str_url)
  File "/usr/lib/python3.6/site-packages/heatclient/common/utils.py", line 417, in read_url_content
    raise exc.CommandError(_('Could not fetch contents for %s') % url)
heatclient.exc.CommandError: Could not fetch contents for file:///home/stack/deployment/tests/test-task-core.yaml

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1433, in take_action
    self._standalone_deploy(parsed_args)
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1352, in _standalone_deploy
    self._kill_heat(parsed_args)
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 460, in _kill_heat
    self.heat_launch.kill_heat(self.heat_pid)
  File "/usr/lib/python3.6/site-packages/tripleoclient/heat_launcher.py", line 450, in kill_heat
    shutil.rmtree(self.install_dir)
  File "/usr/lib64/python3.6/shutil.py", line 490, in rmtree
    onerror(os.rmdir, path, sys.exc_info())
  File "/usr/lib64/python3.6/shutil.py", line 488, in rmtree
    os.rmdir(path)
OSError: [Errno 16] Device or resource busy: '/home/stack/output/heat_launcher/tripleo_deploy-md0hq2tu'
None

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.