os-brick does not automatically creates the hostnqn file even restart of nova_compute.
Code tries to run "nvme show-hostnqn" command and if hostnqn file does not exist it returns 'hostnqn is not available -- use nvme gen-hostnqn\n' with return code 254.
Return code 254 causes putils.ProcessExecutionError and this exception is handled on nvmeof.py -> create_hostnqn method. There is if-else block on exception handling section and latest else statement raises the exception again.
Because of raising the exception again, the create_hostnqn method does not continue to create a new hostnqn file.
Can we handle the error code 254 exception to create a new hostnqn file?
nova-compute log:
2023-09-14 11:35:47.153 534 WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: Unexpected error while running command.
Command: nvme show-hostnqn
Exit code: 254
Stdout: ''
Stderr: 'hostnqn is not available -- use nvme gen-hostnqn\n'
2023-09-14 11:35:47.159 534 WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: Unexpected error while running command.
Command: nvme show-hostnqn
Exit code: 254
Stdout: ''
Stderr: 'hostnqn is not available -- use nvme gen-hostnqn\n'
Code:https://github.com/openstack/os-brick/blob/8282b14889872d48ccc49312cb0901773b914c38/os_brick/privileged/nvmeof.py#L59
@os_brick.privileged.default.entrypoint
def create_hostnqn() -> str:
"""Create the hostnqn file to speed up finding out the nqn.
By having the /etc/nvme/hostnqn not only do we make sure that that value is
always used on this system, but we are also able to just open the file to
get the nqn on each get_connector_properties call instead of having to make
a call to nvme show-hostnqn command.
"""
host_nqn = ''
try:
os.makedirs('/etc/nvme', mode=0o755, exist_ok=True)
# Try to get existing nqn generated from dmi or systemd
try:
host_nqn, err = rootwrap.custom_execute('nvme', 'show-hostnqn')
host_nqn = host_nqn.strip()
# This is different from OSError's ENOENT, which is missing nvme
# command. This ENOENT is when nvme says there isn't an nqn.
except putils.ProcessExecutionError as e:
err_msg = e.stdout[:e.stdout.find('\n')]
show_hostnqn_subcmd_missing = (
"ERROR: Invalid sub-command".casefold() in err_msg.casefold())
if show_hostnqn_subcmd_missing:
LOG.debug('Version too old cannot check current hostnqn.')
elif e.exit_code == errno.ENOENT:
LOG.debug('No nqn could be formed from dmi or systemd.')
else:
LOG.debug('Unknown error from nvme show-hostnqn: %s', err_msg)
---- Here ----->raise
if not host_nqn:
LOG.debug('Generating nqn')
----------->host_nqn, err = rootwrap.custom_execute('nvme', 'gen-hostnqn')
host_nqn = host_nqn.strip()
with open('/etc/nvme/hostnqn', 'w') as f:
LOG.debug('Writing hostnqn file')
f.write(host_nqn)
os.chmod('/etc/nvme/hostnqn', 0o644)
except Exception as e:
LOG.warning("Could not generate host nqn: %s", e)
return host_nqn
Versions:
os-brick==6.1.1
host: Ubuntu 20.04.6 LTS (kernel 5.4.0-153-generic)
Full logs attached
We are using kolla-ansible and kolla nova_compute container does not mount the "/etc/nvme/" directory from host to container. Actualy we also do not want to make some services host files dependent when using containers.
"nvme-cli" package is installed to container image by kolla and "nvme-cli" package creates the /etc/nvme paths for container image.
But the problem is, hostnqn and hostid files embedded to the nova_compute images and all containers which are spawned from this image uses the same hostnqn file which is not unique.
kolla side solution will be deleting the "/etc/nvme/hostnqn" and "/etc/kolla/hostid" files from image after docker build of the container image **but** os_brick should handle the situation of absence of these files.
I am sharing the mounts of nova_compute container:
"Mounts": [
{
},
{
},
{
},
{
},
{
},
{
},
{
},
{
},
{
...