I've run in to this same issue using an Nvidia RTX A5000, however the Ampere cards present an additional issue. The SR-IOV VFs that the MDEVs use are not persisted after reboot either. I've worked around this by having a system service run before nova-compute to run the sriov-manage command provided by Nvidia to enable the VFs. I used "mdevctl define" to persist the mdev definitions and this has worked for me.
I've run in to this same issue using an Nvidia RTX A5000, however the Ampere cards present an additional issue. The SR-IOV VFs that the MDEVs use are not persisted after reboot either. I've worked around this by having a system service run before nova-compute to run the sriov-manage command provided by Nvidia to enable the VFs. I used "mdevctl define" to persist the mdev definitions and this has worked for me.