I set up an environment on rdocloud which ran the toci script on top of pre-provisioned nodes (so almost exact reproducer of CI).
I then made a small script that would run tempest over and over, and record the results. It ran 100 times consecutively without failing. So there is definitely an environmental component to this issue. Maybe need to artificially create some memory or cpu pressure.
I set up an environment on rdocloud which ran the toci script on top of pre-provisioned nodes (so almost exact reproducer of CI).
I then made a small script that would run tempest over and over, and record the results. It ran 100 times consecutively without failing. So there is definitely an environmental component to this issue. Maybe need to artificially create some memory or cpu pressure.