Yes, there's a possible window where your job can be finished executing all the commands, but the artifacts tarball is not yet available on the server. This is because the agent automatically takes care of that part for you. In fact, it's also smart enough to retry it if the upload fails for any reason. At the time, I was thinking it would be better to return control to the user as soon as the test run is complete, and let the agent take care of all this in the background. But if you have something monitoring the test run and trying to collect the results (like jenkins, etc), you want to get those artifacts before finishing. The way I deal with this in our jenkins jobs is to do something like this:
I'm not sure about making poll hang indefinitely until the artifacts are confirmed to be uploaded - it's actually possible that the agent has moved on to processing other test jobs if that happens, but maybe we can try once and if it fails, warn the user that they may need to wait for their artifacts to be available for download. I'm not sure if that really solves your problem though, because I think you may still need to handle the possibility that your artifacts are not there yet. But it might be ok for 99% of the cases, because in reality, the upload rarely if ever fails on the first try.
what do you think?
Yes, there's a possible window where your job can be finished executing all the commands, but the artifacts tarball is not yet available on the server. This is because the agent automatically takes care of that part for you. In fact, it's also smart enough to retry it if the upload fails for any reason. At the time, I was thinking it would be better to return control to the user as soon as the test run is complete, and let the agent take care of all this in the background. But if you have something monitoring the test run and trying to collect the results (like jenkins, etc), you want to get those artifacts before finishing. The way I deal with this in our jenkins jobs is to do something like this:
testflinger-cli artifacts ${{JOB_ID}} || (sleep 30 && testflinger-cli artifacts ${{JOB_ID}})
I'm not sure about making poll hang indefinitely until the artifacts are confirmed to be uploaded - it's actually possible that the agent has moved on to processing other test jobs if that happens, but maybe we can try once and if it fails, warn the user that they may need to wait for their artifacts to be available for download. I'm not sure if that really solves your problem though, because I think you may still need to handle the possibility that your artifacts are not there yet. But it might be ok for 99% of the cases, because in reality, the upload rarely if ever fails on the first try.
what do you think?