When a step is dirty, snapcraft doesn't autoclean and reexecute it

[Leo Arias]

In a python snap, I changed the python-version and then re-run the snapcraft command. I got this:

The 'pull' step of 'my-part' is out of date:

The 'python-version' part property appears to have changed.

Please clean that part's 'pull' step in order to continue

I think that this message should never be displayed to the user. Instead, we should notice that the part is dirty, clean it, and continue.

[Kyle Fazzari (kyrofa)]

I have mixed feelings about this, which is one of the reasons we've been so conservative about it. Take this scenario:

I'm building mysql on a raspberry pi 2 (something I was doing back in the day). It takes HOURS, most of the workday in fact. It finally finishes building, and I realize that I'm missing a stage package. Since it's related to the mysql part I add it to the mysql part's stage-packages. I then run snapcraft again. Here's where reality splits:

1) We have today's snapcraft. It says "The 'pull' step of 'mysql' is out of date. Blah blah you added a stage package, please clean that part's step in order to continue." I think "oh, that's not what I want, I just spent all day building it! I'll add another part with this stage package instead."

2) We have the auto-cleaning snapcraft that you suggest. It automatically cleans the mysql step all the way back to pull and starts over. I throw my hands up in frustration and have lost a day's build.

I admit: neither of these realities is good. They both provide bad experiences. Is there a middle ground somewhere? Would it be terrible to prompt?

[Leo Arias (elopio)]

Middle ground would be to ask if you want to clean and continue. Optionally with a -f flag to force the rebuild. I would be ok with that.

However, I still think that it should just autoclean; but maybe that's because when I change something in the yaml I know what I'm doing, and also I'm usually ok to wait. So it's not good to base this feature on my opinion.

As a middle ground of the middle ground, could we put the duration of the last build in a state file and print the warning only if it's potentially too slow?

[Cris Dywan (kalikiana)]

Just throwing another idea out there: what if snapcraft realized that it's just one more package and didn't need to clean at all? All that'd be needed for that is to have cached the old description of the part and see "aha, one new package".

[Kyle Fazzari (kyrofa)]

> Just throwing another idea out there: what if snapcraft realized that it's just one more package and didn't need to clean at all?

Well, stage packages can be used in order to build and pull, so snapcraft ties them to the pull step in order to enable such use-cases. But that was really just a made up example-- as snapcraft gets smarter this could be a file that changed. And not just in mysql, but maybe you updated a comment in a file for a part mysql depended upon, in which case an automatic clean would need to affect the entire dependency tree.

[Tim Süberkrüb (tim-sueberkrueb)]

Just a small backlink to this forum post about what could improve the snapcraft clean experience for bigger snaps: https://forum.snapcraft.io/t/better-snapcraft-clean-experience-for-big-snaps/979.