Instance type is non-deterministic when deploying machines with cpu/mem constraints in aws

[Andrew Scribner]

While deploying the Charmed Kubernetes bundle a few times with Juju bootstrapped on AWS I noticed the instance types I was assigned would vary between deployment. For example, machines needing 4CPU/16GB RAM received t3.xlarge ($0.0832/hr), m5d.xlarge ($0.226/hr), or m5n.xlarge ($0.238/hr) depending on the deployment, a pretty large cost spread.

I could be wrong, but a quick look at the code made me suspect that (assuming Juju is taking cost into account when picking which type to deploy) Juju is querying the spot price* (eg: discount price for if I want an instance that AWS can kill and take back at any moment) rather than the on-demand price (which I'd argue is more likely what we'd be paying for when Juju deploys a machine). A look at the spot prices when deploying above did find that m5n.xlarge machines were the cheapest of those three when I received them.
* https://github.com/juju/juju/blob/5d0776d51bd313c9f0e12e307fd438904c9f6632/provider/ec2/instancetypes.go#L319

(aside: I do see guidance that I can specify the instance type directly (https://discourse.charmhub.io/t/tutorial-how-to-bootstrap-juju-on-awss-free-tier/1507), but still this seems like a bug as I think anyone setting fuzzy cpu/mem constraints would expect Juju to find the cheapest machine that suits)

[John A Meinel (jameinel)]

The issue is that from what we could tell, Amazon doesn't give a query
endpoint for on-demand prices. They have a large (~500MB) JSON blob that
can be downloaded to list all of their SKUs, but it is an expensive query
(and doesn't support a compressed request or transfer compression). We were
hoping that spot prices would provide a reasonable approximation, but it is
true that isn't a 1:1.

On Fri, Jan 21, 2022 at 10:05 AM Juan M. Tirado <email address hidden>
wrote:

> ** Changed in: juju
> Status: New => Triaged
>
> --
> You received this bug notification because you are subscribed to juju.
> Matching subscriptions: juju bugs
> https://bugs.launchpad.net/bugs/1958574
>
> Title:
> Instance type is non-deterministic when deploying machines with
> cpu/mem constraints in aws
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/1958574/+subscriptions
>
>

[Juan M. Tirado (tiradojm)]

This could be an interesting feature. I add it to the wishlist.

[Andrew Scribner (ca-scribner)]

Oh shoot if there's no easy endpoint to query then that makes sense. Could do something like maintain a simplified price list somewhere in Canonical land that can be queried, but that's more of a pain.

Alternative could be preferring certain classes of machine (take a t3 if it exists) or preferring certain hardware (take something that only has cpu/mem and no storage/gpu/...), but then you're stuck maintaining that list.

If it isn't called out somewhere in the docs, maybe a warning would be good to add so people don't get costs they don't expect.

fwiw I don't think aws is even consistent on their spot prices. The prices [here](https://aws.amazon.com/ec2/spot/pricing/) don't match the prices in the ones in the Console->Spot Requests->Price history. Not sure if I'm comparing the wrong things but it seems odd