Amazon EKS optimized Ubuntu Linux AMIs - new instance family (m7a) fails to join cluster - bootstrap scripts lagging

Bug #2034002 reported by Raoni Timo de Castro Cambiaghi
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-images
Fix Released
High
Thomas Bechtold

Bug Description

## Description

AWS released a new instance family m7a - https://aws.amazon.com/ec2/instance-types/m7a/

This is not yet present in eni-max-pods.txt file

The bootstrap.sh file fails silently due to missing log function if the instance type is missing from the file.

## Steps to Reproduce

Launch an m7a instance with the EKS Ubuntu AMI in an EKS cluster.

It won't join the cluster and you'll get the following error on /var/log/user-data.log:
.....
/etc/eks/bootstrap.sh: line 420: log: command not found
Exited with error on line 420

It seems like the log function, introduced in this PR: https://github.com/awslabs/amazon-eks-ami/pull/1276 is missing in Ubuntu's AMI.

## More context

Adding the log function to bootstrap.sh, it fails in another step:
# /etc/eks/max-pods-calculator.sh --instance-type-from-imds --cni-version 1.10.0 --show-max-allowed
/etc/eks/max-pods-calculator.sh: line 84: imds: command not found
/etc/eks/max-pods-calculator.sh: line 85: imds: command not found

imds is a helpter script that was added in this PR: https://github.com/awslabs/amazon-eks-ami/pull/1044

The script exists under /usr/local/share/eks/imds but it's not in the path.

## What's the fix?

Merge latest EKS AMI bootstrap.sh: https://github.com/awslabs/amazon-eks-ami/blame/master/files/bootstrap.sh with Ubuntu's and make sure any helper scripts can be reached (sym links).

Tags: eks
Changed in cloud-images:
status: New → Confirmed
assignee: nobody → Thomas Bechtold (toabctl)
importance: Undecided → Medium
importance: Medium → High
Robby Pocase (rpocase)
Changed in cloud-images:
status: Confirmed → Fix Released
Revision history for this message
Robby Pocase (rpocase) wrote :

A little delayed, but this is available in serials 20230830 and newer. Please let us know if you see this again.

Revision history for this message
Raoni Timo de Castro Cambiaghi (raonitimo) wrote : Re: [Bug 2034002] Re: Amazon EKS optimized Ubuntu Linux AMIs - new instance family (m7a) fails to join cluster - bootstrap scripts lagging

Thanks very much for the fix.

On Thu, 14 Sept 2023, 5:01 am Robby Pocase, <email address hidden>
wrote:

> A little delayed, but this is available in serials 20230830 and newer.
> Please let us know if you see this again.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/2034002
>
> Title:
> Amazon EKS optimized Ubuntu Linux AMIs - new instance family (m7a)
> fails to join cluster - bootstrap scripts lagging
>
> Status in cloud-images:
> Fix Released
>
> Bug description:
> ## Description
>
> AWS released a new instance family m7a -
> https://aws.amazon.com/ec2/instance-types/m7a/
>
> This is not yet present in eni-max-pods.txt file
>
> The bootstrap.sh file fails silently due to missing log function if
> the instance type is missing from the file.
>
>
> ## Steps to Reproduce
>
> Launch an m7a instance with the EKS Ubuntu AMI in an EKS cluster.
>
> It won't join the cluster and you'll get the following error on
> /var/log/user-data.log:
> .....
> /etc/eks/bootstrap.sh: line 420: log: command not found
> Exited with error on line 420
>
> It seems like the log function, introduced in this PR:
> https://github.com/awslabs/amazon-eks-ami/pull/1276 is missing in
> Ubuntu's AMI.
>
> ## More context
>
> Adding the log function to bootstrap.sh, it fails in another step:
> # /etc/eks/max-pods-calculator.sh --instance-type-from-imds
> --cni-version 1.10.0 --show-max-allowed
> /etc/eks/max-pods-calculator.sh: line 84: imds: command not found
> /etc/eks/max-pods-calculator.sh: line 85: imds: command not found
>
> imds is a helpter script that was added in this PR:
> https://github.com/awslabs/amazon-eks-ami/pull/1044
>
> The script exists under /usr/local/share/eks/imds but it's not in the
> path.
>
> ## What's the fix?
>
> Merge latest EKS AMI bootstrap.sh: https://github.com/awslabs/amazon-
> eks-ami/blame/master/files/bootstrap.sh with Ubuntu's and make sure
> any helper scripts can be reached (sym links).
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/cloud-images/+bug/2034002/+subscriptions
>
>

--
**
** <https://www.canva.com/>
Empowering the world to design
We're hiring,
apply here <https://www.canva.com/careers/>! Check out the latest news and
learnings from our team on the Canva Newsroom
<https://www.canva.com/newsroom/news/>.
 <https://twitter.com/canva>

<https://facebook.com/canva>
 <https://au.linkedin.com/company/canva>

<https://twitter.com/canva>  <https://facebook.com/canva>
<https://www.linkedin.com/company/canva>
 <https://instagram.com/canva>

Revision history for this message
Wesley Yep (wesleyyep) wrote :

Hi Robby,

I work with Raoni and we have been getting issues due to the second point mentioned above on new c7i instance types:

"
Adding the log function to bootstrap.sh, it fails in another step:
# /etc/eks/max-pods-calculator.sh --instance-type-from-imds --cni-version 1.10.0 --show-max-allowed
/etc/eks/max-pods-calculator.sh: line 84: imds: command not found
/etc/eks/max-pods-calculator.sh: line 85: imds: command not found

imds is a helpter script that was added in this PR: https://github.com/awslabs/amazon-eks-ami/pull/1044

The script exists under /usr/local/share/eks/imds but it's not in the path."

Is there a chance that this secondary part for adding "imds" was missed from the fix you mentioned in serials 20230830 and newer?

Let me know if you would prefer us to raise a second ticket for this.

Revision history for this message
Thomas Bechtold (toabctl) wrote :

Hi Wesley,

I see. Working on a fix now. Thanks for reporting this!

Changed in cloud-images:
status: Fix Released → In Progress
tags: added: eks
Changed in cloud-images:
status: In Progress → Fix Committed
Revision history for this message
Thomas Bechtold (toabctl) wrote :

@Wesley, this should be fixed for images with serial >= 20230922 . Please let us know if you still have problems.

Changed in cloud-images:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.