so far, i've been unable to reproduce. my setup:

host: 
ubuntu 20.04
Docker version 24.0.7, build 24.0.7-0ubuntu2~20.04.1
docker-buildx 0.12.1-0ubuntu1~20.04.1

1. t4g.large arm64 ubuntu 20.04 VM on aws ec2
2. ssh in and:
sudo apt update && sudo apt install docker.io docker-buildx
3. copy in the minimal Dockerfile above
4. sudo docker buildx build --platform linux/arm/v7 -f Dockerfile . | tee build.out
[+] Building 13.6s (7/7) FINISHED                                                                                                                                                                           docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                                                  0.0s
 => => transferring dockerfile: 169B                                                                                                                                                                                  0.0s
 => [internal] load .dockerignore                                                                                                                                                                                     0.0s
 => => transferring context: 2B                                                                                                                                                                                       0.0s
 => [internal] load metadata for docker.io/library/ubuntu:20.04                                                                                                                                                       0.4s
 => CACHED [1/3] FROM docker.io/library/ubuntu:20.04@sha256:0b897358ff6624825fb50d20ffb605ab0eaea77ced0adb8c6a4b756513dec6fc                                                                                          0.0s
 => => resolve docker.io/library/ubuntu:20.04@sha256:0b897358ff6624825fb50d20ffb605ab0eaea77ced0adb8c6a4b756513dec6fc                                                                                                 0.0s
 => [2/3] RUN apt-get update && apt-get -y install ca-certificates wget                                                                                                                                              12.5s
 => [3/3] RUN wget -O - https://helloworld.letsencrypt.org                                                                                                                                                            0.5s 
 => exporting to image                                                                                                                                                                                                0.2s 
 => => exporting layers                                                                                                                                                                                               0.2s 
 => => writing image sha256:e6c913231eb0dc5ab2476c9d562eae87c8782479c4ca35e08c59a0cdf9b32342

Can you provide more information? Build host version, docker version, buildx version?