Please provide docker hub tags for ARM64

[ghost]

There are no official powershell docker tags for arm64, whereas the underlying dependency dotnet has multiple architectures in the manifests. For example:

$ docker run --rm -it mcr.microsoft.com/dotnet/runtime:5.0 uname -a
Linux 88af77df151b 5.4.0-1022-raspi #25-Ubuntu SMP PREEMPT TIMESTAMP aarch64 GNU/Linux
$ docker run --rm -it mcr.microsoft.com/dotnet/runtime:5.0-alpine uname -a
Linux a68ba3a4fce9 5.4.0-1022-raspi #25-Ubuntu SMP PREEMPT TIMESTAMP aarch64 Linux

Please offer docker tags for arm64 containers of Powershell. Ideally, apply an fxdependent layer atop the dotnet images to save space client-side.

This will become increasingly important as Apple M1-based devices begin shipping in the wild to customers.

Steps to reproduce

docker run --rm -it mcr.microsoft.com/powershell

Expected behavior

docker run --rm -it mcr.microsoft.com/powershell
PowerShell x.x.x
Copyright (c) Microsoft Corporation. All rights reserved.

https://aka.ms/powershell
Type 'help' to get help.

PS /> 

Actual behavior

docker run --rm -it mcr.microsoft.com/powershell
Unable to find image 'mcr.microsoft.com/powershell:latest' locally
latest: Pulling from powershell
docker: no matching manifest for linux/arm64/v8 in the manifest list entries.
See 'docker run --help'.

Environment data

$ docker info
Client:
 Debug Mode: false

Server:
 Containers: 5
  Running: 0
  Paused: 0
  Stopped: 5
 Images: 86
 Server Version: 19.03.13
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8fba4e9a7d01810a393d5d25a3621dc101981175
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.4.0-1022-raspi
 Operating System: Ubuntu 20.04.1 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 3.704GiB
 Name: kapi
 ID: MPJM:3WIN:V7AA:LXFC:RCRC:NQ2Q:ILAU:CUKC:DFHC:Q4AT:JQ7E:Q734
 Docker Root Dir: /mnt/firecuda/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory limit support
WARNING: No kernel memory TCP limit support
WARNING: No oom kill disable support

[RDIL]

If I remember correctly, this wasn't done due to a limitation with the CI I believe. cc @TravisEz13

edit: issue label bot where you at?

[TravisEz13]

The question is does qemu support ARM64? We won't be able to test the image, but we have an ARM32 image that has to run in QEMU

[ghost]

I'm not versed in QEMU, but a quick search says it does support arm64. The dotnet team already solved for this; there's probably an opportunity to piggyback atop how they did it rather than reinvent it.

Otherwise, source one of these native arm64 machines:
Surface Pro X (Windows arm64)
M1 MacBook Air (macOS arm64)
Raspberry pi 4 (Ubuntu arm64, what I tested with above)

[TravisEz13]

The solution for windows is to use the .NET SDK images, which include PowerShell.

is this for Windows or Linux?

[TravisEz13]

We don't have the dedicated hardware that the .NET teams has. Resolving this as answered as it appears to be a solved problem.

[ghost]

The solution for windows is to use the .NET SDK images, which include PowerShell.

is this for Windows or Linux?

Linux docker images.

If the solution to use the .NET SDK docker images instead of PowerShell docker images was satisfactory, why not deprecate mcr.microsoft.com/powershell then?

[TravisEz13]

Windows arm cannot be emulated currently. We don't have a solution to build the images. .NET has the hardware required to build the images. Hopefully at some point, Azure will provide a windows ARM service and we will be able to build the windows images.

In the meantime, we can try to provide the Linux arm64 images.

[pcgeek86]

Hey @TravisEz13 would this be an easy thing to implement?

[TravisEz13]

@anamnavi We should investigate if there is something we can do here.

[potatoqualitee]

fwiw, I ended up using this within my Dockerfile

COPY --from=mcr.microsoft.com/dotnet/sdk:3.1-bionic-arm64v8 /usr/share/powershell /bin

It's in the builder stage so it's not a deal where it goes. looking forward to when you all have the functionality to build and test 👍🏼

[potatoqualitee]

JK it turns out that sdk has a much older version of PowerShell so I created this Dockerfile

# use the 730MB sdk image to build
FROM mcr.microsoft.com/dotnet/sdk as builder

# install the latest version of PowerShell
RUN dotnet tool install -g PowerShell

# discard all that builder data then just copy the required changed files from "builder" to the smaller 180MB base image
FROM mcr.microsoft.com/dotnet/runtime
COPY --from=builder /root/.dotnet/tools/ /bin

# run powershell
ENTRYPOINT pwsh

[TravisEz13]

I just discussed getting an agent that supports ARM64. We should be able to support this, but it will take some time.

[pcgeek86]

Tried running PowerShell ARM64 container image in a multi-architecture k3s Kubernetes cluster, on Amazon Web Services (AWS), using an AMD64 master node, and an ARM64 worker node. The ARM64 worker node failed to pull an ARM64 container image for PowerShell. Below is the result.

Name:         pwsh
Namespace:    default
Priority:     0
Node:         ip-172-31-52-227/172.31.52.227
Start Time:   Sat, 07 May 2022 04:01:36 +0000
Labels:       run=pwsh
Annotations:  <none>
Status:       Pending
IP:           10.42.1.4
IPs:
  IP:  10.42.1.4
Containers:
  pwsh:
    Container ID:
    Image:          mcr.microsoft.com/powershell
    Image ID:
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-fvbg6 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  kube-api-access-fvbg6:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  84s                default-scheduler  Successfully assigned default/pwsh to ip-172-31-52-227
  Normal   Pulling    41s (x3 over 84s)  kubelet            Pulling image "mcr.microsoft.com/powershell"
  Warning  Failed     41s (x3 over 84s)  kubelet            Failed to pull image "mcr.microsoft.com/powershell": rpc error: code = NotFound desc = failed to pull and unpack image "mcr.microsoft.com/powershell:latest": no match for platform in manifest: not found
  Warning  Failed     41s (x3 over 84s)  kubelet            Error: ErrImagePull
  Normal   BackOff    3s (x5 over 83s)   kubelet            Back-off pulling image "mcr.microsoft.com/powershell"
  Warning  Failed     3s (x5 over 83s)   kubelet            Error: ImagePullBackOff

[TravisEz13]

@pcgeek86 the ARM tag is arm32v7-ubuntu-bionic. We are working to get better ARM availability.

[pcgeek86]

Looks like ARM is missing Ubuntu 22.04 Jammy Jellyfish images.

The Intel / AMD 64-bit version works:

docker pull mcr.microsoft.com/powershell:ubuntu-jammy

The ARM version doesn't work:

docker pull mcr.microsoft.com/powershell:arm32v7-ubuntu-jammy

[github-actions]

@TravisEz13 set a reminder for 11/5/2022

[github-actions]

👋 @TravisEz13,

[vstepko]

@TravisEz13 Hi! Do you have any updates on this?

[TravisEz13]

I believe we are unblocked but at this point we need to schedule the work. I'm hoping can get this done the first quarter of next year.

[TravisEz13]

/remind me in 30 days

[github-actions]

@TravisEz13 set a reminder for 1/15/2023

[bdovaz]

@TravisEz13 Do you have any updates on this?

[github-actions]

👋 @TravisEz13,

[mayerlench]

Any update on this?

[pcgeek86]

Amazing how it takes Microsoft literally years to simply add a new tag to a container image. If I had the necessary access to update the Microsoft Container Registry, I could have gotten this done months ago.

1. Spin up ARM64 Linux virtual machine in the cloud
2. Run container image build
3. Push to registry
4. Destroy VM

AWS has offered Graviton CPUs since 2018. Source: https://en.wikipedia.org/wiki/AWS_Graviton

I'd be happy to run the build on my own ARM64 devices as well. Like I said, just need the access to do so.

AWS Graviton - Wikipedia

[TravisEz13]

#729 is in progress to address this issue.
Support has been added to the CI system.

[Nickcandy]

@TravisEz13 Hi Travis, I'm from AzPs team. Do you get any updates on this? We are waiting for this for a long time

[TravisEz13]

Images are released:

[emil2k]

@TravisEz13 Is there a plan to add arm64 images for any of the other distros (i.e. debian, ubuntu)?

Also, we noticed on your DockerHub page that some images are being labeled arm64 even though they are providing an arm32 v7 build:

[anamnavi]

@emil2k thanks for pointing out the 2nd point, I'll work to correct this to just show arm

[TravisEz13]

Please file an issue for the images that are building with the wrong platform. We probably need to specify the platform in the FROM statement in the dockerfile.

[TravisEz13]

cc @anamnavi

[TravisEz13]

Currently, we are only testing mariner with ARM64. Please file an issue in the PowerShell repo with which distro/architecture you would like supported and how you are using/planning on using it.

You can always build you own image using the tar.gz, but I cannot put an image for a distro we don't support in MCR.

[pcgeek86]

docker pull mcr.microsoft.com/powershell
Using default tag: latest
latest: Pulling from powershell
no matching manifest for linux/arm64/v8 in the manifest list entries

FYI the latest tag doesn't work for arm64 devices. It works fine for AMD / Intel 64-bit systems though. As a matter of convenience, the latest tag is generally used for container images, so that you don't have to spend extra effort looking up specific tags (ie. nginx:latest, mysql:latest, python:latest, node:latest, etc. etc.).

[potatoqualitee]

I don't know if you all are familiar with how to do this (maybe everyone is and I don't know, or maybe it's how it's done with other arches) but in case... this is how I do it for my own setup.

      - name: Build docker images
        run: |
          cd ./sqlinstance
          docker compose up -d
      - name: Tag images
        run: |
          docker tag dbatools/sqlinstance dbatools/sqlinstance:latest-amd64
          docker tag dbatools/sqlinstance2 dbatools/sqlinstance2:latest-amd64
      - name: Push images
        run: |
          docker push dbatools/sqlinstance:latest-amd64
          docker push dbatools/sqlinstance2:latest-amd64
      # push one multiarch image
      - name: Create and push multiarch manifest
        run: |
          docker manifest create dbatools/sqlinstance:latest --amend dbatools/sqlinstance:latest-amd64 --amend dbatools/sqlinstance:latest-arm64
          docker manifest create dbatools/sqlinstance2:latest --amend dbatools/sqlinstance2:latest-amd64 --amend dbatools/sqlinstance2:latest-arm64
      - name: Push manifest
        run: |
          docker manifest push docker.io/dbatools/sqlinstance:latest --purge
          docker manifest push docker.io/dbatools/sqlinstance2:latest --purge
      - name: View manifest
        run: |
          docker manifest inspect docker.io/dbatools/sqlinstance:latest
          docker manifest inspect docker.io/dbatools/sqlinstance2:latest