[RFC]: add devcontainer support for arm64 architectures

[anandkaranubc]

Description

This RFC proposes adding ARM64 architecture support to DevContainers to improve compatibility and extend usability for ARM64-based systems.

Related Issues

None

Questions

No.

Other

The current DevContainer support uses the mcr.microsoft.com/devcontainers/universal:2 Docker image, but it lacks native support for ARM64 architectures.

stdlib/.devcontainer/devcontainer.json

Line 3 in 9f88c8c

"image": "mcr.microsoft.com/devcontainers/universal:2",

This is what happens when you run docker pull mcr.microsoft.com/devcontainers/universal:2 on Apple Silicon:

Error response from daemon: no matching manifest for linux/arm64/v8 in the manifest list entries: no match for platform in manifest: not found

Related links to support that:

• mcr.microsoft.com/devcontainers/universal:2 for M1 Mac devcontainers/images#545
• M series support (macOS) for devcontainer All-Hands-AI/OpenHands#5414
• Support arm64 architecture on Oryx microsoft/Oryx#2133

Based on the above links, there doesn't seem to be an active attempt to make the universal image compatible with arm64.

However for stdlib development, in order to support DevContainer environments on Apple Silicon, alternative Docker images compatible with ARM64 can be explored, such as mcr.microsoft.com/devcontainers/base with Ubuntu or Debian.

I explored the possibility of using "image": "mcr.microsoft.com/devcontainers/base:debian-11" and updating the features accordingly (using node:1 and r-rig:1 instead of "r-apt:0"). The initial results seem promising. However, I believe a comprehensive design approach is needed to selectively support both images based on the developer's architecture. This also includes a thorough testing strategy to check if everything works as expected after introducing the second docker image.

Any advice on this would be greatly appreciated, and I am happy to take on this issue.

Checklist

• I have read and understood the Code of Conduct.
• Searched for existing issues and pull requests.
• The issue name begins with RFC:.

[kgryte]

I can definitely see a need, so thank you for opening this issue. Maybe @Planeshifter has some thoughts since he originally created the container files.

[anandkaranubc]

Some preliminary findings:

• GitHub Codespaces: Codespaces can be used with the devcontainer environment. It takes a long time to set up, but the process is automatic. However, Codespaces is paid. Not a great solution.

• PyTorch: They use Miniconda (mcr.microsoft.com/vscode/devcontainers/miniconda:0-3) with LLVM/Clang and CUDA support. They target both CPU and GPU, but I don’t see any features for R, Julia, or JS. That makes sense because the Miniconda image is not ideal for those languages and is also not meant for their use case.

• JAX: They don’t use devcontainer, but they do use Docker for running JAX-related tasks. It’s not for development, but for automated testing in CI/CD pipelines. The script checks the OS (x86_64/ARM64/Windows) and automatically selects the correct Docker image for building and testing.

• Apache Spark, Polar, Boost: No use of devcontainer.

• Pandas: They use a Docker image, but instead of pulling a prebuilt dev container from Microsoft’s dev container library, they build their own image from a base image (python:3.10.8).

• SciPy: Uses the same Docker image as stdlib.

• OpenVSX: Uses Gitpod.

• Julia: Uses the official Julia Docker image (docker.io/library/julia:latest).

• Sage: This is the most interesting one. They use different Docker images and take advantage of the VS Code functionality that supports multiple devcontainer.json files.

This comes down to two possible approaches:

1. Update the current devcontainer.json to use a Docker image (such as Debian or Ubuntu) that works for both architectures.
2. Create a separate devcontainer.json file for ARM64 architecture development.

Adding to the development aspect, is there a way to test whether the updated devcontainer implementation works as expected for EVERYTHING?

[anandkaranubc]

@Planeshifter Open to your feedback on this. Thanks!

[kgryte]

@anandkaranubc Thanks for the overview! Having multiple devcontainer files seems sensible to me, but I don't know what that user experience is actually like. I'm curious to hear what @Planeshifter thinks, as well.

[Planeshifter]

From a usability perspective, having multiple devcontainer files should be fine. When creating them with custom options on the GitHub UI, there is a dropdown to select a container:

Similarly, when using the Dev Containers extension in VS Code, one can manually switch between them via the command palette.

As for any kind of automated CI/CD testing, this is not something I looked into. Docs only seem to mention manual testing via Codespaces, but maybe there is a way to easily set up some kind of integration tests too to test that various things have been properly setup. Not convinced the effort would be worth it, though.