Skip to content

Improve the experience of running Docker locally #123678

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 17, 2024
Merged

Improve the experience of running Docker locally #123678

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9ed6e60
Use `isCI` in `run.sh`
dianqk Apr 9, 2024
d92f01a
Only output to `GITHUB_STEP_SUMMARY` in a CI environment
dianqk Apr 9, 2024
86e939e
Change the output artifact path to `$root_dir/obj/$image` when runnin…
dianqk Apr 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 4 additions & 7 deletions src/ci/docker/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,20 +14,17 @@ for example:
./src/ci/docker/run.sh x86_64-gnu
```

Images will output artifacts in an `obj` dir at the root of a repository. Note
Images will output artifacts in an `obj/$image_name` dir at the root of a repository. Note
that the script will overwrite the contents of this directory.

To match conditions in rusts CI, also set the environment variable `DEPLOY=1`, e.g.:
```
DEPLOY=1 ./src/ci/docker/run.sh x86_64-gnu
```

**NOTE**: Re-using the same `obj` dir with different docker images with
the same target triple (e.g. `dist-x86_64-linux` and `dist-various-1`)
may result in strange linker errors, due shared library versions differing between platforms.

If you encounter any issues when using multiple Docker images, try deleting your `obj` directory
before running your command.
**NOTE**: In CI, the script outputs the artifacts to the `obj` directory,
while locally, to the `obj/$image_name` directory. This is primarily to prevent
strange linker errors when using multiple Docker images.

## Filesystem layout

Expand Down
20 changes: 13 additions & 7 deletions src/ci/docker/run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,20 +33,24 @@ ci_dir="`dirname $script_dir`"
src_dir="`dirname $ci_dir`"
root_dir="`dirname $src_dir`"

objdir=$root_dir/obj
source "$ci_dir/shared.sh"

if isCI; then
objdir=$root_dir/obj
else
objdir=$root_dir/obj/$image
fi
dist=$objdir/build/dist


if [ -d "$root_dir/.git" ]; then
IS_GIT_SOURCE=1
fi

source "$ci_dir/shared.sh"

CACHE_DOMAIN="${CACHE_DOMAIN:-ci-caches.rust-lang.org}"

if [ -f "$docker_dir/$image/Dockerfile" ]; then
if [ "$CI" != "" ]; then
if isCI; then
hash_key=/tmp/.docker-hash-key.txt
rm -f "${hash_key}"
echo $image >> $hash_key
Expand Down Expand Up @@ -102,7 +106,7 @@ if [ -f "$docker_dir/$image/Dockerfile" ]; then
CACHE_IMAGE_TAG=${REGISTRY}/${REGISTRY_USERNAME}/rust-ci-cache:${cksum}

# On non-CI jobs, we don't do any caching.
if [[ "$CI" == "" ]];
if ! isCI;
then
retry docker build --rm -t rust-ci -f "$dockerfile" "$context"
# On PR CI jobs, we don't have permissions to write to the registry cache,
Expand Down Expand Up @@ -289,7 +293,7 @@ else
command=(/checkout/src/ci/run.sh)
fi

if [ "$CI" != "" ]; then
if isCI; then
# Get some needed information for $BASE_COMMIT
#
# This command gets the last merge commit which we'll use as base to list
Expand Down Expand Up @@ -339,7 +343,9 @@ docker \
rust-ci \
"${command[@]}"

cat $objdir/${SUMMARY_FILE} >> "${GITHUB_STEP_SUMMARY}"
if isCI; then
cat $objdir/${SUMMARY_FILE} >> "${GITHUB_STEP_SUMMARY}"
fi

if [ -f /.dockerenv ]; then
rm -rf $objdir
Expand Down
Loading