Skip to content

Commit 98522f2

Browse files
MaxDesiatovkateinoigakukun
MaxDesiatov
authored and
kateinoigakukun
committed
Move packaging scripts from swiftwasm-package-sdk (#18)
Currently our CI scripts rely on validity of scripts in the [`swiftwasm-package-sdk`](https://github.com/swiftwasm/swiftwasm-package-sdk) repository. That can't always be the case, especialy as we don't checkout a specific commit in that repository. And even if we did, managing this isn't convenient. Adding a submodule has its own set of issues and I personally think that a monorepo approach is much simpler for a small set of scripts, at least at an early stage. In fact, it would make sense to have these scripts in the upstream [`swift`](https://github.com/apple/swift) repository at some point, similarly to [what Android people have previously done](https://github.com/apple/swift/tree/master/utils/android). Thus, these scripts and a few small helper files are copied to `utils/webassembly` directory and are executed directly on CI. After this PR is merged, I don't see a particular need for our [`swiftwasm-package-sdk`](https://github.com/swiftwasm/swiftwasm-package-sdk) and [`swiftwasm-sdk`](https://github.com/swiftwasm/swiftwasm-sdk) repos, those probably can be archived. As a small cleanup addition, `.github/workflows/main.yml` file now has consistent indentation. * Move packaging scripts from swiftwasm-package-sdk * Rename `wasm` directory to `webassembly` * Make all .sh scripts executable after download * Make sdkroot/swiftwasm script executable * Add newline to build-mac-package.sh * Add newline to build-packages.sh * Remove swift_start.o and swift_end.o
1 parent 16de707 commit 98522f2

19 files changed

+321
-25
lines changed

Diff for: ‎.github/workflows/main.yml

+35-25
Original file line numberDiff line numberDiff line change
@@ -3,45 +3,50 @@ name: CI
33
on:
44
push:
55
branches:
6-
- swiftwasm
6+
- swiftwasm
77
pull_request:
88
branches:
9-
- swiftwasm
9+
- swiftwasm
1010

1111
jobs:
1212
linux_build:
1313
timeout-minutes: 0
1414
runs-on: ubuntu-18.04
1515

1616
steps:
17-
- uses: actions/checkout@v1
18-
- name: Build Linux installable archive
19-
run: ./ci-linux.sh
20-
- name: Upload Linux installable archive
21-
uses: actions/upload-artifact@v1
22-
with:
23-
name: linux-installable
24-
path: ../swiftwasm-linux.tar.gz
17+
- uses: actions/checkout@v1
18+
- name: Build Linux installable archive
19+
run: ./ci-linux.sh
20+
- name: Upload Linux installable archive
21+
uses: actions/upload-artifact@v1
22+
with:
23+
name: linux-installable
24+
path: ../swiftwasm-linux.tar.gz
2525

2626
macos_build:
2727
timeout-minutes: 0
2828
runs-on: macOS-10.14
2929

3030
steps:
31-
- uses: actions/checkout@v1
32-
- name: Build macOS installable archive
33-
run: ./ci-mac.sh
34-
- name: Upload macOS installable archive
35-
uses: actions/upload-artifact@v1
36-
with:
37-
name: macos-installable
38-
path: ../swiftwasm-macos.tar.gz
31+
- uses: actions/checkout@v1
32+
- name: Build macOS installable archive
33+
run: ./ci-mac.sh
34+
- name: Upload macOS installable archive
35+
uses: actions/upload-artifact@v1
36+
with:
37+
name: macos-installable
38+
path: ../swiftwasm-macos.tar.gz
39+
- name: Upload packaging scripts
40+
uses: actions/upload-artifact@v1
41+
with:
42+
name: packaging-scripts
43+
path: utils/webassembly
3944

4045
package:
4146
name: Build SwiftWasm packages
4247
needs:
43-
- linux_build
44-
- macos_build
48+
- linux_build
49+
- macos_build
4550
runs-on: ubuntu-18.04
4651
steps:
4752
- name: Download installable Linux archive
@@ -52,11 +57,16 @@ jobs:
5257
uses: actions/download-artifact@v1
5358
with:
5459
name: macos-installable
60+
- name: Download packaging scripts
61+
uses: actions/download-artifact@v1
62+
with:
63+
name: packaging-scripts
5564
- name: Build the packages
5665
shell: bash
5766
run: |
58-
git clone https://github.com/swiftwasm/swiftwasm-package-sdk.git
59-
cd swiftwasm-package-sdk
67+
cd packaging-scripts
68+
find . -name '*.sh' -exec chmod +x {} \;
69+
chmod +x sdkroot/swiftwasm
6070
./download-prebuilts.sh
6171
6272
cp ../linux-installable/swiftwasm-linux.tar.gz \
@@ -74,19 +84,19 @@ jobs:
7484
uses: actions/upload-artifact@v1
7585
with:
7686
name: macos-package
77-
path: swiftwasm-package-sdk/output/swiftwasm-sdk-macos.tar.xz
87+
path: packaging-scripts/output/swiftwasm-sdk-macos.tar.xz
7888

7989
- name: Upload Linux package
8090
uses: actions/upload-artifact@v1
8191
with:
8292
name: linux-package
83-
path: swiftwasm-package-sdk/output/swiftwasm-sdk-linux.tar.xz
93+
path: packaging-scripts/output/swiftwasm-sdk-linux.tar.xz
8494

8595
- name: Upload hello.wasm compiled with Linux package
8696
uses: actions/upload-artifact@v1
8797
with:
8898
name: linux-hello.wasm
89-
path: swiftwasm-package-sdk/output/swiftwasm-sdk/hello.wasm
99+
path: packaging-scripts/output/swiftwasm-sdk/hello.wasm
90100

91101
macos_smoke_test:
92102
name: Compile hello.swift on macOS

Diff for: ‎utils/webassembly/.gitignore

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
compiler
2+
swiftwasm-sdk
3+
prebuilt
4+
output
5+
tmpdir

Diff for: ‎utils/webassembly/README.md

+23
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
Creates packages containing everything needed to build WebAssembly programs with Swift.
2+
3+
# Building
4+
5+
```
6+
./download-prebuilts.sh
7+
./download-installable-prebuilts.sh
8+
./build-packages.sh
9+
```
10+
11+
# Contents of package
12+
13+
- Swift toolchain from [swiftwasm-sdk](https://github.com/swiftwasm/swiftwasm-sdk)
14+
- WASI modified sysroot from [wasi-sdk](https://github.com/swiftwasm/wasi-sdk)
15+
- libicu from [icu4c-wasi](https://github.com/swiftwasm/icu4c-wasi)
16+
- linking helpers from [swiftwasm-wasi-stubs](https://github.com/swiftwasm/swiftwasm-wasi-stubs)
17+
- wasi-ld, either from wasi-sdk (on Linux) or upstream LLVM 9.0 (on Mac)
18+
- build script for compiling a Swift file to a .wasm
19+
- a Getting Started guide
20+
21+
# Notes
22+
23+
This shares a lot with the [swiftwasm-compile-service](https://github.com/swiftwasm/swiftwasm-compile-service).

Diff for: ‎utils/webassembly/build-linux-package.sh

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/bin/bash
2+
3+
echo "Unpacking Linux prebuilts"
4+
mkdir -p output
5+
cd linux
6+
./unpack-prebuilts.sh
7+
echo "Compressing"
8+
tar cJf ../output/swiftwasm-sdk-linux.tar.xz swiftwasm-sdk
9+
cd ..

Diff for: ‎utils/webassembly/build-mac-package.sh

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/bin/bash
2+
3+
echo "Unpacking macOS prebuilts"
4+
mkdir -p output
5+
cd macos
6+
./unpack-prebuilts.sh
7+
echo "Compressing macOS package"
8+
tar cJf ../output/swiftwasm-sdk-macos.tar.xz swiftwasm-sdk
9+
cd ..

Diff for: ‎utils/webassembly/build-packages.sh

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
#!/bin/bash
2+
set -e
3+
rm -rf output || true
4+
./build-linux-package.sh
5+
./build-mac-package.sh

Diff for: ‎utils/webassembly/copy-shared-files.sh

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
#!/bin/bash
2+
set -e
3+
cp -r ../sdkroot/* compiler/
4+
cp ../linux/compiler/opt/swiftwasm-sdk/lib/swift/wasm/wasm32/glibc.modulemap compiler/extra_utils

Diff for: ‎utils/webassembly/download-installable-prebuilts.sh

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
#!/bin/bash
2+
set -e
3+
mkdir -p prebuilt
4+
cd prebuilt
5+
wget -O swiftwasm-linux.tar.gz https://github.com/swiftwasm/swiftwasm-sdk/releases/download/20191112.1.linux/swiftwasm.tar.gz
6+
# Mac specific
7+
wget -O swiftwasm-macos.tar.gz https://github.com/swiftwasm/swiftwasm-sdk/releases/download/20191112.1.mac/swiftwasm-mac.tar.gz

Diff for: ‎utils/webassembly/download-prebuilts.sh

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
#!/bin/bash
2+
set -e
3+
mkdir -p prebuilt
4+
cd prebuilt
5+
wget https://github.com/swiftwasm/wasi-sdk/releases/download/20191022.1/wasi-sdk-4.39g3025a5f47c04-linux.tar.gz
6+
wget https://github.com/swiftwasm/icu4c-wasi/releases/download/20190421.3/icu4c-wasi.tar.xz
7+
# Mac specific
8+
wget http://releases.llvm.org/9.0.0/clang+llvm-9.0.0-x86_64-darwin-apple.tar.xz

Diff for: ‎utils/webassembly/linux/unpack-prebuilts.sh

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
#!/bin/bash
2+
set -e
3+
rm -rf swiftwasm-sdk compiler
4+
mkdir swiftwasm-sdk
5+
ln -s swiftwasm-sdk compiler
6+
cd compiler
7+
untar="../../prebuilt/wasi-sdk-"*"-linux.tar.gz
8+
../../prebuilt/swiftwasm-linux.tar.gz
9+
../../prebuilt/icu4c-wasi.tar.xz"
10+
for i in $untar
11+
do
12+
echo $i
13+
tar xf $i
14+
done
15+
cd ..
16+
mv "compiler/wasi-sdk-"* "compiler/wasi-sdk"
17+
mv compiler/wasi-sdk/share/wasi-sysroot compiler/wasi-sdk/share/sysroot
18+
../remove-swift-extra-files.sh || true
19+
../remove-wasi-extra-files.sh || true
20+
../copy-shared-files.sh || true

Diff for: ‎utils/webassembly/macos/unpack-prebuilts.sh

+30
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
#!/bin/bash
2+
set -e
3+
rm -rf swiftwasm-sdk compiler tmpdir
4+
mkdir swiftwasm-sdk tmpdir
5+
ln -s swiftwasm-sdk compiler
6+
cd compiler
7+
untar="../../prebuilt/wasi-sdk-"*"-linux.tar.gz
8+
../../prebuilt/swiftwasm-macos.tar.gz
9+
../../prebuilt/icu4c-wasi.tar.xz"
10+
for i in $untar
11+
do
12+
echo $i
13+
tar xf $i
14+
done
15+
# Mac: unpack the Linux one and copy stdlibs
16+
cd ..
17+
cd tmpdir
18+
tar xf ../../prebuilt/clang+llvm-*-x86_64-darwin-apple.tar.xz
19+
tar xf ../../prebuilt/swiftwasm-linux.tar.gz
20+
cd ..
21+
mv "compiler/wasi-sdk-"* "compiler/wasi-sdk"
22+
mv compiler/wasi-sdk/share/wasi-sysroot compiler/wasi-sdk/share/sysroot
23+
../remove-swift-extra-files.sh || true
24+
rm -r compiler/wasi-sdk/bin
25+
mkdir compiler/wasi-sdk/bin
26+
cp tmpdir/clang+llvm-*-x86_64-darwin-apple/bin/wasm-ld compiler/wasi-sdk/bin
27+
cp -a tmpdir/opt/swiftwasm-sdk/lib/swift/wasm compiler/opt/swiftwasm-sdk/lib/swift/wasm
28+
cp -a tmpdir/opt/swiftwasm-sdk/lib/swift_static compiler/opt/swiftwasm-sdk/lib/swift_static
29+
# ok, finally copy over the shared files
30+
../copy-shared-files.sh || true

Diff for: ‎utils/webassembly/remove-swift-extra-files.sh

+26
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
#!/bin/bash
2+
basepath="compiler/opt/swiftwasm-sdk"
3+
filestoremove="bin/sil-*
4+
bin/lldb*
5+
bin/sourcekitd-*
6+
bin/swift-api-digester
7+
bin/swift-autolink-extract
8+
bin/swift-demangle
9+
bin/swift-demangle-yamldump
10+
bin/swift-format
11+
bin/swift-llvm-opt
12+
bin/swift-refactor
13+
bin/swift-reflection-dump
14+
bin/swift-*-test
15+
lib/libsourcekitdInProc.so
16+
lib/swift/clang/lib/linux/*
17+
lib/swift_static/linux/*
18+
lib/swift/linux/x86_64/*
19+
lib/swift/linux/*"
20+
for i in $filestoremove
21+
do
22+
echo $basepath/$i
23+
rm $basepath/$i
24+
done
25+
# Mac only
26+
rm -r $basepath/lib/swift/macosx $basepath/lib/sourcekitd.framework

Diff for: ‎utils/webassembly/remove-wasi-extra-files.sh

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
#!/bin/bash
2+
basepath="compiler/wasi-sdk"
3+
filestoremove="bin/clang*
4+
bin/llvm*
5+
bin/llc"
6+
for i in $filestoremove
7+
do
8+
echo $basepath/$i
9+
rm $basepath/$i
10+
done

Diff for: ‎utils/webassembly/sdkroot/README.md

+87
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
SwiftWasm: Getting started
2+
==========================
3+
4+
Thank you for trying SwiftWasm! Here's how to get started.
5+
6+
Please visit our website at https://swiftwasm.org for the latest updates.
7+
8+
9+
Install dependencies
10+
====================
11+
12+
Before running SwiftWasm, you will need to install some dependencies.
13+
14+
Ubuntu:
15+
16+
```
17+
sudo apt-get install libatomic1
18+
```
19+
20+
macOS:
21+
22+
(No dependencies needed.)
23+
24+
Windows:
25+
26+
Install Windows Subsystem for Linux, then follow the Ubuntu instructions.
27+
28+
29+
30+
31+
Compile SwiftWasm
32+
=================
33+
34+
Run
35+
36+
```
37+
./swiftwasm example/hello.swift hello.wasm
38+
```
39+
40+
To compile example/hello.swift to hello.wasm.
41+
42+
43+
44+
45+
Running Wasm files
46+
==================
47+
48+
To run the resulting hello.wasm file:
49+
50+
- Visit https://swiftwasm.org/polyfill/
51+
- select "Browse", and choose the hello.wasm file
52+
- you should get output in the textbox.
53+
54+
This polyfill should work in Firefox 66, Chrome 74, and Safari 12.1.
55+
56+
You can also run the file outside a browser with:
57+
58+
- Wasmtime https://github.com/CraneStation/wasmtime
59+
- Lucet https://github.com/swiftwasm/lucet/tree/swiftwasm
60+
- or any other WASI-compatible WebAssembly runtime.
61+
62+
63+
64+
65+
Questions and support
66+
=====================
67+
68+
If you have any questions, please open an issue on
69+
70+
https://github.com/swiftwasm/swift
71+
72+
73+
74+
Third-party licenses
75+
====================
76+
77+
This package contains components with their own license requirements.
78+
79+
Swift compiler: https://github.com/apple/swift/blob/master/LICENSE.txt
80+
81+
LLVM/Clang: https://github.com/llvm/llvm-project/blob/master/lld/LICENSE.TXT
82+
83+
WASI sysroot: https://github.com/CraneStation/wasi-sysroot/blob/master/LICENSE
84+
85+
ICU: https://github.com/unicode-org/icu/blob/master/icu4c/LICENSE
86+
87+
WASI polyfill: https://github.com/CraneStation/wasmtime/blob/master/wasmtime-wasi/LICENSE

Diff for: ‎utils/webassembly/sdkroot/example/hello.swift

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
print("Hello, 🌐!")

Diff for: ‎utils/webassembly/sdkroot/extra_objs/fakelocaltime.o

423 Bytes
Binary file not shown.

Diff for: ‎utils/webassembly/sdkroot/extra_objs/fakepthread.o

3.97 KB
Binary file not shown.

Diff for: ‎utils/webassembly/sdkroot/extra_utils/generateModulemap.sh

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
#!/bin/sh
2+
exec sed -e "s@\"/include@\"$1/include@g" "$(dirname $0)/glibc.modulemap"

0 commit comments

Comments
 (0)