Skip to content

Commit 8cc6ded

Browse files
andrurogerzandrurogerz
authored
[android] run a subset of lldb tests on arm64 (#157)
## Purpose Enable a small subset of lldb tests to run on Android `arm64-v8a` emulator for better test coverage. Resolves #161 ## Overview * Rework the test matrix in the `test-android` job to include a single `arm64-v8a` instance that runs tests from a subset of lldb test directories (android, api, linux, and sanity). * Add `emulator-build` field to test matrix so that the `arm64-v8a` variant can select version 35.1.21.0 (build_id 12325540), which supports arm images. Leave this field unspecified in the `x86_64` jobs because they don't need it. * Add `emulator-options` field to test matrix inputs and specify `-qemu -machine virt -cpu max` options. Leave this field unspecified in the `x86_64` jobs because they don't need it. * Rename `api-level` matrix field to `android-api-level` for clarity. * Pass `-gpu swiftshader_indirect` argument to all emulator instances (arm and x86). This shouldn't really matter since we're running headless tests anyway. * Split the `processor` matrix field and into two fields: `arch` and `llvm-arch`. For `x86_64`, these fields are set to the same value. For `arm64-v8a`, the `llvm-arch` field is set to `aarch64` which is the value expected by the lldb test suite. * Add an upstream test exclusion list for the test cases that fail running lldb-server on arm64 Android. * Add a ds2 test exclusion list for the test cases that fail running ds2 on arm64 Android (that don't fail with lldb-server). ## Background By using a slightly outdated version of the Android emulator (see https://issuetracker.google.com/u/1/issues/373790939) and passing some extra flags to qemu (-machine virt, -cpu max), it is possible to run an Android arm64-v8a guest on an x86_64 host on an GitHub runner. Because a qemu arm guest running on an x86_64 host uses software emulation instead of virtualization, we'll limit the number of tests that run to keep the job run time down. Passing the `-machine virt` argument to qemu selects the generic virtual hardware platform described [here](https://www.qemu.org/docs/master/system/arm/virt.html). Passing the `-cpu max` argument to qemu selects the best emulated cpu option available. ## Validation Verified the job runs and passes on this PR. Job run time is ~5 min, which is briefer than the longest x86_64 test job which takes ~8 min.
1 parent a825114 commit 8cc6ded

File tree

3 files changed

+60
-23
lines changed

3 files changed

+60
-23
lines changed

‎.github/workflows/build.yml

+45-23
Original file line numberDiff line numberDiff line change
@@ -408,12 +408,11 @@ jobs:
408408
strategy:
409409
fail-fast: false
410410
matrix:
411-
# We don't currently have a good way to run arm64 Android emulator on
412-
# the available GitHub test runners.
413411
include:
414412
- test-category: misc
415-
processor: x86_64
416-
api-level: 28
413+
arch: x86_64
414+
llvm-arch: x86_64
415+
android-api-level: 28
417416
test-subdirs: >-
418417
SourceCache/llvm/lldb/test/API/android
419418
SourceCache/llvm/lldb/test/API/api
@@ -424,24 +423,45 @@ jobs:
424423
SourceCache/llvm/lldb/test/API/types
425424
426425
- test-category: commands
427-
processor: x86_64
428-
api-level: 28
426+
arch: x86_64
427+
llvm-arch: x86_64
428+
android-api-level: 28
429429
test-subdirs: >-
430430
SourceCache/llvm/lldb/test/API/commands
431431
432432
- test-category: Python API
433-
processor: x86_64
434-
api-level: 28
433+
arch: x86_64
434+
llvm-arch: x86_64
435+
android-api-level: 28
435436
test-subdirs: >-
436437
SourceCache/llvm/lldb/test/API/python_api
437438
438439
- test-category: functionalities
439-
processor: x86_64
440-
api-level: 28
440+
arch: x86_64
441+
llvm-arch: x86_64
442+
android-api-level: 28
441443
test-subdirs: >-
442444
SourceCache/llvm/lldb/test/API/functionalities
443445
444-
name: Android ${{ matrix.processor }} test ${{ matrix.test-category}}
446+
# Run a small sub-set of tests on arm64. The emulator is very slow
447+
# so it is not practical to run more tests until we can run on an
448+
# arm64 runner with virtualization.
449+
- test-category: sanity
450+
arch: arm64-v8a
451+
llvm-arch: aarch64
452+
# Android emulator supports ARM on x86 only on API < 28
453+
android-api-level: 27
454+
# This is the most recent build of the emulator that supports
455+
# booting arm64 images.
456+
emulator-build: 12234392
457+
emulator-options: -qemu -machine virt -cpu max
458+
test-subdirs: >-
459+
SourceCache/llvm/lldb/test/API/android
460+
SourceCache/llvm/lldb/test/API/api
461+
SourceCache/llvm/lldb/test/API/linux
462+
SourceCache/llvm/lldb/test/API/sanity
463+
464+
name: Test Android ${{ matrix.arch}} ${{ matrix.test-category }}
445465

446466
steps:
447467
- uses: actions/download-artifact@v4
@@ -454,7 +474,7 @@ jobs:
454474
455475
- uses: actions/download-artifact@v4
456476
with:
457-
name: android-${{ matrix.processor }}-ds2
477+
name: android-${{ matrix.arch }}-ds2
458478
path: ${{ github.workspace }}/BinaryCache
459479
- name: untar ds2 artifacts
460480
run: |
@@ -479,7 +499,7 @@ jobs:
479499
id: avd-cache
480500
with:
481501
path: ~/.android/**
482-
key: avd-${{ runner.os }}-${{ matrix.processor }}-${{ matrix.api-level }}
502+
key: avd-${{ runner.os }}-${{ matrix.arch }}-${{ matrix.android-api-level }}
483503

484504
- name: Enable KVM
485505
run: |
@@ -491,26 +511,28 @@ jobs:
491511
if: steps.avd-cache.outputs.cache-hit != 'true'
492512
uses: reactivecircus/android-emulator-runner@v2
493513
with:
494-
api-level: ${{ matrix.api-level }}
495-
arch: ${{ matrix.processor }}
514+
emulator-build: ${{ matrix.emulator-build }}
515+
api-level: ${{ matrix.android-api-level }}
516+
arch: ${{ matrix.arch }}
496517
force-avd-creation: false
497-
emulator-options: -no-window -noaudio -no-boot-anim -camera-back none
518+
emulator-options: -no-window -noaudio -no-boot-anim -camera-back none -gpu swiftshader_indirect ${{ matrix.emulator-options }}
498519
disable-animations: false
499520
script: echo "Generated AVD snapshot for caching."
500521

501522
- name: Run lldb tests against ds2 on Android emulator
502523
uses: reactivecircus/android-emulator-runner@v2
503524
with:
504-
api-level: ${{ matrix.api-level }}
505-
arch: ${{ matrix.processor }}
525+
emulator-build: ${{ matrix.emulator-build }}
526+
api-level: ${{ matrix.android-api-level }}
527+
arch: ${{ matrix.arch }}
506528
force-avd-creation: false
507529
# NOTE: -no-snapshot will not load or save state and the emulator will
508530
# cold boot. Ideally, we would specify -no-snapshot-save instead so it
509531
# would load the snapshot created in the caching step (but not save a
510532
# new one). However, doing that causes the emulator to hang during
511533
# shutdown. We're still saving time by reusing the device created in
512534
# the caching step and not wiping userdata on boot.
513-
emulator-options: -no-snapshot -no-window -noaudio -no-boot-anim -camera-back none
535+
emulator-options: -no-snapshot -no-window -noaudio -no-boot-anim -camera-back none -gpu swiftshader_indirect ${{ matrix.emulator-options }}
514536
disable-animations: true
515537
script: >
516538
$ANDROID_SDK_ROOT/platform-tools/adb forward tcp:5432 tcp:5432
@@ -523,12 +545,12 @@ jobs:
523545
524546
${{ github.workspace }}/BinaryCache/llvm/bin/lldb-dotest
525547
--out-of-tree-debugserver
526-
--arch ${{ matrix.processor}}
548+
--arch ${{ matrix.llvm-arch }}
527549
--platform-name remote-android
528550
--platform-url connect://localhost:5432
529551
--platform-working-dir /data/local/tmp
530-
--compiler=$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin/${{ matrix.processor }}-linux-android${{ matrix.api-level }}-clang
531-
--excluded ${{ github.workspace }}/SourceCache/ds2/Support/Testing/Excluded/ds2/android-${{ matrix.processor }}.excluded
532-
--excluded ${{ github.workspace }}/SourceCache/ds2/Support/Testing/Excluded/upstream/android-${{ matrix.processor }}.excluded
552+
--compiler=$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin/${{ matrix.llvm-arch }}-linux-android${{ matrix.android-api-level }}-clang
553+
--excluded ${{ github.workspace }}/SourceCache/ds2/Support/Testing/Excluded/ds2/android-${{ matrix.llvm-arch }}.excluded
554+
--excluded ${{ github.workspace }}/SourceCache/ds2/Support/Testing/Excluded/upstream/android-${{ matrix.llvm-arch }}.excluded
533555
--excluded ${{ github.workspace }}/SourceCache/ds2/Support/Testing/Excluded/upstream/non-debugserver-tests.excluded
534556
${{ matrix.test-subdirs }}

‎Support/Testing/Excluded/ds2/android-aarch64.excluded

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
skip
2+
TestAArch64LinuxTaggedMemoryAccess.AArch64LinuxTaggedMemoryReadTestCase.test_tagged_memory_find
3+
TestAArch64LinuxTaggedMemoryAccess.AArch64LinuxTaggedMemoryReadTestCase.test_tagged_memory_read
4+
TestAArch64LinuxTaggedMemoryRegion.AArch64LinuxTaggedMemoryRegionTestCase.test_mte_regions
5+
TestUnwindSignal.UnwindSignalTestCase.test_unwind_signal

‎Support/Testing/Excluded/upstream/android-aarch64.excluded

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
skip
2+
TestGdbRemoteMemoryTagging.TestGdbRemoteMemoryTagging.test_tag_write_QMemTags_packets_llgs
3+
TestGdbRemoteMemoryTagging.TestGdbRemoteMemoryTagging.test_tag_read_qMemTags_packets_llgs
4+
TestGdbRemotePlatformFile.TestGdbRemotePlatformFile.test_platform_file_wronly_creat_llgs
5+
TestGdbRemotePlatformFile.TestGdbRemotePlatformFile.test_platform_file_wronly_creat_excl_llgs
6+
TestGdbRemotePlatformFile.TestGdbRemotePlatformFile.test_platform_file_mode_llgs
7+
TestGdbRemotePlatformFile.TestGdbRemotePlatformFile.test_platform_file_fstat_llgs
8+
TestLldbGdbServer.LldbGdbServerTestCase.test_qMemoryRegionInfo_reports_heap_address_as_rw_llgs
9+
TestAArch64LinuxTLSRegisters.AArch64LinuxTLSRegisters.test_tpidr2_no_sme
10+
TestAArch64LinuxTLSRegisters.AArch64LinuxTLSRegisters.test_tls_no_sme

0 commit comments

Comments
 (0)