Skip to content

[rfw] Run tests for package:rfw on stable as well. #888

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 23, 2022
Merged

[rfw] Run tests for package:rfw on stable as well. #888

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d395c23
Run tests for package:rfw on stable as well.
Hixie Feb 17, 2022
2974407
Merge branch 'main' into rfw-tests
stuartmorgan-g Mar 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 5 additions & 32 deletions .cirrus.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,16 +103,8 @@ task:
CHANNEL: "master"
CHANNEL: "stable"
script:
# Exclude:
# - flutter_image; its tests need a test server, so are run via
# local_tests.sh.
# - rfw on stable; it includes golden tests whose results differ
# between branch
- if [[ "$CHANNEL" == "master" ]]; then
- ./script/tool_runner.sh test --exclude=flutter_image
- else
- ./script/tool_runner.sh test --exclude=flutter_image,rfw
- fi
# We exclude flutter_image because its tests need a test server, so are run via local_tests.sh.
- ./script/tool_runner.sh test --exclude=flutter_image
depends_on:
- format+analyze
- name: android-build+platform-tests
Expand All @@ -123,18 +115,8 @@ task:
script:
# extension_google_sign_in_as_googleapis_auth is currently not building, see
# https://github.com/flutter/flutter/issues/89301
# rfw is excluded until the next Flutter stable release because it depends
# on features that have never shipped to stable. (The rfw package has
# never worked on stable so this is not going to break anyone.)
# When updating this, also look at the ios tests below.
# When updating this, also update the `rfw/run_tests.sh` file.
- if [[ "$CHANNEL" == "master" ]]; then
- ./script/tool_runner.sh build-examples --apk --exclude=extension_google_sign_in_as_googleapis_auth
- ./script/tool_runner.sh native-test --android --no-integration
- else
- ./script/tool_runner.sh build-examples --apk --exclude=extension_google_sign_in_as_googleapis_auth,rfw
- ./script/tool_runner.sh native-test --android --no-integration --exclude=rfw
- fi
- ./script/tool_runner.sh build-examples --apk --exclude=extension_google_sign_in_as_googleapis_auth
- ./script/tool_runner.sh native-test --android --no-integration
depends_on:
- format+analyze
- name: web_benchmarks_test
Expand Down Expand Up @@ -164,16 +146,7 @@ task:
CHANNEL: "master"
CHANNEL: "stable"
build_script:
# Exclude rfw until the next Flutter stable release because it depends
# on features that have never shipped to stable. (The rfw package has
# never worked on stable so this is not going to break anyone.)
# When updating this, also look at the android tests above.
# When updating this, also update the `rfw/run_tests.sh` file.
- if [[ "$CHANNEL" == "master" ]]; then
- ./script/tool_runner.sh build-examples --ios
- else
- ./script/tool_runner.sh build-examples --ios --exclude=rfw
- fi
- ./script/tool_runner.sh build-examples --ios
- name: local_tests
env:
PATH: $PATH:/usr/local/bin
Expand Down
4 changes: 4 additions & 0 deletions packages/rfw/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -266,3 +266,7 @@ When contributing code, ensure that `flutter test --coverage; lcov
update `test_coverage/bin/test_coverage.dart` with the appropriate
expectations to prevent future coverage regressions. (That program is
run by `run_tests.sh`.)

Golden tests are only run against the Flutter master channel and only
run on Linux, since minor rendering differences are expected on
different platforms and on different versions of Flutter.
9 changes: 8 additions & 1 deletion packages/rfw/test/argument_decoders_test.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,18 @@

// This file is hand-formatted.

import 'dart:io' show Platform;

import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:rfw/formats.dart' show parseLibraryFile;
import 'package:rfw/rfw.dart';

// See Contributing section of README.md file.
final bool runGoldens = Platform.isLinux &&
(!Platform.environment.containsKey('CHANNEL') ||
Platform.environment['CHANNEL'] == 'master');

void main() {
testWidgets('String example', (WidgetTester tester) async {
Duration? duration;
Expand Down Expand Up @@ -494,5 +501,5 @@ void main() {
find.byType(RemoteWidget),
matchesGoldenFile('goldens/argument_decoders_test.gridview.custom.png'),
);
});
}, skip: !runGoldens);
}
9 changes: 8 additions & 1 deletion packages/rfw/test/material_widgets_test.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,18 @@
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'dart:io' show Platform;

import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:rfw/formats.dart' show parseLibraryFile;
import 'package:rfw/rfw.dart';

// See Contributing section of README.md file.
final bool runGoldens = Platform.isLinux &&
(!Platform.environment.containsKey('CHANNEL') ||
Platform.environment['CHANNEL'] == 'master');

void main() {
testWidgets('Material widgets', (WidgetTester tester) async {
final Runtime runtime = Runtime()
Expand Down Expand Up @@ -140,5 +147,5 @@ void main() {
find.byType(RemoteWidget),
matchesGoldenFile('goldens/material_test.drawer.png'),
);
});
}, skip: !runGoldens);
}
27 changes: 10 additions & 17 deletions packages/rfw/test_coverage/bin/test_coverage.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,39 +15,32 @@ const String lastUpdate = '2021-08-30';
Future<void> main(List<String> arguments) async {
// This script is mentioned in the README.md file.

if (Platform.environment['CHANNEL'] == 'stable') {
// For now these are disabled because this package has never been supported
// on the stable channel and requires newer language features that have not
// yet shipped to a stable build. It will be possible to test this with the
// first stable to ship after October 2021.
print(
'Skipping tests on stable channel.\n'
'These tests can be unskipped once we ship a stable after October 2021.',
);
exit(0);
}

final Directory coverageDirectory = Directory('coverage');

if (coverageDirectory.existsSync()) {
coverageDirectory.deleteSync(recursive: true);
}

// We run with --update-goldens because the goal here is not to verify the tests
// pass but to verify the coverage, and the goldens are not always going to pass
// when run on different platforms (e.g. on Cirrus we run this on a mac but the
// goldens expect a linux box).
final ProcessResult result = Process.runSync(
'flutter',
<String>['test', '--coverage', '--update-goldens'],
<String>['test', '--coverage'],
);
if (result.exitCode != 0) {
print(result.stdout);
print(result.stderr);
print('Tests failed.');
// leave coverage directory around to aid debugging
exit(1);
}

if (Platform.environment['CHANNEL'] != 'master') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you want the "no channel set" condition here as well for local runs?

print(
'Tests passed. (Coverage verification skipped; not on master channel.)',
);
coverageDirectory.deleteSync(recursive: true);
exit(0);
}

final List<lcov.Record> records = await lcov.Parser.parse(
'coverage/lcov.info',
);
Expand Down