Skip to content

Run webdev tests in a copy of the sdk directory #1987

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Feb 23, 2023
Merged

Run webdev tests in a copy of the sdk directory #1987

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
f45fd0b
Validate only needed summaries in expression_compiler_service
Jan 27, 2023
2fab3a0
Move shared test functionality into test_common package
Feb 8, 2023
e781731
Rebase on master
Feb 14, 2023
15b0837
Fix bad merge
Feb 14, 2023
6647335
Re-enable webdev weak null safety e2e tests
Feb 14, 2023
4ddc0e4
Create copy of the SDK for tests
Feb 14, 2023
f890bfa
Modify tests to run with sdk copy
Feb 15, 2023
7dc33e8
Merge with master
Feb 15, 2023
d431a3e
Cleanup
Feb 15, 2023
11aa23f
Pull out project definitions into separate file
Feb 16, 2023
230ae10
Merge branch 'master' of github.com:dart-lang/webdev into annagrin/sd…
Feb 16, 2023
caeb6f9
Add package name to the project and use it in tests
Feb 16, 2023
0acd810
Merged with master
Feb 16, 2023
8e1a7bf
Fix sdk configiuration tests
Feb 17, 2023
68bd0d3
Merged with master
Feb 18, 2023
a3a68cf
Update instance_inspection tests
Feb 18, 2023
5c6cfce
Merge with master
Feb 22, 2023
3214af1
Less changes in chrome_proxy_service.dart
Feb 22, 2023
71657bb
Merge with master
Feb 23, 2023
d9a949a
Run webdev tests in a copy of the sdk
Feb 23, 2023
98773b9
Merge with master
Feb 23, 2023
8170139
Fix bad merge
Feb 23, 2023
5eebda3
Cleanup
Feb 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion dwds/test/fixtures/context.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ class TestContext {
}) async {
// Generate missing SDK assets if needed.
final sdkConfigurationProvider =
TestSdkConfigurationProvider(verboseCompiler: verboseCompiler);
TestSdkConfigurationProvider(verbose: verboseCompiler);
final sdkLayout = sdkConfigurationProvider.sdkLayout;
final configuration = await sdkConfigurationProvider.configuration;
configuration.validate();
Expand Down
6 changes: 3 additions & 3 deletions test_common/lib/sdk_asset_generator.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,14 @@ class SdkAssetGenerator {
final _logger = Logger('SdkAssetGenerator');

final FileSystem fileSystem;
final bool verboseCompiler;
final bool verbose;

late final TestSdkLayout sdkLayout;

SdkAssetGenerator({
this.fileSystem = const LocalFileSystem(),
required this.sdkLayout,
this.verboseCompiler = false,
this.verbose = false,
});

/// Generate all SDK assets, once for the current executable run.
Expand Down Expand Up @@ -180,7 +180,7 @@ class SdkAssetGenerator {
'--no-sound-null-safety',
'--output',
summaryPath,
if (verboseCompiler) '--verbose',
if (verbose) '--verbose',
];

_logger.fine('Executing dart ${args.join(' ')}');
Expand Down
9 changes: 4 additions & 5 deletions test_common/lib/test_sdk_configuration.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ class TestSdkCopyConfigurationProvider extends SdkConfigurationProvider {
try {
final assetGenerator = SdkAssetGenerator(
sdkLayout: sdkLayout,
verboseCompiler: _verbose,
verbose: _verbose,
);

await assetGenerator.generateSdkAssets();
Expand Down Expand Up @@ -92,13 +92,12 @@ class TestSdkCopyConfigurationProvider extends SdkConfigurationProvider {
/// TODO(annagrin): update to only generating missing sound artifacts
/// for frontend server after we have no uses of weak null safety.
class TestSdkConfigurationProvider extends SdkConfigurationProvider {
final bool _verboseCompiler;
final bool _verbose;
SdkConfiguration? _configuration;

final sdkLayout = TestSdkLayout.defaultSdkLayout;

TestSdkConfigurationProvider({bool verboseCompiler = false})
: _verboseCompiler = verboseCompiler;
TestSdkConfigurationProvider({bool verbose = false}) : _verbose = verbose;

@override
Future<SdkConfiguration> get configuration async =>
Expand All @@ -108,7 +107,7 @@ class TestSdkConfigurationProvider extends SdkConfigurationProvider {
Future<SdkConfiguration> _create() async {
final assetGenerator = SdkAssetGenerator(
sdkLayout: sdkLayout,
verboseCompiler: _verboseCompiler,
verbose: _verbose,
);

await assetGenerator.generateSdkAssets();
Expand Down
2 changes: 1 addition & 1 deletion test_common/test/sdk_asset_generator_test.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ void main() {
final configuration = TestSdkLayout.createConfiguration(sdkLayout);

final assetGenerator =
SdkAssetGenerator(sdkLayout: sdkLayout, verboseCompiler: true);
SdkAssetGenerator(sdkLayout: sdkLayout, verbose: true);
await assetGenerator.generateSdkAssets();

// Make sure SDK configuration and asset generator agree on the file paths.
Expand Down
38 changes: 21 additions & 17 deletions webdev/test/daemon/app_domain_test.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,39 +14,43 @@ import 'utils.dart';
void main() {
late String exampleDirectory;

final testRunner = TestRunner();
setUpAll(() async {
exampleDirectory = await prepareWorkspace();
await testRunner.setUpAll();
exampleDirectory = await testRunner.prepareWorkspace();
});

tearDownAll(testRunner.tearDownAll);

group('AppDomain', () {
group('Events', () {
test('.start', () async {
var webdev =
await runWebDev(['daemon'], workingDirectory: exampleDirectory);
var webdev = await testRunner
.runWebDev(['daemon'], workingDirectory: exampleDirectory);
await expectLater(
webdev.stdout, emitsThrough(startsWith('[{"event":"app.start"')));
await exitWebdev(webdev);
});

test('.started', () async {
var webdev =
await runWebDev(['daemon'], workingDirectory: exampleDirectory);
var webdev = await testRunner
.runWebDev(['daemon'], workingDirectory: exampleDirectory);
await expectLater(
webdev.stdout, emitsThrough(startsWith('[{"event":"app.started"')));
await exitWebdev(webdev);
});

test('.debugPort', () async {
var webdev =
await runWebDev(['daemon'], workingDirectory: exampleDirectory);
var webdev = await testRunner
.runWebDev(['daemon'], workingDirectory: exampleDirectory);
await expectLater(webdev.stdout,
emitsThrough(startsWith('[{"event":"app.debugPort"')));
await exitWebdev(webdev);
});

test('.log', () async {
var webdev =
await runWebDev(['daemon'], workingDirectory: exampleDirectory);
var webdev = await testRunner
.runWebDev(['daemon'], workingDirectory: exampleDirectory);
var appId = await waitForAppId(webdev);
// The example app does an initial print.
await expectLater(
Expand All @@ -60,8 +64,8 @@ void main() {

group('Methods', () {
test('.callServiceExtension', () async {
var webdev =
await runWebDev(['daemon'], workingDirectory: exampleDirectory);
var webdev = await testRunner
.runWebDev(['daemon'], workingDirectory: exampleDirectory);
var appId = await waitForAppId(webdev);
if (Platform.isWindows) {
// Windows takes a bit longer to run the application and register
Expand All @@ -81,8 +85,8 @@ void main() {
});

test('.reload', () async {
var webdev =
await runWebDev(['daemon'], workingDirectory: exampleDirectory);
var webdev = await testRunner
.runWebDev(['daemon'], workingDirectory: exampleDirectory);
var appId = await waitForAppId(webdev);
var extensionCall = '[{"method":"app.restart","id":0,'
'"params" : { "appId" : "$appId", "fullRestart" : false}}]';
Expand All @@ -97,8 +101,8 @@ void main() {
});

test('.restart', () async {
var webdev =
await runWebDev(['daemon'], workingDirectory: exampleDirectory);
var webdev = await testRunner
.runWebDev(['daemon'], workingDirectory: exampleDirectory);
var appId = await waitForAppId(webdev);
var extensionCall = '[{"method":"app.restart","id":0,'
'"params" : { "appId" : "$appId", "fullRestart" : true}}]';
Expand All @@ -117,8 +121,8 @@ void main() {
});

test('.stop', () async {
var webdev =
await runWebDev(['daemon'], workingDirectory: exampleDirectory);
var webdev = await testRunner
.runWebDev(['daemon'], workingDirectory: exampleDirectory);
var appId = await waitForAppId(webdev);
var stopCall = '[{"method":"app.stop","id":0,'
'"params" : { "appId" : "$appId"}}]';
Expand Down
18 changes: 11 additions & 7 deletions webdev/test/daemon/daemon_domain_test.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,19 @@ import 'utils.dart';
void main() {
late String exampleDirectory;

final testRunner = TestRunner();
setUpAll(() async {
exampleDirectory = await prepareWorkspace();
await testRunner.setUpAll();
exampleDirectory = await testRunner.prepareWorkspace();
});

tearDownAll(testRunner.tearDownAll);

group('Daemon', () {
group('Events', () {
test('.connected', () async {
var webdev =
await runWebDev(['daemon'], workingDirectory: exampleDirectory);
var webdev = await testRunner
.runWebDev(['daemon'], workingDirectory: exampleDirectory);
await expectLater(
webdev.stdout, emits(startsWith('[{"event":"daemon.connected"')));
await exitWebdev(webdev);
Expand All @@ -31,17 +35,17 @@ void main() {

group('Methods', () {
test('.version', () async {
var webdev =
await runWebDev(['daemon'], workingDirectory: exampleDirectory);
var webdev = await testRunner
.runWebDev(['daemon'], workingDirectory: exampleDirectory);
webdev.stdin.add(utf8.encode('[{"method":"daemon.version","id":0}]\n'));
await expectLater(
webdev.stdout, emitsThrough(equals('[{"id":0,"result":"0.4.2"}]')));
await exitWebdev(webdev);
});

test('.shutdown', () async {
var webdev =
await runWebDev(['daemon'], workingDirectory: exampleDirectory);
var webdev = await testRunner
.runWebDev(['daemon'], workingDirectory: exampleDirectory);
webdev.stdin
.add(utf8.encode('[{"method":"daemon.shutdown","id":0}]\n'));
await expectLater(webdev.stdout, emitsThrough(equals('[{"id":0}]')));
Expand Down
9 changes: 7 additions & 2 deletions webdev/test/daemon/launch_app_test.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,17 @@ import 'utils.dart';
void main() {
late String exampleDirectory;

final testRunner = TestRunner();
setUpAll(() async {
exampleDirectory = await prepareWorkspace();
await testRunner.setUpAll();
exampleDirectory = await testRunner.prepareWorkspace();
});

tearDownAll(testRunner.tearDownAll);

test('--launch-app launches the specified app', () async {
var webdev = await runWebDev(['daemon', '--launch-app=web/scopes.html'],
var webdev = await testRunner.runWebDev(
['daemon', '--launch-app=web/scopes.html'],
workingDirectory: exampleDirectory);
var appId = await waitForAppId(webdev);

Expand Down
15 changes: 0 additions & 15 deletions webdev/test/daemon/utils.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,9 @@
import 'dart:async';
import 'dart:convert';

import 'package:path/path.dart' as p;
import 'package:test/test.dart';
import 'package:test_process/test_process.dart';
import 'package:vm_service/vm_service.dart';
import 'package:webdev/src/util.dart';

import '../test_utils.dart';

const isRPCError = TypeMatcher<RPCError>();

Expand All @@ -37,17 +33,6 @@ Future<String> waitForAppId(TestProcess webdev) async {
return appId;
}

Future<String> prepareWorkspace() async {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember adding this recently to fix some of the flakes we were hitting for our CI tests on Windows. Is this now done elsewhre?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see where it has been moved to, nevermind!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it just moved to the test_utils inside TestRunner class, as I needed to run it using correct dart path from the sdk layout.

var exampleDirectory =
p.absolute(p.join(p.current, '..', 'fixtures', '_webdevSoundSmoke'));

var process = await TestProcess.start(dartPath, ['pub', 'upgrade'],
workingDirectory: exampleDirectory, environment: getPubEnvironment());

await process.shouldExit(0);
return exampleDirectory;
}

String? getDebugServiceUri(String line) {
var regex = RegExp(r'Debug service listening on (?<wsUri>[^\s^\\]*)');
var match = regex.firstMatch(line);
Expand Down
Loading