Skip to content
This repository was archived by the owner on Feb 11, 2024. It is now read-only.

move to package:args for cli arg parsing #15

Merged
merged 6 commits into from
Jul 16, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,12 @@
## 0.0.3

* Using `package:args` for handling CLI arguments.
* Dependency versions increased.
* Corrected wrong cli command suggestions in case of un-locateable dylibs.
* Fixed throughput benchmark's `RangeError` in case of 0 result.

## 0.0.2

* Added support for MacOS.

## 0.0.1+1
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ dart run benchmark/throughput.dart # For parallel requests benchmark.
dart run benchmark/run_all.dart # To run all the benchmarks and get reports.
```

All the benchmarking scripts take test server url as a cli argument. `throughput.dart` and `run_all.dart` also take `N` where `2^N` is the maximum possible parallel requests and the max duration for each run to complete in seconds.
Use `-h` to see available cli arguments and usage informations.

To know how to setup local test servers, read [benchmarking guide](benchmark/benchmarking.md).

Expand Down
22 changes: 18 additions & 4 deletions benchmark/latency.dart
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

import 'dart:io' as io;

import 'package:args/args.dart';
import 'package:cronet/cronet.dart';

abstract class LatencyBenchmark {
Expand Down Expand Up @@ -119,11 +120,24 @@ class CronetLatencyBenchmark extends LatencyBenchmark {
}

void main(List<String> args) async {
// Accepts test url as optional cli parameter.
var url = 'https://example.com';
if (args.isNotEmpty) {
url = args[0];
final parser = ArgParser();
parser
..addOption('url',
abbr: 'u',
help: 'The server to ping for running this benchmark.',
defaultsTo: 'https://example.com')
..addFlag('help',
abbr: 'h', negatable: false, help: 'Print this usage information.');
final arguments = parser.parse(args);
if (arguments.wasParsed('help')) {
print(parser.usage);
return;
}
if (arguments.rest.isNotEmpty) {
print(parser.usage);
throw ArgumentError();
}
final url = arguments['url'] as String;
// TODO: https://github.com/google/cronet.dart/issues/11
await CronetLatencyBenchmark.main(url);
// Used as an delemeter while parsing output in run_all script.
Expand Down
59 changes: 43 additions & 16 deletions benchmark/run_all.dart
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ import 'dart:convert';
import 'dart:io';
import 'dart:math';

import 'package:args/args.dart';

import 'latency.dart';
import 'throughput.dart';

Expand All @@ -24,27 +26,46 @@ List<int> throughputParserHelper(String aotThroughputStdout) {
}

void main(List<String> args) async {
var url = 'https://example.com';
var throughputPrallelLimit = 10;
var duration = const Duration(seconds: 1);
if (args.isNotEmpty) {
url = args[0];
}
if (args.length > 1) {
throughputPrallelLimit = int.parse(args[1]).toInt();
final parser = ArgParser();
parser
..addOption('url',
abbr: 'u',
help: 'The server to ping for running this benchmark.',
defaultsTo: 'https://example.com')
..addOption('limit',
abbr: 'l',
help: 'Limits the maximum number of parallel requests to 2^N where N'
' is provided through this option.',
defaultsTo: '10')
..addOption('time',
abbr: 't',
help: 'Maximum second(s) the benchmark should wait for each request.',
defaultsTo: '1')
..addFlag('help',
abbr: 'h', negatable: false, help: 'Print this usage information.');
final arguments = parser.parse(args);
if (arguments.wasParsed('help')) {
print(parser.usage);
return;
}
if (args.length > 2) {
duration = Duration(seconds: int.parse(args[2]));
if (arguments.rest.isNotEmpty) {
print(parser.usage);
throw ArgumentError();
}
final url = arguments['url'] as String;
final throughputPrallelLimit = int.parse(arguments['limit'] as String);
final duration = Duration(seconds: int.parse(arguments['time'] as String));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work with half a second?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. Duration's seconds param takes int.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should do milliseconds instead then, or convert a double manually. Because otherwise we can't test shorter durations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's go with milliseconds then 😀

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got this result under short duration. Cronet is so much faster.

Latency Test Results

Mode package:cronet dart:io
JIT 247.333 ms 403.000 ms
AOT 240.444 ms 373.000 ms

Throughput Test Results (Duration: 500 ms)

Mode package:cronet dart:io
JIT 30 out of 1024 0 out of 0
AOT 26 out of 1024 0 out of 0


print('Latency Test against: $url');
print('JIT');
final jitCronetLatency = await CronetLatencyBenchmark.main(url);
final jitDartIOLatency = await DartIOLatencyBenchmark.main(url);

print('AOT');
print('Compiling...');
Process.runSync('dart', ['compile', 'exe', 'benchmarks/latency.dart']);
final aotLantencyProc = await Process.start('benchmarks/latency.exe', [url]);
Process.runSync('dart', ['compile', 'exe', 'benchmark/latency.dart']);
final aotLantencyProc =
await Process.start('benchmark/latency.exe', ['-u', url]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do the 'exe' files compiled by dart only work on windows?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No 😅. It is named as .exe by default regardless of the platform.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yes. I see it now in the documentation. Dart supports quite a few different compilation formats 😀

stderr.addStream(aotLantencyProc.stderr);
var latencyStdout = '';
await for (final chunk in aotLantencyProc.stdout.transform(utf8.decoder)) {
Expand All @@ -67,9 +88,15 @@ void main(List<String> args) async {

print('AOT');
print('Compiling...');
Process.runSync('dart', ['compile', 'exe', 'benchmarks/throughput.dart']);
final aotThroughputProc = await Process.start('benchmarks/throughput.exe',
[url, throughputPrallelLimit.toString(), duration.inSeconds.toString()]);
Process.runSync('dart', ['compile', 'exe', 'benchmark/throughput.dart']);
final aotThroughputProc = await Process.start('benchmark/throughput.exe', [
'-u',
url,
'-l',
throughputPrallelLimit.toString(),
'-t',
duration.inSeconds.toString()
]);
stderr.addStream(aotThroughputProc.stderr);
var throughputStdout = '';
await for (final chunk in aotThroughputProc.stdout.transform(utf8.decoder)) {
Expand All @@ -90,7 +117,7 @@ void main(List<String> args) async {
' ${jitDartIOLatency.toStringAsFixed(3)} ms |');
print('| AOT | ${aotCronetLatency.toStringAsFixed(3)} ms |'
' ${aotDartIOLatency.toStringAsFixed(3)} ms |');
print('\n\nThroughput Test Results (Duration ${duration.inSeconds}');
print('\nThroughput Test Results (Duration: ${duration.inSeconds}s)');
print('| Mode | package:cronet | dart:io |');
print('| :-----------: |:--------------: | :-----------: |');
print('| JIT | ${jitCronetThroughput[1]} out of'
Expand Down
46 changes: 32 additions & 14 deletions benchmark/throughput.dart
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import 'dart:async';
import 'dart:io' as io;
import 'dart:math';

import 'package:args/args.dart';
import 'package:cronet/cronet.dart';

abstract class ThroughputBenchmark {
Expand Down Expand Up @@ -56,7 +57,7 @@ abstract class ThroughputBenchmark {

Future<List<int>> report() async {
var maxReturn = 0;
var throughput = List<int>.empty();
var throughput = [0, 0];
// Run the benchmark for 1, 2, 4...spawnThreshold.
for (int currentThreshold = 1;
currentThreshold <= spawnThreshold;
Expand Down Expand Up @@ -151,19 +152,36 @@ class CronetThroughputBenchmark extends ThroughputBenchmark {
}

void main(List<String> args) async {
// Accepts test url & parallel request threshold as optional cli parameter.
var url = 'https://example.com';
var spawnThreshold = 1024;
var duration = const Duration(seconds: 1);
if (args.isNotEmpty) {
url = args[0];
}
if (args.length > 1) {
spawnThreshold = pow(2, int.parse(args[1])).toInt();
}
if (args.length > 2) {
duration = Duration(seconds: int.parse(args[2]));
}
final parser = ArgParser();
parser
..addOption('url',
abbr: 'u',
help: 'The server to ping for running this benchmark.',
defaultsTo: 'https://example.com')
..addOption('limit',
abbr: 'l',
help: 'Limits the maximum number of parallel requests to 2^N where N'
' is provided through this option.',
defaultsTo: '10')
..addOption('time',
abbr: 't',
help: 'Maximum second(s) the benchmark should wait for each request.',
defaultsTo: '1')
..addFlag('help',
abbr: 'h', negatable: false, help: 'Print this usage information.');
final arguments = parser.parse(args);
if (arguments.wasParsed('help')) {
print(parser.usage);
return;
}
if (arguments.rest.isNotEmpty) {
print(parser.usage);
throw ArgumentError();
}
final url = arguments['url'] as String;
final spawnThreshold =
pow(2, int.parse(arguments['limit'] as String)).toInt();
final duration = Duration(seconds: int.parse(arguments['time'] as String));
// TODO: https://github.com/google/cronet.dart/issues/11
await CronetThroughputBenchmark.main(url, spawnThreshold, duration);
// Used as an delemeter while parsing output in run_all script.
Expand Down
72 changes: 51 additions & 21 deletions bin/setup.dart
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import 'dart:io';

import 'package:archive/archive.dart';
import 'package:args/command_runner.dart';
import 'package:cli_util/cli_logging.dart' show Ansi, Logger;
import 'package:cronet/src/constants.dart';
import 'package:cronet/src/third_party/ffigen/find_resource.dart';
Expand Down Expand Up @@ -183,32 +184,61 @@ void verifyCronetBinary() {
sample.deleteSync();
}

Future<void> main(List<String> args) async {
const docStr = """
dart run cronet:setup [option]
Downloads the cronet binaries.\n
clean\tClean downloaded or built binaries.
build\tBuilds the wrapper. Requires cmake.
verify\tVerifies the cronet binary.
""";
final logger = Logger.standard();
if (args.length > 1) {
logger.stderr('Expected 1 argument only.');
logger.stdout(docStr);
} else if (args.contains('-h')) {
logger.stdout(docStr);
} else if (args.contains('clean')) {
logger.stdout('cleaning...');
Directory(binaryStorageDir).deleteSync(recursive: true);
logger.stdout('Done!');
} else if (args.contains('build')) {
// Available Commands.

class BuildCommand extends Command<void> {
@override
String get description => 'Builds the wrapper binaries. Requires cmake.';

@override
String get name => 'build';

@override
void run() {
buildWrapper();
} else if (args.contains('verify')) {
}
}

class CleanCommand extends Command<void> {
@override
String get description => 'Cleans downloaded or built binaries.';

@override
String get name => 'clean';

@override
void run() {
print('cleaning...');
Directory(binaryStorageDir).deleteSync(recursive: true);
}
}

class VerifyCommand extends Command<void> {
@override
String get description => 'Verifies the cronet binary.';

@override
String get name => 'verify';

@override
void run() {
verifyCronetBinary();
} else {
}
}

Future<void> main(List<String> args) async {
final runner =
CommandRunner<void>('setup', 'Downloads/Builds the cronet binaries.');
runner
..addCommand(BuildCommand())
..addCommand(CleanCommand())
..addCommand(VerifyCommand());
if (args.isEmpty) {
// Targeting only 64bit OS. (At least for the time being.)
if (validPlatforms.contains('${Platform.operatingSystem}64')) {
await downloadCronetBinaries('${Platform.operatingSystem}64');
}
} else {
await runner.run(args);
}
}
4 changes: 4 additions & 0 deletions example/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
## Examples

- [Bare Minimum](https://github.com/google/cronet.dart#example)
- [Simple (Dart CLI)](https://github.com/google/cronet.dart/tree/main/example/)
2 changes: 1 addition & 1 deletion lib/src/dylib_handler.dart
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ DynamicLibrary loadDylib(String name) {
logger.stdout(
'To download the binaries, please run the following from the root of'
' your project:');
logger.stdout('${ansi.yellow}dart run cronet <platform>${ansi.none}');
logger.stdout('${ansi.yellow}dart run cronet:setup${ansi.none}');
logger.stdout('${ansi.green}Valid platforms are:');
for (final platform in validPlatforms) {
logger.stdout(platform);
Expand Down
7 changes: 4 additions & 3 deletions pubspec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,18 @@
# BSD-style license that can be found in the LICENSE file.

name: cronet
version: 0.0.2
version: 0.0.3
homepage: https://github.com/google/cronet.dart
description: Experimental Cronet dart bindings.

environment:
sdk: '>=2.12.0 <3.0.0'

dependencies:
ffi: ^1.0.0
ffi: ^1.1.2
path: ^1.8.0
cli_util: ^0.3.0
args: ^2.1.1
cli_util: ^0.3.3
archive: ^3.1.2

dev_dependencies:
Expand Down
2 changes: 1 addition & 1 deletion tool/update_bindings.dart
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

import 'dart:io';

void main(List<String> args) async {
void main() async {
final root =
Directory.fromUri(Platform.script).parent.parent.uri.toFilePath();

Expand Down