Skip to content

Add incremental compilation. #198

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
Jan 13, 2019
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
b17381c
extracted interface from IncrementalChecker.
0xorial Dec 27, 2018
67f737c
got something working with TypeScript incremental compilation API.
0xorial Dec 27, 2018
b8eb249
added some linting.
0xorial Dec 27, 2018
c5d7216
got new checker passing old integration tests.
0xorial Dec 28, 2018
654afab
more reliable way of finding if compilation started.
0xorial Dec 28, 2018
e5774ed
keep errors from previous run.
0xorial Dec 28, 2018
53d690b
added some tests to incremental typescript compiler.
0xorial Dec 30, 2018
591f55a
added some vue support.
0xorial Dec 30, 2018
666d3d6
output compilation time.
0xorial Jan 6, 2019
6db2175
add option to disable time measurement and use logger for output.
0xorial Jan 10, 2019
9df72d0
added some docs.
0xorial Jan 10, 2019
e94b062
added some comments.
0xorial Jan 10, 2019
6af7dfb
fix 'should find the same errors on multi-process mode' test
johnnyreilly Jan 12, 2019
0b8b09b
improve WorkSet typing - bye bye any
johnnyreilly Jan 12, 2019
553e789
update changelog / version
johnnyreilly Jan 12, 2019
d54b2c1
name incremental api testpack
johnnyreilly Jan 12, 2019
03230ba
fixed useCaseSensitiveFileNames.
0xorial Jan 12, 2019
cbdd847
added tests comments.
0xorial Jan 12, 2019
b6ee86c
vue tests for incremental API are actually testing inremental API now.
0xorial Jan 12, 2019
1b1dec7
added tslintAutofix test.
0xorial Jan 12, 2019
e847ddd
reorganise tests
johnnyreilly Jan 12, 2019
07a4bd2
split out tests
johnnyreilly Jan 12, 2019
8a48402
run common tests with useTypescriptIncrementalApi both true and false
johnnyreilly Jan 12, 2019
7bb0fb2
removed typescript-collections.
0xorial Jan 12, 2019
2b0b680
added should find semantic errors test
johnnyreilly Jan 13, 2019
0030218
upgrade deps
johnnyreilly Jan 13, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,15 @@ number **can increase checking time**. Default: `ForkTsCheckerWebpackPlugin.ONE_
If `true`, the linter and compiler will process VueJs single-file-component (.vue) files. See the
[Vue section](https://github.com/Realytics/fork-ts-checker-webpack-plugin#vue) further down for information on how to correctly setup your project.

* **useTypescriptIncrementalApi** `boolean`:
If true, the plugin will use incremental compilation API introduced in typescript 2.7. In this mode you can only have 1
worker, but if the changes in your code are small (like you normally have when you work in 'watch' mode), the compilation
may be much faster, even compared to multi-threaded compilation.

* **measureCompilationTime** `boolean`:
If true, the plugin will measure the time spent inside the compilation code. This may be useful to compare modes,
especially if there are other loaders/plugins involved in the compilation.

### Pre-computed consts:
* `ForkTsCheckerWebpackPlugin.ONE_CPU` - always use one CPU
* `ForkTsCheckerWebpackPlugin.ALL_CPUS` - always use all CPUs (will increase build time)
Expand Down
6 changes: 4 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@
"test:watch": "mocha -R spec --watch ./test/unit",
"test:coverage": "rimraf coverage && istanbul cover -root lib --include-all-sources mocha -- -R spec ./test/unit ./test/integration",
"lint": "tslint --project src/tsconfig.json && eslint ./test",
"lint:fix": "tslint --project src/tsconfig.json --fix && eslint ./test --fix"
"lint:fix": "tslint --project src/tsconfig.json --fix && eslint ./test --fix",
"watch": "tsc --version && tsc --project \"./src\" --watch"
},
"repository": {
"url": "https://github.com/Realytics/fork-ts-checker-webpack-plugin.git",
Expand Down Expand Up @@ -88,7 +89,8 @@
"chokidar": "^2.0.4",
"micromatch": "^3.1.10",
"minimatch": "^3.0.4",
"tapable": "^1.0.0"
"tapable": "^1.0.0",
"typescript-collections": "^1.3.2"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This dependency doesn't seems to be used, was it maybe included for the linked list at one point?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are right. removed.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your awesome work, I am looking forward to trying this feature out when it is merged :)

},
"husky": {
"hooks": {
Expand Down
150 changes: 150 additions & 0 deletions src/ApiIncrementalChecker.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
import * as ts from 'typescript';
import * as minimatch from 'minimatch';
import * as path from 'path';
import { IncrementalCheckerInterface } from './IncrementalCheckerInterface';
import { CancellationToken } from './CancellationToken';
import { NormalizedMessage } from './NormalizedMessage';
import { Configuration, Linter, LintResult } from 'tslint';
import { CompilerHost } from './CompilerHost';
import { FsHelper } from './FsHelper';

// Need some augmentation here - linterOptions.exclude is not (yet) part of the official
// types for tslint.
interface ConfigurationFile extends Configuration.IConfigurationFile {
linterOptions?: {
typeCheck?: boolean;
exclude?: string[];
};
}

export class ApiIncrementalChecker implements IncrementalCheckerInterface {
private linterConfig?: ConfigurationFile;

private readonly tsIncrementalCompiler: CompilerHost;
private linterExclusions: minimatch.IMinimatch[] = [];

private currentLintErrors = new Map<string, LintResult>();
private lastUpdatedFiles: string[] = [];
private lastRemovedFiles: string[] = [];

constructor(
programConfigFile: string,
compilerOptions: ts.CompilerOptions,
private linterConfigFile: string | false,
private linterAutoFix: boolean,
checkSyntacticErrors: boolean
) {
this.initLinterConfig();

this.tsIncrementalCompiler = new CompilerHost(
programConfigFile,
compilerOptions,
checkSyntacticErrors
);
}

private initLinterConfig() {
if (!this.linterConfig && this.linterConfigFile) {
this.linterConfig = ApiIncrementalChecker.loadLinterConfig(
this.linterConfigFile
);

if (
this.linterConfig.linterOptions &&
this.linterConfig.linterOptions.exclude
) {
// Pre-build minimatch patterns to avoid additional overhead later on.
// Note: Resolving the path is required to properly match against the full file paths,
// and also deals with potential cross-platform problems regarding path separators.
this.linterExclusions = this.linterConfig.linterOptions.exclude.map(
pattern => new minimatch.Minimatch(path.resolve(pattern))
);
}
}
}

private static loadLinterConfig(configFile: string): ConfigurationFile {
const tslint = require('tslint');

return tslint.Configuration.loadConfigurationFromPath(
configFile
) as ConfigurationFile;
}

private createLinter(program: ts.Program): Linter {
const tslint = require('tslint');

return new tslint.Linter({ fix: this.linterAutoFix }, program);
}

public hasLinter(): boolean {
return !!this.linterConfig;
}

public isFileExcluded(filePath: string): boolean {
return (
filePath.endsWith('.d.ts') ||
this.linterExclusions.some(matcher => matcher.match(filePath))
);
}

public nextIteration() {
// do nothing
}

public async getDiagnostics(_cancellationToken: CancellationToken) {
const diagnostics = await this.tsIncrementalCompiler.processChanges();
this.lastUpdatedFiles = diagnostics.updatedFiles;
this.lastRemovedFiles = diagnostics.removedFiles;

return NormalizedMessage.deduplicate(
diagnostics.results.map(NormalizedMessage.createFromDiagnostic)
);
}

public getLints(_cancellationToken: CancellationToken) {
if (!this.linterConfig) {
return [];
}

for (const updatedFile of this.lastUpdatedFiles) {
if (this.isFileExcluded(updatedFile)) {
continue;
}

try {
const linter = this.createLinter(
this.tsIncrementalCompiler.getProgram()
);
// const source = fs.readFileSync(updatedFile, 'utf-8');
linter.lint(updatedFile, undefined!, this.linterConfig);
const lints = linter.getResult();
this.currentLintErrors.set(updatedFile, lints);
} catch (e) {
if (
FsHelper.existsSync(updatedFile) &&
// check the error type due to file system lag
!(e instanceof Error) &&
!(e.constructor.name === 'FatalError') &&
!(e.message && e.message.trim().startsWith('Invalid source file'))
) {
// it's not because file doesn't exist - throw error
throw e;
}
}

for (const removedFile of this.lastRemovedFiles) {
this.currentLintErrors.delete(removedFile);
}
}

const allLints = [];
for (const [, value] of this.currentLintErrors) {
allLints.push(...value.failures);
}

return NormalizedMessage.deduplicate(
allLints.map(NormalizedMessage.createFromLint)
);
}
}
Loading