-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
update gherkin etc, validate against cck #1318
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
134 commits
Select commit
Hold shift + click to select a range
7062f4e
add cck
davidjgoss bca4d7d
add a script to run cck tests
davidjgoss 9e66fe6
direct message formatter output to file
davidjgoss f97930e
test via mocha with one run per feature
davidjgoss f239f9f
add compatibility dir to eslint
davidjgoss 7510313
step defs for attachments suite
davidjgoss 2945b96
add World and IWorldOptions to public API
davidjgoss 38149e1
add data-tables steps
davidjgoss 7f746ae
exclude uris from the cck diff
davidjgoss 5591c70
implement DataTable::transpose
davidjgoss aad5595
implement steps for remaining cck bits
davidjgoss 9948792
add dependency-lint setup for new cck stuff
davidjgoss 29145d1
factor out ids for now
davidjgoss 8679afe
add pre script for cck
davidjgoss ff69321
merge master
davidjgoss 4e2f1ee
update gherkin and messages deps
davidjgoss 7cd14db
add dep on cucumber query
davidjgoss e2db9bf
allow json modules to be imported
davidjgoss dda37cc
first big wave of changes to get things compiling/running
davidjgoss 6b9db47
stop emitting pickleAccepted and pickleRejected messages
davidjgoss 3dac234
implement lookup of worst result in event data collector
davidjgoss 340ca4d
fix some of pickle runner spec
davidjgoss 1ea2ac9
mostly fix up progress_bar_formatter
davidjgoss d2cc938
remove superfluous gitignore
davidjgoss c79c365
hook up cck profile so it doesnt pick up feature-test support code
davidjgoss fac9203
emit messages for step definitions
davidjgoss ebe7795
stop ignoring ids so we can see what doesn't line up
davidjgoss 2cd89be
import cck stuff from monorepo for now
davidjgoss 1eda297
emi meta message
davidjgoss 09d0f08
refactor `emitSupportCodeMessages` to take whole library
davidjgoss 540a9f9
emit before/after hooks
davidjgoss 75cf679
emit step definitions before hooks
davidjgoss dd04c06
emit beforeAll and afterAll hooks too
davidjgoss 25569e0
emit a zero timestamp within testRunStarted
davidjgoss fffe791
add `durations` package
davidjgoss 6779fe2
start hooking up a stopwatch for test run timestamps
davidjgoss 3dc0726
emit timestamps from pickle runner
davidjgoss 5bb68cb
start to sort out attachments cck
davidjgoss dd0d6e4
add `log` function that attaches as cucumber log
davidjgoss 9942d76
fix some unit test issues
davidjgoss ea23311
emit correct message structure for attachments
davidjgoss 46e9c23
dont exclude body any more
davidjgoss 970da95
merge master
davidjgoss cc1b57e
formatting and lint
davidjgoss 6c22da7
fix config builder test
davidjgoss 40d3ab9
fix helpers spec
davidjgoss 99dc049
fix tests around attachments
davidjgoss 68aa293
fix what pickle runner spec is testing (still fails tho)
davidjgoss 4ce14b5
fix stopwatch in parallel runtime
davidjgoss 07cc622
fix coordinator maybe
davidjgoss a775c07
fix wrong structure in test
davidjgoss 7713cae
fix the stopwatch
davidjgoss 3e8ee19
clean up, remove comment
davidjgoss 9773e62
add willBeRetried to testStepFinished result, dont try to emit result…
davidjgoss a8acb5b
add a todo comment because this is bad
davidjgoss f8a686f
fix pickle runner test
davidjgoss a43f23f
remove todo comments that are done
davidjgoss 5322922
merge master
davidjgoss b918f3b
if you say so yarn
davidjgoss a1f026c
use a map to make attachment steps clearer
davidjgoss cd6492d
split out a function to get timestamps for assertions on messages
davidjgoss 327cea6
remove ref to pickleAccepted from test
davidjgoss 0a9a02a
filter off meta messages from formatter feature tests
davidjgoss d230704
update rejected pickle message fixture
davidjgoss 0df7b4f
update passed scenario message fixture
davidjgoss ab28122
update passed rule message fixture
davidjgoss 4e71a3d
update failed message fixture
davidjgoss 841000d
update retried message fixture
davidjgoss 638499f
fix wrong path in fixtures
davidjgoss df5e703
emit messages for parameter types
davidjgoss fadc831
fix json formatter handling of attachments and add a unit test
davidjgoss 995beae
merge master
davidjgoss 50e73f6
use latest expressions packages, fix a couple of usages
davidjgoss 6c02576
update all the monorepo deps
davidjgoss e215d8d
fix some test code re protbuf and messages
davidjgoss 1c08d8f
bring attachments cck up to date
davidjgoss 6535fd5
update cck fixture for datatable
davidjgoss 25ec4bc
update cck fixture for hooks
davidjgoss ff94a72
update other cck fixtures
davidjgoss 8e6a9f3
add new cck specs
davidjgoss d3e0b4e
first pass at supporting stepMatchArgumentLists
davidjgoss 6760bb7
handle children in arg group
davidjgoss 5408611
fix existing pickle runner tests
davidjgoss eeca00c
add test for stepMatchArgumentLists
davidjgoss f3bd88b
update fixtures
davidjgoss af242fa
use new library to emit meta
davidjgoss 2f02b12
remove success bool from testRunFinished message
davidjgoss e20e81b
Merge branch 'master' into cck
davidjgoss de48fc3
fix formatter output helper for renamed step result structure, amend …
davidjgoss d15bb32
fix handling of gherkin parse error to line up with its changes
davidjgoss 583f4ee
update monorepo deps again
davidjgoss 427a23c
add downlevelIteration to make browser example work
davidjgoss 8f1eac7
docs and change log
davidjgoss a31ac17
normalize uris in support code messages
davidjgoss 6d1e92a
also output messages from feature test, for conveninent debugging
davidjgoss d1a34df
bump gherkin again
davidjgoss 45391bc
bump cck dep
davidjgoss 5abf80b
remove local versions of features and fixtures
davidjgoss 1879b29
run against features and fixtures in cck dep
davidjgoss bbd3ccc
emit undefined params
davidjgoss db8baa0
whoops
davidjgoss 44a552c
some cck starting to pass by ignoring ids and time
davidjgoss faa27df
fix testCase ordering in the test
davidjgoss c016cb6
undefined/ambiguous/skipped test steps emit with a zero duration
davidjgoss 3bcba31
remove temp exclusion
davidjgoss 885fdc5
dont compare error messages yet - need to figure out how to make this…
davidjgoss 0246210
emit after/afterall in original declaration order
davidjgoss 276aa86
add cck tests to main test script
davidjgoss fb61a3b
fix usage formatter to handle zero durations on skipped steps
davidjgoss 29e65d2
better way to fix usage formatter
davidjgoss 316be7c
change up order of feature
davidjgoss 2ddaaf1
keep after(all) hooks in original order in support code library, reve…
davidjgoss 3fb68a4
install html-formatter dep
davidjgoss 56b2369
get minimal feature running and outputting to a file
davidjgoss 7ee80e1
move feature patb to profile
davidjgoss 3c0c235
run-script to pipe to html-formatter
davidjgoss 84139c4
workaround local install issue using npx
davidjgoss 17c28ee
better naming, add to main test script
davidjgoss 3fa94d8
include majority of cck in formatters test
davidjgoss 67d3a31
include cck in coverage, test formatters after coverage run
davidjgoss b823bc8
Merge remote-tracking branch 'origin/master' into cck
davidjgoss ad710a9
bump gherkin again
davidjgoss b6a3884
log undefined param types in summary formatter, fix test
davidjgoss a13f281
formatters test last in both versions
davidjgoss 0f0af99
progress bar formatter too
davidjgoss 56fdd79
fix time reporting
charlierudolph 33bbe81
add missing features to formatters run
davidjgoss 819023e
remove globby dep, just use glob
davidjgoss a228ec4
split seconds and nanos correctly in stopwatch
davidjgoss 8246870
report test run timestamp + step duration sum
charlierudolph 9e660f8
fix parallel time
charlierudolph 3306709
fix lint
charlierudolph 650c284
fix unit tests
charlierudolph c1541c0
fix lint
charlierudolph File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,97 @@ | ||
import { describe, it } from 'mocha' | ||
import { config, expect, use } from 'chai' | ||
import chaiExclude from 'chai-exclude' | ||
import glob from 'glob' | ||
import fs from 'fs' | ||
import ndjsonParse from 'ndjson-parse' | ||
import path from 'path' | ||
import { PassThrough } from 'stream' | ||
import { Cli } from '../lib' | ||
import toString from 'stream-to-string' | ||
import { doesHaveValue, doesNotHaveValue } from '../src/value_checker' | ||
import { normalizeMessageOutput } from '../features/support/formatter_output_helpers' | ||
|
||
const PROJECT_PATH = path.join(__dirname, '..') | ||
const CCK_FEATURES_PATH = 'node_modules/@cucumber/compatibility-kit/features' | ||
const CCK_IMPLEMENTATIONS_PATH = 'compatibility/features' | ||
|
||
config.truncateThreshold = 100 | ||
use(chaiExclude) | ||
|
||
describe('Cucumber Compatibility Kit', () => { | ||
glob.sync(`${CCK_FEATURES_PATH}/**/*.ndjson`).forEach((fixturePath) => { | ||
const suiteName = /^.+\/(.+)\.ndjson$/.exec(fixturePath)[1] | ||
it(`passes the cck suite for '${suiteName}'`, async () => { | ||
const args = [ | ||
'node', | ||
path.join(PROJECT_PATH, 'bin', 'cucumber-js'), | ||
].concat([ | ||
`${CCK_FEATURES_PATH}/${suiteName}/${suiteName}.feature`, | ||
'--require', | ||
`${CCK_IMPLEMENTATIONS_PATH}/${suiteName}/${suiteName}.ts`, | ||
'--profile', | ||
'cck', | ||
]) | ||
const stdout = new PassThrough() | ||
try { | ||
await new Cli({ | ||
argv: args, | ||
cwd: PROJECT_PATH, | ||
stdout, | ||
}).run() | ||
} catch (ignored) { | ||
console.error(ignored) | ||
} | ||
stdout.end() | ||
|
||
const rawOutput = await toString(stdout) | ||
const actualMessages = normalize(ndjsonParse(rawOutput)) | ||
const expectedMessages = ndjsonParse( | ||
fs.readFileSync(fixturePath, { encoding: 'utf-8' }) | ||
) | ||
expect(actualMessages) | ||
.excludingEvery([ | ||
'meta', | ||
// sources | ||
'uri', | ||
'line', | ||
// ids | ||
'astNodeId', | ||
'astNodeIds', | ||
'hookId', | ||
'id', | ||
'pickleId', | ||
'pickleStepId', | ||
'stepDefinitionIds', | ||
'testCaseId', | ||
'testCaseStartedId', | ||
'testStepId', | ||
// time | ||
'nanos', | ||
'seconds', | ||
// errors | ||
'message', | ||
]) | ||
.to.deep.eq(expectedMessages) | ||
}) | ||
}) | ||
}) | ||
|
||
function normalize(messages: any[]): any[] { | ||
messages = normalizeMessageOutput( | ||
messages, | ||
path.join(PROJECT_PATH, 'compatibility') | ||
) | ||
const testCases: any[] = messages.filter((message) => | ||
doesHaveValue(message.testCase) | ||
) | ||
const everythingElse: any[] = messages.filter((message) => | ||
doesNotHaveValue(message.testCase) | ||
) | ||
const testRunStarted = everythingElse.findIndex((message) => | ||
doesHaveValue(message.testRunStarted) | ||
) | ||
// move all `testCase` messages to just after `testRunStarted` | ||
davidjgoss marked this conversation as resolved.
Show resolved
Hide resolved
|
||
everythingElse.splice(testRunStarted + 1, 0, ...testCases) | ||
return everythingElse | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,75 @@ | ||
import { Before, When, World } from '../../..' | ||
import { ReadableStreamBuffer } from 'stream-buffers' | ||
import fs from 'fs' | ||
import path from 'path' | ||
|
||
Before((): void => undefined) | ||
davidjgoss marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
When('the string {string} is attached as {string}', async function ( | ||
this: World, | ||
text: string, | ||
mediaType: string | ||
) { | ||
await this.attach(text, mediaType) | ||
}) | ||
|
||
When('the string {string} is logged', async function ( | ||
this: World, | ||
text: string | ||
) { | ||
await this.log(text) | ||
}) | ||
|
||
When('text with ANSI escapes is logged', async function (this: World) { | ||
await this.log( | ||
'This displays a \x1b[31mr\x1b[0m\x1b[91ma\x1b[0m\x1b[33mi\x1b[0m\x1b[32mn\x1b[0m\x1b[34mb\x1b[0m\x1b[95mo\x1b[0m\x1b[35mw\x1b[0m' | ||
) | ||
}) | ||
|
||
When('the following string is attached as {string}:', async function ( | ||
this: World, | ||
mediaType: string, | ||
text: string | ||
) { | ||
await this.attach(text, mediaType) | ||
}) | ||
|
||
When('an array with {int} bytes is attached as {string}', async function ( | ||
this: World, | ||
size: number, | ||
mediaType: string | ||
) { | ||
const data = [...Array(size).keys()] | ||
const buffer = Buffer.from(data) | ||
await this.attach(buffer, mediaType) | ||
}) | ||
|
||
When('a stream with {int} bytes are attached as {string}', async function ( | ||
this: World, | ||
size: number, | ||
mediaType: string | ||
) { | ||
const data = [...Array(size).keys()] | ||
const buffer = Buffer.from(data) | ||
const stream = new ReadableStreamBuffer({ chunkSize: 1, frequency: 1 }) | ||
stream.put(buffer) | ||
stream.stop() | ||
await this.attach(stream, mediaType) | ||
}) | ||
|
||
When('a JPEG image is attached', async function (this: World) { | ||
await this.attach( | ||
fs.createReadStream( | ||
path.join( | ||
process.cwd(), | ||
'node_modules', | ||
'@cucumber', | ||
'compatibility-kit', | ||
'features', | ||
'attachments', | ||
'cucumber-growing-on-vine.jpg' | ||
) | ||
), | ||
'image/jpg' | ||
) | ||
}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
import { When, Then, DataTable } from '../../..' | ||
import { expect } from 'chai' | ||
|
||
When('the following table is transposed:', function ( | ||
this: any, | ||
table: DataTable | ||
) { | ||
this.transposed = table.transpose() | ||
}) | ||
|
||
Then('it should be:', function (this: any, expected: DataTable) { | ||
expect(this.transposed.raw()).to.deep.eq(expected.raw()) | ||
}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
import assert from 'assert' | ||
import { Given, When, Then } from '../../..' | ||
|
||
Given('there are {int} cucumbers', function (this: any, initialCount: number) { | ||
this.count = initialCount | ||
}) | ||
|
||
When('I eat {int} cucumbers', function (this: any, eatCount: number) { | ||
this.count -= eatCount | ||
}) | ||
|
||
Then('I should have {int} cucumbers', function ( | ||
this: any, | ||
expectedCount: number | ||
) { | ||
assert.strictEqual(this.count, expectedCount) | ||
}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
import { When, Before, After } from '../../..' | ||
|
||
Before(function () { | ||
// no-op | ||
}) | ||
|
||
When('a step passes', function () { | ||
// no-op | ||
}) | ||
|
||
When('a step throws an exception', function () { | ||
throw new Error('Exception in step') | ||
}) | ||
|
||
After(function () { | ||
throw new Error('Exception in hook') | ||
}) | ||
|
||
After('@some-tag or @some-other-tag', function () { | ||
throw new Error('Exception in conditional hook') | ||
}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
import assert from 'assert' | ||
import { Given } from '../../..' | ||
|
||
Given('I have {int} cukes in my belly', function (cukeCount: number) { | ||
assert(cukeCount) | ||
}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
import { Given, defineParameterType } from '../../..' | ||
import { expect } from 'chai' | ||
|
||
class Flight { | ||
constructor(public readonly from: string, public readonly to: string) {} | ||
} | ||
|
||
defineParameterType({ | ||
name: 'flight', | ||
regexp: /([A-Z]{3})-([A-Z]{3})/, | ||
transformer(from: string, to: string) { | ||
return new Flight(from, to) | ||
}, | ||
}) | ||
|
||
Given('{flight} has been delayed {int} minutes', function ( | ||
flight: Flight, | ||
delay: number | ||
) { | ||
expect(flight.from).to.eq('LHR') | ||
expect(flight.to).to.eq('CDG') | ||
}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
import assert from 'assert' | ||
import { Given, When, Then } from '../../..' | ||
|
||
Given('there are {int} {float} coins inside', function ( | ||
count: number, | ||
denomination: number | ||
) { | ||
// TODO: implement this | ||
assert(count) | ||
assert(denomination) | ||
}) | ||
|
||
Given('there are no chocolates inside', function () { | ||
// TODO: implement this | ||
}) | ||
|
||
Given('there are {int} chocolates inside', function (chocolateCount: number) { | ||
// TODO: implement this | ||
assert(chocolateCount) | ||
}) | ||
|
||
When( | ||
'the customer tries to buy a {float} chocolate with a {float} coin', | ||
function (price: number, paid: number) { | ||
// TODO: implement this | ||
assert(price) | ||
assert(paid) | ||
} | ||
) | ||
|
||
Then('the sale should not happen', function () { | ||
// TODO: implement this | ||
}) | ||
|
||
Then("the customer's change should be {int} {float} coin(s)", function ( | ||
count: number, | ||
denomination: number | ||
) { | ||
// TODO: implement this | ||
assert(count) | ||
assert(denomination) | ||
}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
import { When } from '../../..' | ||
|
||
When('a step throws an exception', function () { | ||
throw new Error('BOOM') | ||
}) |
6 changes: 6 additions & 0 deletions
6
compatibility/features/unknown-parameter-type/unknown-parameter-type.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
import { Given } from '../../..' | ||
|
||
// eslint-disable-next-line @typescript-eslint/no-unused-vars | ||
Given('{airport} is closed because of a strike', function (airport: any) { | ||
throw new Error('Should not be called because airport type not defined') | ||
}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,13 +1,45 @@ | ||
var common = [ | ||
const feature = [ | ||
'--require-module ts-node/register', | ||
'--require features/**/*.ts', | ||
`--format ${ | ||
process.env.CI || !process.stdout.isTTY ? 'progress' : 'progress-bar' | ||
}`, | ||
'--format rerun:@rerun.txt', | ||
'--format usage:usage.txt', | ||
'--format message:messages.ndjson', | ||
].join(' ') | ||
|
||
const cck = [ | ||
'--require-module', | ||
'ts-node/register', | ||
'--format', | ||
'message', | ||
].join(' ') | ||
|
||
const FORMATTERS_INCLUDE = [ | ||
'attachments', | ||
'data-tables', | ||
'examples-tables', | ||
'minimal', | ||
'parameter-types', | ||
'rules', | ||
'stack-traces', | ||
] | ||
davidjgoss marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
const formatters = [ | ||
`node_modules/@cucumber/compatibility-kit/features/{${FORMATTERS_INCLUDE.join( | ||
',' | ||
)}}/*.feature`, | ||
'--require-module', | ||
'ts-node/register', | ||
'--require', | ||
`compatibility/features/{${FORMATTERS_INCLUDE.join(',')}}/*.ts`, | ||
'--format', | ||
'message', | ||
].join(' ') | ||
|
||
module.exports = { | ||
default: common, | ||
default: feature, | ||
cck, | ||
formatters, | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are we expecting all of these to be excluded long term? I would hope with predictable ids that big block could be fixed. Do you think its worth having an issue to try to drive this list down.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree - when I was working against a slightly earlier CDK version I think it was only the
testCase
ordering that made the ids not line up.But cck has changed from using predictable ids to uuids so not sure what the intent is there or where that leaves us.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The predictable ids and timestamps are a dead end. The cck no longer uses them.
It's a dead end because various Cucumber implementations emit messages in slightly different order than what
fake-cucumber
(the engine behind the cck) emits. It would be too much work to make them all emit in the same order.So what we opted for instead is a validation mechanism where certain values (ids, timestamps) are not compared. We pair this with smoke tests that send the generated messages through 2 of the most advanced message consumers we have - cucumber-html and json-formatter. If these don't crash and a cursory eye-balling looks OK, we're confident enough that the emitted messages are ok.