Skip to content

[tests] get task fails n seed 84799 #4167

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
Mpdreamz opened this issue Oct 21, 2019 · 5 comments
Closed

[tests] get task fails n seed 84799 #4167

Mpdreamz opened this issue Oct 21, 2019 · 5 comments

Comments

@Mpdreamz
Copy link
Member

build.bat seed:84799 integrate 7.0.0 "writable" "gettask"

The random seeded data throws:

[ 2019-10-21T17:13:26,386][DEBUG][o.e.a.b.TransportShardBulkAction] [writable-node-f5f9929200] [project-get-task][0] failed to execute bulk item (index) index {[project-get-task][_doc][Auer, Turner and Walter380664], source[n/a, actual length: [3.5kb], max length: 2kb]}
org.elasticsearch.index.mapper.MapperParsingException: failed to parse field [tags.added] of type [date] in document with id 'Auer, Turner and Walter380664'
at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:280) ~[elasticsearch-7.0.0.jar:7.0.0]

Found in #4163 https://github.com/elastic/elasticsearch-net/pull/4163/checks?check_run_id=268611435
@Mpdreamz
Copy link
Member Author

[2019-10-21T18:25:38,374][INFO ][o.e.t.LoggingTaskListener] [writable-node-5294029200] 3203 finished with response BulkByScrollResponse[took=7.1s,timed_out=false,sliceId=null,updated=0,created=500,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2019-10-21T18:25:39,377][INFO ][o.e.c.m.MetaDataCreateIndexService] [writable-node-5294029200] [nest-fluentasync-660dc1bc] creating index, cause [api], templates [nest_tests], shards [2]/[0], mappings []
[2019-10-21T18:25:40,134][DEBUG][o.e.a.b.TransportShardBulkAction] [writable-node-5294029200] [project-get-task][0] failed to execute bulk item (index) index {[project-get-task][_doc][Klocko - Conroy188046], source[n/a, actual length: [3.9kb], max length: 2kb]}
org.elasticsearch.index.mapper.MapperParsingException: failed to parse field [tags.added] of type [date] in document with id 'Klocko - Conroy188046'

From

build.bat seed:17572 integrate 7.0.0 "writable" "gettask"

@Mpdreamz Mpdreamz mentioned this issue Oct 21, 2019
Merged
@codebrain
Copy link
Contributor

build.bat seed:84799 integrate 7.0.0 "writable" "gettask"

The random seeded data throws:

[ 2019-10-21T17:13:26,386][DEBUG][o.e.a.b.TransportShardBulkAction] [writable-node-f5f9929200] [project-get-task][0] failed to execute bulk item (index) index {[project-get-task][_doc][Auer, Turner and Walter380664], source[n/a, actual length: [3.5kb], max length: 2kb]}
org.elasticsearch.index.mapper.MapperParsingException: failed to parse field [tags.added] of type [date] in document with id 'Auer, Turner and Walter380664'
at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:280) ~[elasticsearch-7.0.0.jar:7.0.0]

Found in #4163 https://github.com/elastic/elasticsearch-net/pull/4163/checks?check_run_id=268611435

Ran on my machine and the tests passed without error.

@codebrain
Copy link
Contributor

[2019-10-21T18:25:38,374][INFO ][o.e.t.LoggingTaskListener] [writable-node-5294029200] 3203 finished with response BulkByScrollResponse[took=7.1s,timed_out=false,sliceId=null,updated=0,created=500,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2019-10-21T18:25:39,377][INFO ][o.e.c.m.MetaDataCreateIndexService] [writable-node-5294029200] [nest-fluentasync-660dc1bc] creating index, cause [api], templates [nest_tests], shards [2]/[0], mappings []
[2019-10-21T18:25:40,134][DEBUG][o.e.a.b.TransportShardBulkAction] [writable-node-5294029200] [project-get-task][0] failed to execute bulk item (index) index {[project-get-task][_doc][Klocko - Conroy188046], source[n/a, actual length: [3.9kb], max length: 2kb]}
org.elasticsearch.index.mapper.MapperParsingException: failed to parse field [tags.added] of type [date] in document with id 'Klocko - Conroy188046'

From

build.bat seed:17572 integrate 7.0.0 "writable" "gettask"

Ran on my machine and the tests passed without error.

@russcam
Copy link
Contributor

russcam commented Oct 23, 2019

For

build.bat seed:84799 integrate 7.0.0 "writable" "gettask"

The random seeded data throws:

[ 2019-10-21T17:13:26,386][DEBUG][o.e.a.b.TransportShardBulkAction] [writable-node-f5f9929200] [project-get-task][0] failed to execute bulk item (index) index {[project-get-task][_doc][Auer, Turner and Walter380664], source[n/a, actual length: [3.5kb], max length: 2kb]}
org.elasticsearch.index.mapper.MapperParsingException: failed to parse field [tags.added] of type [date] in document with id 'Auer, Turner and Walter380664'
at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:280) ~[elasticsearch-7.0.0.jar:7.0.0]

Found in #4163 https://github.com/elastic/elasticsearch-net/pull/4163/checks?check_run_id=268611435

The date format in question is

2019-10-20T21:40:40.5+00:00

Note the single fractional value. Nest uses the DateTimeFormatter's Serialize method:

elasticsearch-net/src/Elasticsearch.Net/Utf8Json/Formatters/DateTimeFormatter.cs

Lines 97 to 259 in 86bcc2c

public void Serialize(ref JsonWriter writer, DateTime value, IJsonFormatterResolver formatterResolver)
{
var year = value.Year;
var month = value.Month;
var day = value.Day;
var hour = value.Hour;
var minute = value.Minute;
var second = value.Second;
var nanosec = value.Ticks % TimeSpan.TicksPerSecond;
const int baseLength = 19 + 2; // {YEAR}-{MONTH}-{DAY}T{Hour}:{Minute}:{Second} + quotation
const int nanosecLength = 8; // .{nanoseconds}
switch (value.Kind)
{
case DateTimeKind.Local:
// +{Hour}:{Minute}
writer.EnsureCapacity(baseLength + ((nanosec == 0) ? 0 : nanosecLength) + 6);
break;
case DateTimeKind.Utc:
// Z
writer.EnsureCapacity(baseLength + ((nanosec == 0) ? 0 : nanosecLength) + 1);
break;
case DateTimeKind.Unspecified:
default:
writer.EnsureCapacity(baseLength + ((nanosec == 0) ? 0 : nanosecLength));
break;
}
writer.WriteRawUnsafe((byte)'\"');
if (year < 10)
{
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
}
else if (year < 100)
{
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
}
else if (year < 1000)
{
writer.WriteRawUnsafe((byte)'0');
}
writer.WriteInt32(year);
writer.WriteRawUnsafe((byte)'-');
if (month < 10)
{
writer.WriteRawUnsafe((byte)'0');
}
writer.WriteInt32(month);
writer.WriteRawUnsafe((byte)'-');
if (day < 10)
{
writer.WriteRawUnsafe((byte)'0');
}
writer.WriteInt32(day);
writer.WriteRawUnsafe((byte)'T');
if (hour < 10)
{
writer.WriteRawUnsafe((byte)'0');
}
writer.WriteInt32(hour);
writer.WriteRawUnsafe((byte)':');
if (minute < 10)
{
writer.WriteRawUnsafe((byte)'0');
}
writer.WriteInt32(minute);
writer.WriteRawUnsafe((byte)':');
if (second < 10)
{
writer.WriteRawUnsafe((byte)'0');
}
writer.WriteInt32(second);
if (nanosec != 0)
{
writer.WriteRawUnsafe((byte)'.');
if (nanosec < 10)
{
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
}
else if (nanosec < 100)
{
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
}
else if (nanosec < 1000)
{
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
}
else if (nanosec < 10000)
{
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
}
else if (nanosec < 100000)
{
writer.WriteRawUnsafe((byte)'0');
writer.WriteRawUnsafe((byte)'0');
}
else if (nanosec < 1000000)
{
writer.WriteRawUnsafe((byte)'0');
}
writer.WriteInt64(nanosec);
}
switch (value.Kind)
{
case DateTimeKind.Local:
// should not use `BaseUtcOffset` - https://stackoverflow.com/questions/10019267/is-there-a-generic-timezoneinfo-for-central-europe
var localOffset = TimeZoneInfo.Local.GetUtcOffset(value);
var minus = (localOffset < TimeSpan.Zero);
if (minus) localOffset = localOffset.Negate();
var h = localOffset.Hours;
var m = localOffset.Minutes;
writer.WriteRawUnsafe(minus ? (byte)'-' : (byte)'+');
if (h < 10)
{
writer.WriteRawUnsafe((byte)'0');
}
writer.WriteInt32(h);
writer.WriteRawUnsafe((byte)':');
if (m < 10)
{
writer.WriteRawUnsafe((byte)'0');
}
writer.WriteInt32(m);
break;
case DateTimeKind.Utc:
writer.WriteRawUnsafe((byte)'Z');
break;
case DateTimeKind.Unspecified:
default:
break;
}
writer.WriteRawUnsafe((byte)'\"');
}

which pads nanoseconds, so this error is surprising. Need to dig into further

@russcam russcam mentioned this issue Oct 23, 2019
Merged
@russcam
Copy link
Contributor

russcam commented Oct 23, 2019

Closing this as #4175 is merged

@russcam russcam closed this as completed Oct 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@codebrain @russcam @Mpdreamz