Skip to content

Update type selection for string parameters #218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 23, 2020

Conversation

bpintea
Copy link
Collaborator

@bpintea bpintea commented Mar 11, 2020

This PR changes the way a parameter's SQL type is resolved to an ES
type. So far, the size of the parameter was taken into account, to
select the ES type with the lowest "precision" that would accomodate it.

This worked well so far because IP type was having a precision of 0 (=ignored).
This precision for IP type had been updated in Elasticsearch (to maximum
IP length), which would lead to the driver tagging small length strings
as IP types.

With current change, all string type parameterss are considered as TEXT
types, leaving Elasticsearch/SQL to apply a better, context-informed
conversion.

The PR also extends the size of binary-to-hexa logging and does a better
cleaning with "proper" build script parameter.

bpintea added 4 commits March 11, 2020 21:08
When invoking 'proper', the build script now invokes the build scripts
for both x86 and x64, if any of them are present.
wmi application used to uninstall driver by name during integration
testing can take over 1.5 minutes on slower VMs with many installed
applications.
When extended logging debugging define is enabled, the hex dumping of
binary buffers (used for CBOR replies logging) are now also dynamically
allocated and significantly larger (4k -> 4M).
This allows easier troubleshooting.
This commit changes the way a parameter's SQL type is resolved to an ES
type. So far, the size of the parameter was taking into account, to
select the ES type with the lowest "precision" that would accomodate it.

This worked well so far because IP type was having a precision of 0.
This precision for IP type had been updated in Elasticsearch (to maximum
IP length), which would lead to the driver tagging small length strings
as IP types.

With current change, all string type parameterss are considered as TEXT
types, leaving Elasticsearch/SQL to apply a better, context-informed
conversion.
Copy link
Contributor

@droberts195 droberts195 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bpintea bpintea merged commit 290e287 into elastic:master Mar 23, 2020
@bpintea bpintea deleted the fix/text_es_type_selection branch March 23, 2020 16:56
bpintea added a commit that referenced this pull request Mar 30, 2020
* proper cleans all builds

When invoking 'proper', the build script now invokes the build scripts
for both x86 and x64, if any of them are present.

* inc uninstallation timeout while integ-testing

wmi application used to uninstall driver by name during integration
testing can take over 1.5 minutes on slower VMs with many installed
applications.

* extend logging buffers for (CBOR) hex printers

When extended logging debugging define is enabled, the hex dumping of
binary buffers (used for CBOR replies logging) are now also dynamically
allocated and significantly larger (4k -> 4M).
This allows easier troubleshooting.

* change the ES type selection for text parameters

This commit changes the way a parameter's SQL type is resolved to an ES
type. So far, the size of the parameter was taking into account, to
select the ES type with the lowest "precision" that would accomodate it.

This worked well so far because IP type was having a precision of 0.
This precision for IP type had been updated in Elasticsearch (to maximum
IP length), which would lead to the driver tagging small length strings
as IP types.

With current change, all string type parameterss are considered as TEXT
types, leaving Elasticsearch/SQL to apply a better, context-informed
conversion.

(cherry picked from commit 290e287)
bpintea added a commit that referenced this pull request Mar 30, 2020
* proper cleans all builds

When invoking 'proper', the build script now invokes the build scripts
for both x86 and x64, if any of them are present.

* inc uninstallation timeout while integ-testing

wmi application used to uninstall driver by name during integration
testing can take over 1.5 minutes on slower VMs with many installed
applications.

* extend logging buffers for (CBOR) hex printers

When extended logging debugging define is enabled, the hex dumping of
binary buffers (used for CBOR replies logging) are now also dynamically
allocated and significantly larger (4k -> 4M).
This allows easier troubleshooting.

* change the ES type selection for text parameters

This commit changes the way a parameter's SQL type is resolved to an ES
type. So far, the size of the parameter was taking into account, to
select the ES type with the lowest "precision" that would accomodate it.

This worked well so far because IP type was having a precision of 0.
This precision for IP type had been updated in Elasticsearch (to maximum
IP length), which would lead to the driver tagging small length strings
as IP types.

With current change, all string type parameterss are considered as TEXT
types, leaving Elasticsearch/SQL to apply a better, context-informed
conversion.

(cherry picked from commit 290e287)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants