Skip to content

[Transform] transform fails with "specified fields can't be null or empty" #67333

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
hendrikmuhs opened this issue Jan 12, 2021 · 1 comment · Fixed by #67334
Closed

[Transform] transform fails with "specified fields can't be null or empty" #67333

hendrikmuhs opened this issue Jan 12, 2021 · 1 comment · Fixed by #67334
Labels

Comments

@hendrikmuhs
Copy link

hendrikmuhs commented Jan 12, 2021

found by @szabosteve

Affected versions: 7.9.3 - 7.11.0

The transform fails calling _preview or on _start with the error message: specified fields can't be null or empty. This happens for transforms that don't use any field parameter in either group_by or aggregation, e.g. because in group_by/aggregation only scripts are used or a filter aggregation without a child aggregation (for a full example see below).

Mitigation:

Create the destination index, before starting the transform yourself, e.g. PUT myindex.

or

Add an aggregation that uses a field into your config, e.g:

    "aggregations": {
      "count": {
        "value_count": {
          "field": "my_grouping_field"
        }
      },

Solution:

The bug is a regression of #62945. In a special case transform tries to retrieve field capabilities for 0 fields, which is not allowed.

full failure example:

POST _transform/_preview
{
  "source": {
    "index": [
      "kibana_sample_data_logs"
    ]
  },
  "pivot": {
    "group_by": {
      "agent": {
        "terms": {
          "script": {
            "source": """String agent = doc['agent.keyword'].value;
            if (agent.contains("MSIE")) {
              return "internet explorer";
            } else if (agent.contains("AppleWebKit")) {
              return "safari";
            } else if (agent.contains('Firefox')) {
              return "firefox";
            } else { return agent }""",
            "lang": "painless"
          }
        }
      }
    },
    "aggregations": {
      "200": {
        "filter": {
          "term": {
            "response": "200"
          }
        }
      },
      "404": {
        "filter": {
          "term": {
            "response": "404"
          }
        }
      },
      "503": {
        "filter": {
          "term": {
            "response": "503"
          }
        }
      }
    }
  }
}

Error:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "specified fields can't be null or empty"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "specified fields can't be null or empty"
  },
  "status" : 400
}
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ml-core (:ml/Transform)

hendrikmuhs pushed a commit that referenced this issue Jan 12, 2021
…#67334)

avoid illegal argument exception when fields is empty, which is a valid case

fixes #67333
hendrikmuhs pushed a commit to hendrikmuhs/elasticsearch that referenced this issue Jan 12, 2021
…elastic#67334)

avoid illegal argument exception when fields is empty, which is a valid case

fixes elastic#67333
hendrikmuhs pushed a commit to hendrikmuhs/elasticsearch that referenced this issue Jan 12, 2021
…elastic#67334)

avoid illegal argument exception when fields is empty, which is a valid case

fixes elastic#67333
hendrikmuhs pushed a commit that referenced this issue Jan 13, 2021
…meter… (#67381)

avoid illegal argument exception when fields is empty, which is a valid case

fixes #67333
hendrikmuhs pushed a commit that referenced this issue Jan 13, 2021
…ameter… (#67380)

avoid illegal argument exception when fields is empty, which is a valid case

fixes #67333
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants