Support semantic_text in ES|QL

[ChrisHegarty]

This meta issue tracks adding support for semantic_text to ES|QL.

• ES|QL Add initial support for semantic_text field type #113920
• Add mixed mode BWC testing support for test-specific plugins #115166
• semantic_text as string type in ES|QL - support for functions and operators #115243
• Make search functions translation aware #118355
• Add support for semantic_text in the match function Semantic search with query builder rewrite #118676
• Enhance support for semantic_text such that it has the same support as keyword and text which are simply seen as string fields:
  

  elasticsearch/x-pack/plugin/esql-core/src/main/java/org/elasticsearch/xpack/esql/core/type/DataType.java

  Lines 356 to 358 in dcbbbab

  	public static boolean isString(DataType t) {
  	return t == KEYWORD || t == TEXT;
  	}

  
  With this change we will be able to use semantic_text fields in the same way we use keyword and text and be able to pass semantic_text field to any function/operator that can accept string fields.
  • support for functions/operators semantic_text as string type in ES|QL - support for functions and operators #115243
  • support for all commands Add support for aggregations, GROK and DISSECT for semantic_text #117337
• Enable query rewrites on the coordinator for ESQL #119667
• Make semantic text part of the text family #119792
• ESQL: Document support for semantic_text field mapping #120052
• Full support in Kibana (it should just work since field caps will return text for semantic_text fields?)

Follow-ups:

• enable integration tests for multi clusters and mixed nodes (depends on Add mixed mode BWC testing support for test-specific plugins #115166)
• add semantic_text to mapping-all-types.json and test semantic-text fields within LocalPhysicalPlanOptimizerTests.java
  Semantic search with query builder rewrite #118676 (comment)
  fixed with Add semantic_text to mapping_all_types and switch to TranslationAware in PushFiltersToSource #118982
• address difference in score between multi vs single node Semantic search with query builder rewrite #118676 (comment)
• Remove SEMANTIC_TEXT data type in ES|QL #125581

Refactor work (might be done separate from the semantic_text support project but tracking here):

• Make search functions translation aware #118355 (comment)
• look into whether we can do async mapping/physical planning using a CoordinatorMappingPreProcessor (from Esql expression lifecycle events proposal) Bogdan made this happen 🎉 ESQL: introduce a pre-mapping logical plan processing step #121260

[elasticsearchmachine]

Pinging @elastic/es-search (Team:Search)

[elasticsearchmachine]

Pinging @elastic/kibana-esql (ES|QL-ui)

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[ioanatia]

Since field caps will report semantic_text fields as text, the initial plan on how we want to support semantic_text in ES|QL changes.
ES|QL will handle semantic_text fields as text fields.
We already support semantic_text as part of the match query and match function (through the query builder rewrite (#118676).

After the field caps change is merged, we might need to go back and clean up some of the changes that specifically accounted for a semantic_text field type in ES|QL. These changes are under snapshot (under the the SEMANTIC_TEXT_FIELD_TYPE capability), so there are no bwc concerns.

[ioanatia]

closing this issue - we do have an outstanding issue with #115166 that does not allow us to test semantic search in ES|QL with mixed version clusters.
this is a larger issue that's not specific just to semantic search in ES|QL.

we do test for single-node/multi-node clusters and the rest of the follow ups have been addressed so I think we can just close this one.