Skip to content

Only unnest source for EmptyRelation #15159

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 24, 2025

Conversation

blaginin
Copy link
Contributor

@blaginin blaginin commented Mar 11, 2025

Which issue does this PR close?

Rationale for this change

We currently override source losing some data, see issue for examples

What changes are included in this PR?

Are these changes tested?

Added tests

Are there any user-facing changes?

No

Comment on lines 382 to 384
if matches!(
projection.input.as_ref(),
LogicalPlan::EmptyRelation(_)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to add some comments to explain why we match EmptyRelation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, added in 28f5301

Copy link
Contributor Author

@blaginin blaginin Mar 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to remove this new test because now (even in main) it produces SELECT * FROM (SELECT [1, 2, 3] AS c) CROSS JOIN UNNEST(c). Feels like this should be discussed separately

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this comment -- doesn't this PR add a new test?

Copy link
Contributor Author

@blaginin blaginin Mar 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry -- before I added one more test, but then I have to remove it due to ef4a79d - I think we can discuss in that issue and work on it further

@blaginin blaginin marked this pull request as ready for review March 17, 2025 17:58
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this comment -- doesn't this PR add a new test?

@@ -377,8 +377,23 @@ impl Unparser<'_> {
};
if self.dialect.unnest_as_table_factor() && unnest_input_type.is_some() {
if let LogicalPlan::Unnest(unnest) = &p.input.as_ref() {
return self
.unnest_to_table_factor_sql(unnest, query, select, relation);
if let LogicalPlan::Projection(projection) = unnest.input.as_ref()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like something that is a limitation of unnest_to_table_factor_sql -- would it make sense to put the check in there so the check and the code with the assumption are in the same place?

@alamb alamb marked this pull request as draft March 19, 2025 20:01
@alamb
Copy link
Contributor

alamb commented Mar 19, 2025

Marking as draft as I think this PR is no longer waiting on feedback. Please mark it as ready for review when it is ready for another look

@blaginin blaginin marked this pull request as ready for review March 21, 2025 13:57
@blaginin blaginin requested review from goldmedal and alamb March 21, 2025 13:59
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me -- thanks @blaginin

Comment on lines +636 to +647
TestStatementWithDialect {
sql: "SELECT unnest([1, 2, 3, 4]) from unnest([1, 2, 3]);",
expected: r#"SELECT UNNEST([1, 2, 3, 4]) AS UNNEST(make_array(Int64(1),Int64(2),Int64(3),Int64(4))) FROM UNNEST([1, 2, 3])"#,
parser_dialect: Box::new(GenericDialect {}),
unparser_dialect: Box::new(CustomDialectBuilder::default().with_unnest_as_table_factor(true).build()),
},
TestStatementWithDialect {
sql: "SELECT unnest([1, 2, 3, 4]) from unnest([1, 2, 3]);",
expected: r#"SELECT UNNEST([1, 2, 3, 4]) AS UNNEST(make_array(Int64(1),Int64(2),Int64(3),Int64(4))) FROM UNNEST([1, 2, 3])"#,
parser_dialect: Box::new(GenericDialect {}),
unparser_dialect: Box::new(CustomDialectBuilder::default().with_unnest_as_table_factor(true).build()),
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They look like the same. 🤔 I think we can remove one of them.

@goldmedal goldmedal merged commit d68fca9 into apache:main Mar 24, 2025
27 checks passed
@goldmedal
Copy link
Contributor

Thanks @blaginin and @alamb for reviewing. I'll create a follow-up PR to remove the duplicate test.

qstommyshu pushed a commit to qstommyshu/datafusion that referenced this pull request Mar 27, 2025
* Only unnest source for `EmptyRelation`

* Add a note on new condition

* Remove new test

* Put all unnest assumptions into one function
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sql SQL Planner
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Different unnests on plan_to_sql are merged
3 participants