Skip to content

Glue Data catalog csv files with header parameter cause error GENERIC_INTERNAL_ERROR: Failed to construct table metadata for table xxxx #2967

Closed
@ctalucci

Description

@ctalucci

When using wr.s3.to_csv with glue catalog and the header parameter the corresponding table schema parameter skip.header.line.count is getting set to True which causes an GENERIC_INTERNAL_ERROR: Failed to construct table metadata for table xxxx on querying with Athena. Manually updating the value to 1 fixes the error

https://github.com/aws/aws-sdk-pandas/blob/8cef7504c4b4aadd44306cde49001fcd062c5bb7/awswrangler/s3/_write_text.py#L585C1-L585C68

"skip_header_line_count": True if header else None,
should be:
"skip_header_line_count": 1 if header else None,

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions