Skip to content

Commit 722414f

Browse files
Remove accidental duplicated conversion of pyarrow Table to pandas (dask#7983)
1 parent 9beab9a commit 722414f

File tree

1 file changed

+0
-1
lines changed

1 file changed

+0
-1
lines changed

distributed/shuffle/_arrow.py

-1
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,6 @@ def convert_partition(data: bytes, meta: pd.DataFrame) -> pd.DataFrame:
5656
sr = pa.RecordBatchStreamReader(file)
5757
shards.append(sr.read_all())
5858
table = pa.concat_tables(shards, promote=True)
59-
df = table.to_pandas(self_destruct=True)
6059

6160
def default_types_mapper(pyarrow_dtype: pa.DataType) -> object:
6261
# Avoid converting strings from `string[pyarrow]` to `string[python]`

0 commit comments

Comments
 (0)