Skip to content

BUG: resolve divide by 0 error when uploading empty dataframe #252

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Feb 26, 2019
Merged

BUG: resolve divide by 0 error when uploading empty dataframe #252

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
42ccab2
resolve divide by 0 error when uploading empty dataframe
bwanglzu Feb 22, 2019
3864d46
reformat with black
bwanglzu Feb 22, 2019
272aa7b
add unit test when uploading empty dataframe
bwanglzu Feb 22, 2019
8bfde50
add empty data upload system test
Feb 23, 2019
59a7950
remove empty df unit test
Feb 23, 2019
87ae125
update empty df
Feb 23, 2019
02c897a
add 0.10.0 release note
bwanglzu Feb 25, 2019
a9ec115
Merge branch 'bug/divide-by-zero' of https://github.com/bwanglzu/pand…
bwanglzu Feb 25, 2019
922707c
Merge branch 'master' into bug/divide-by-zero
bwanglzu Feb 25, 2019
1fe7c13
update release note version number to 0.11.0
bwanglzu Feb 25, 2019
d62c36e
update empty dataframe bug fix in change log
bwanglzu Feb 26, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions pandas_gbq/gbq.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -519,8 +519,8 @@ def load_data(
chunks = tqdm.tqdm(chunks)
for remaining_rows in chunks:
logger.info(
"\rLoad is {0}% Complete".format(
((total_rows - remaining_rows) * 100) / total_rows
"\r{} out of {} rows loaded.".format(
total_rows - remaining_rows, total_rows
)
)
except self.http_error as ex:
Expand Down
22 changes: 22 additions & 0 deletions tests/unit/test_gbq.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,28 @@ def test_to_gbq_doesnt_run_query(
mock_bigquery_client.query.assert_not_called()


def test_to_gbq_uploading_empty_dataframe(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very reasonable to copy code also testing writes, but this is doing something slightly more involved re pandas versioning

Could you copy the code here: https://github.com/pydata/pandas-gbq/blob/272aa7bebef5dc99869e7c44adf8011258c8d7c9/tests/system/test_gbq.py#L879 and put the test there?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @max-sixty thanks for your feedback. Two questions:

  1. Shall I create a test method in pandas-gbq/tests/system/test_gbq.py named test_upload_empty_data, or just create an empty pd.DataFrame within test_upload_data? Which one is better?
  2. Do I need to delete test_to_gbq_uploading_empty_dataframe in pandas-gbq/tests/unit/test_gbq.py?

Thanks!

Copy link
Contributor

@max-sixty max-sixty Feb 22, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall I create a test method in pandas-gbq/tests/system/test_gbq.py named test_upload_empty_data

This would be perfect!
That's the only test that's needed (no need to keep the existing test_to_gbq_uploading_empty_dataframe)

recwarn, min_bq_version, monkeypatch
):
import pkg_resources

pandas_version = pkg_resources.parse_version("0.23.0")
with pytest.warns(FutureWarning), mock.patch(
"pkg_resources.Distribution.parsed_version",
new_callable=mock.PropertyMock,
) as mock_version:
mock_version.side_effect = [min_bq_version, pandas_version]
try:
gbq.to_gbq(
DataFrame(),
"dataset.tablename",
project_id="my-project",
verbose=True,
)
except gbq.TableCreationError:
pass


def test_read_gbq_with_no_project_id_given_should_fail(monkeypatch):
import pydata_google_auth

Expand Down