Skip to content

ENH: Convert read_gbq() function to use google-cloud-python #25

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 43 commits into from
Dec 20, 2017
Merged

ENH: Convert read_gbq() function to use google-cloud-python #25

Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
83b66c2
Add new from_gbq() method and tweak requirements
Apr 8, 2017
438ff59
Remove unecessary type conversion
Apr 8, 2017
03a9548
Remove old read_gbq, rename from_gbq to read_gbq, and add verbose que…
Apr 9, 2017
ae7c0e9
Handle new iterator return type from fetch_data
jasonqng Oct 3, 2017
d2d01bc
Remove old read_gbq
jasonqng Oct 4, 2017
97410dd
Change requirement to google-cloud-bigquery and pin to 0.25.0
jasonqng Oct 4, 2017
a0847f8
Pass any private key to client for auth
jasonqng Oct 4, 2017
12bacd8
Pin google-cloud-bigquery to 0.25.0 in setup.py
jasonqng Oct 5, 2017
f50bcca
Catch renaming of results to result in 0.26.0
jasonqng Oct 5, 2017
cd17b8d
Pin requirements to include 0.26.0
jasonqng Oct 5, 2017
48ac795
Remove legacy read_gbq code and add initial user auth flow
jasonqng Oct 5, 2017
c4a2c36
Use GbqConnector for credentials checking
jasonqng Oct 5, 2017
04373eb
Dealing with tests part 1
jasonqng Oct 5, 2017
dcf014a
Fix formatting to make linter happy plus remove obj conversion in test
jasonqng Oct 6, 2017
72253aa
Rewrite query_parameters test with documentation
jasonqng Oct 6, 2017
751f39b
Rewrite sync and async job flow, handle timeouts, add BadRequest erro…
jasonqng Oct 6, 2017
d83fd22
Remove two configuration tests which are no longer relevant
jasonqng Oct 6, 2017
e597a76
Rewrite tests part 2 and can specify return type
jasonqng Oct 6, 2017
347bca2
Pin bigquery to 0.26.0 and more linting
jasonqng Oct 6, 2017
fc6134a
Remove uuid from requirements
jasonqng Oct 7, 2017
61db191
Clean up/revert some of the documentation
jasonqng Oct 9, 2017
28c0ae7
Unpin in testing requirements and expand config error message
jasonqng Oct 27, 2017
cd75cda
Include 0.27 in setup.py install_requires
jasonqng Oct 27, 2017
6264177
Initial run_query function for modularizing read_gbq
jasonqng Oct 27, 2017
208e39f
Move schema handling and creating the df to separate functions, updat…
jasonqng Oct 27, 2017
2cd32c7
Linting
jasonqng Oct 27, 2017
4468197
Update schema test
jasonqng Oct 27, 2017
6a6fc2d
Fix read_gbq docstring for Sphinx
jasonqng Oct 28, 2017
9ab0de3
Remove timeout_ms paramater from read_gbq, it should be set in config…
jasonqng Oct 30, 2017
e203ba4
Add more documentation
jasonqng Oct 30, 2017
92f3943
Moved read_query to GbqConnector, update credentials and client gener…
jasonqng Nov 15, 2017
3cc153a
Move sizeof_fmt back into GbqConnector
jasonqng Nov 15, 2017
91f900e
Fix sizeof_fmt test and move create_df to top-level function
jasonqng Nov 15, 2017
4d967e6
Add import error message
jasonqng Nov 15, 2017
476d72b
Revert test_query_response_bytes test to original
jasonqng Nov 16, 2017
6da0ef2
Convert rest of methods to use google-cloud-bigquery
tswast Nov 23, 2017
cd551bb
Indentation
jasonqng Nov 27, 2017
a6865e0
Ignore mode property when comparing schemas.
tswast Nov 27, 2017
c636783
Document new dependency on google-cloud-bigquery.
tswast Nov 27, 2017
362e9db
Merge branch 'master' into read-gbq-google-cloud-library
tswast Nov 28, 2017
e959571
Document dependencies for previous verions.
tswast Nov 30, 2017
6448abb
Remove print statement used for debugging.
tswast Dec 8, 2017
26d6431
Add deps and StreamingInsertError to changelog.
tswast Dec 20, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions ci/requirements-2.7-0.19.2.pip
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
google-api-python-client
google-auth
google-auth-httplib2
google-auth-oauthlib
PyCrypto
python-gflags
mock
google-cloud-bigquery
3 changes: 1 addition & 2 deletions ci/requirements-3.5-0.18.1.pip
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
google-api-python-client==1.6.0
google-auth==1.0.0
google-auth-httplib2==0.0.1
google-auth-oauthlib==0.0.1
mock
google-cloud-bigquery==0.28.0
3 changes: 1 addition & 2 deletions ci/requirements-3.6-0.20.1.conda
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
google-api-python-client
google-auth
google-auth-httplib2
google-auth-oauthlib
mock
google-cloud-bigquery
3 changes: 1 addition & 2 deletions ci/requirements-3.6-MASTER.pip
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
google-api-python-client
google-auth
google-auth-httplib2
google-auth-oauthlib
mock
google-cloud-bigquery
5 changes: 5 additions & 0 deletions docs/source/changelog.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
Changelog
=========

0.3.0 / 2017-??-??
------------------

- Use the `google-cloud-bigquery <https://googlecloudplatform.github.io/google-cloud-python/latest/bigquery/usage.html>`__ library for API calls instead of ``google-api-client`` and ``httplib2``. (:issue:`93`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs a more comprehensive note here. show what you used to import / depend on, and what it should be now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can link to install.rst as well


0.2.1 / 2017-11-27
------------------

Expand Down
15 changes: 12 additions & 3 deletions docs/source/install.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,17 @@ Dependencies

This module requires following additional dependencies:

- `httplib2 <https://github.com/httplib2/httplib2>`__: HTTP client
- `google-api-python-client <http://github.com/google/google-api-python-client>`__: Google's API client
- `google-auth <https://github.com/GoogleCloudPlatform/google-auth-library-python>`__: authentication and authorization for Google's API
- `google-auth-oauthlib <https://github.com/GoogleCloudPlatform/google-auth-library-python-oauthlib>`__: integration with `oauthlib <https://github.com/idan/oauthlib>`__ for end-user authentication
- `google-auth-httplib2 <https://github.com/GoogleCloudPlatform/google-auth-library-python-httplib2>`__: adapter to use ``httplib2`` HTTP client with ``google-auth``
- `google-cloud-bigquery <http://github.com/GoogleCloudPlatform/google-cloud-python>`__: Google Cloud client library for BigQuery

.. note::

The dependency on `google-cloud-bigquery <http://github.com/GoogleCloudPlatform/google-cloud-python>`__ is new in version 0.3.0 of ``pandas-gbq``.
Versions less than 0.3.0 required the following dependencies:

- `httplib2 <https://github.com/httplib2/httplib2>`__: HTTP client (no longer required)
- `google-api-python-client <http://github.com/google/google-api-python-client>`__: Google's API client (no longer required, replaced by `google-cloud-bigquery <http://github.com/GoogleCloudPlatform/google-cloud-python>`__:)
- `google-auth <https://github.com/GoogleCloudPlatform/google-auth-library-python>`__: authentication and authorization for Google's API
- `google-auth-oauthlib <https://github.com/GoogleCloudPlatform/google-auth-library-python-oauthlib>`__: integration with `oauthlib <https://github.com/idan/oauthlib>`__ for end-user authentication
- `google-auth-httplib2 <https://github.com/GoogleCloudPlatform/google-auth-library-python-httplib2>`__: adapter to use ``httplib2`` HTTP client with ``google-auth`` (no longer required)
Loading