Skip to content

docs: add samples from bigquery/cloud-client #148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 109 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
109 commits
Select commit Hold shift + click to select a range
b0a8173
Add veneer samples for BigQuery. [(#444)](https://github.com/GoogleCl…
tswast Aug 10, 2016
8ed04d1
Add data loading samples
Aug 18, 2016
baaa4a5
Add bigquery snippets
Aug 19, 2016
9e60f76
Remove unnecessary fixture
Aug 19, 2016
6f03bef
Address review comments
Aug 19, 2016
dbe847d
Fix lint
Aug 19, 2016
ecbf0f4
Merge pull request #467 from GoogleCloudPlatform/bigquery-cloud-clien…
dpebot Aug 19, 2016
09c139b
Address post-review comments for BigQuery samples
Aug 22, 2016
c4feee5
Merge pull request #471 from GoogleCloudPlatform/fixup-bigquery-samples
dpebot Aug 22, 2016
39e556a
Add bigquery create and copy table examples [(#514)](https://github.c…
Sep 12, 2016
dd47c91
Auto-update dependencies. [(#540)](https://github.com/GoogleCloudPlat…
dpebot Sep 23, 2016
72defef
Auto-update dependencies. [(#542)](https://github.com/GoogleCloudPlat…
dpebot Sep 26, 2016
9f01c6b
Move to google-cloud [(#544)](https://github.com/GoogleCloudPlatform/…
Sep 30, 2016
af0b251
Add new "quickstart" samples [(#547)](https://github.com/GoogleCloudP…
jmdobry Oct 5, 2016
b3b063a
BigQuery - use new QueryJob.results() method. [(#555)](https://github…
tswast Oct 5, 2016
5b2f9bf
Quickstart tests [(#569)](https://github.com/GoogleCloudPlatform/pyth…
Oct 12, 2016
2140994
Generate readmes for most service samples [(#599)](https://github.com…
Oct 24, 2016
2ce6397
Generate most non-appengine readmes
Oct 25, 2016
f4aca02
Add missing bigquery samples [(#622)](https://github.com/GoogleCloudP…
Oct 27, 2016
ab4d7b7
Update samples to support latest Google Cloud Python [(#656)](https:/…
Nov 15, 2016
884d136
Update readmes
Nov 15, 2016
59d5ce0
Fix bigquery load from file sample
Nov 30, 2016
bd8f47c
Fix lint
Nov 30, 2016
3d3475c
Adjusted error handling based on Googler feedback [(#693)](https://gi…
ryanmats Nov 30, 2016
cff9fb7
Changed error handling for all other big query samples as well [(#694…
ryanmats Nov 30, 2016
f45c6bf
Add bigquery shakespeare example [(#604)](https://github.com/GoogleCl…
Dec 6, 2016
961f326
BigQuery parameterized query sample [(#699)](https://github.com/Googl…
tswast Dec 6, 2016
a0837f9
Auto-update dependencies. [(#715)](https://github.com/GoogleCloudPlat…
dpebot Dec 13, 2016
b1b1f59
Remove cloud config fixture [(#887)](https://github.com/GoogleCloudPl…
Apr 4, 2017
51fb449
Remove resource [(#890)](https://github.com/GoogleCloudPlatform/pytho…
Apr 5, 2017
f7c3c4c
BigQuery: named parameter query samples
tswast Dec 19, 2016
34fd8e9
Rename query file and use async method.
tswast Apr 21, 2017
8d11523
Auto-update dependencies. [(#914)](https://github.com/GoogleCloudPlat…
dpebot Apr 24, 2017
47d3068
Indent queries.
tswast Apr 24, 2017
dcadda9
Merge branch 'master' into tswast-bq
tswast Apr 24, 2017
588ac15
Merge pull request #730 from GoogleCloudPlatform/tswast-bq
tswast Apr 24, 2017
9c3c2d8
Auto-update dependencies. [(#916)](https://github.com/GoogleCloudPlat…
dpebot Apr 25, 2017
f595798
Re-generate all readmes
Apr 27, 2017
4d7ce41
BigQuery: user credentials to run a query. [(#925)](https://github.co…
tswast May 3, 2017
5c41eea
Auto-update dependencies. [(#942)](https://github.com/GoogleCloudPlat…
dpebot May 12, 2017
1f0ce24
Fix README rst links [(#962)](https://github.com/GoogleCloudPlatform/…
waprin May 24, 2017
21c0802
Auto-update dependencies. [(#1004)](https://github.com/GoogleCloudPla…
dpebot Jun 27, 2017
484cf15
BigQuery: add auth samples for service accounts.
tswast May 1, 2017
4fa7bd1
Merge pull request #1027 from GoogleCloudPlatform/tswast-bq
tswast Jul 19, 2017
d50ad1d
Fix SQL query to return 10 rows & simplify [(#1041)](https://github.c…
PicardParis Jul 31, 2017
a6e89e1
Fix test randomness [(#1043)](https://github.com/GoogleCloudPlatform/…
PicardParis Aug 1, 2017
944a6dc
Fix test randomness [(#1044)](https://github.com/GoogleCloudPlatform/…
PicardParis Aug 1, 2017
ab214ba
Auto-update dependencies. [(#1055)](https://github.com/GoogleCloudPla…
dpebot Aug 7, 2017
1326a4d
Update readme references. Resolves 1059
Aug 8, 2017
8d19f90
Use futures API to wait for jobs to finish.
tswast Aug 8, 2017
4bfaa98
BQ: Use futures API for quickstart.
tswast Aug 8, 2017
b0e9ee1
BigQuery : max_results changes page size not full list size
tswast Aug 21, 2017
74d0871
Remove max_results argument from table.fetch_data() call.
tswast Aug 22, 2017
d8be82a
Remove unnecessary list() call
tswast Aug 22, 2017
ce3e978
Merge pull request #1086 from GoogleCloudPlatform/tswast-patch-1
tswast Aug 22, 2017
8ca914a
Auto-update dependencies. [(#1093)](https://github.com/GoogleCloudPla…
dpebot Aug 29, 2017
e65d62f
BigQuery : add sample for writing query results to a destination tabl…
tswast Sep 19, 2017
9f92234
Auto-update dependencies. [(#1128)](https://github.com/GoogleCloudPla…
dpebot Sep 20, 2017
aecfab4
BigQuery: Updates samples for BigQuery Beta 2 (do not merge until rel…
alixhami Oct 31, 2017
c009f25
Auto-update dependencies. [(#1186)](https://github.com/GoogleCloudPla…
dpebot Nov 1, 2017
c504546
Auto-update dependencies. [(#1227)](https://github.com/GoogleCloudPla…
dpebot Nov 16, 2017
0a1858e
/s/buckets/datasets in BigQuery auth sample [(#1242)](https://github.…
tswast Dec 1, 2017
8780a15
BigQuery: rewrite simple app tutorial.
tswast Dec 2, 2017
eaafc3c
Fix broken link to core Client service account helper. [(#1256)](http…
tswast Dec 7, 2017
a83001f
Added "Open in Cloud Shell" buttons to README files [(#1254)](https:/…
Dec 7, 2017
a55e274
Move imports into bigquery samples
Dec 11, 2017
bc84421
Merge pull request #1270 from GoogleCloudPlatform/tswast-simple-app
tswast Dec 12, 2017
7d80661
Auto-update dependencies. [(#1272)](https://github.com/GoogleCloudPla…
dpebot Dec 12, 2017
db715f3
Auto-update dependencies. [(#1307)](https://github.com/GoogleCloudPla…
dpebot Jan 9, 2018
69f2aff
Auto-update dependencies. [(#1355)](https://github.com/GoogleCloudPla…
dpebot Feb 9, 2018
b1a430f
Auto-update dependencies. [(#1377)](https://github.com/GoogleCloudPla…
dpebot Mar 5, 2018
5d14340
Regenerate the README files and fix the Open in Cloud Shell link for …
Apr 7, 2018
f5dd3b2
BigQuery: Remove unused samples [(#1455)](https://github.com/GoogleCl…
alixhami Apr 17, 2018
1bc5a16
BigQuery: Removes unused API samples [(#1459)](https://github.com/Goo…
alixhami Apr 18, 2018
d60ebc2
Update READMEs to fix numbering and add git clone [(#1464)](https://g…
frankyn Apr 26, 2018
02b0a53
BigQuery: Moves BigQuery tutorial for Dataproc to python-docs-samples…
alixhami May 18, 2018
bdd51ba
BigQuery: Adds authorized view tutorial [(#1535)](https://github.com/…
alixhami Jun 21, 2018
066abf4
Region tag end typo [(#1536)](https://github.com/GoogleCloudPlatform/…
alixhami Jun 21, 2018
a93f1cc
Code snippets for Jupyter data visualization tutorial [(#1560)](https…
alixhami Jul 12, 2018
adde57a
updates last query per Steve's feedback [(#1578)](https://github.com/…
alixhami Jul 17, 2018
b48dd87
updates tutorial to add datasets to to_delete before running [(#1579)…
alixhami Jul 17, 2018
27babfe
Remove ipython Python 2 modifier from requirements.txt [(#1675)](http…
tswast Aug 31, 2018
c8de756
Auto-update dependencies. [(#1846)](https://github.com/GoogleCloudPla…
dpebot Nov 20, 2018
d22c0e7
Refactor BQ user credentials sample to use region tags [(#1952)](http…
tswast Jan 2, 2019
2105e25
Make TODO clearer what action to take. [(#1963)](https://github.com/G…
tswast Jan 8, 2019
25335f9
Add missing explanation about local_server callback URL
tswast Jan 9, 2019
d4f004d
Auto-update dependencies. [(#1980)](https://github.com/GoogleCloudPla…
dpebot Feb 6, 2019
1998fe4
BigQuery: Add sample for explicitly creating client from service acco…
tswast Aug 5, 2019
6ecfa78
Adds updates including compute [(#2436)](https://github.com/GoogleClo…
gguuss Oct 23, 2019
b695f03
Simplify noxfile setup. [(#2806)](https://github.com/GoogleCloudPlatf…
kurtisvg Apr 2, 2020
fb9da7d
fix: address issues with concurrent BigQuery tests [(#3426)](https://…
shollyman Apr 17, 2020
797349b
chore(deps): update dependency pytz to v2019.3 [(#3190)](https://gith…
renovate-bot Apr 18, 2020
544f861
chore(deps): update dependency matplotlib to v3.2.1 [(#3180)](https:/…
renovate-bot Apr 18, 2020
3e4c835
chore(deps): update dependency ipython to v7.13.0 [(#3177)](https://g…
renovate-bot Apr 18, 2020
9deac72
chore(deps): update dependency google-cloud-bigquery to v1.24.0 [(#30…
renovate-bot Apr 18, 2020
93f3841
chore(deps): update dependency google-auth-oauthlib to v0.4.1 [(#2797…
renovate-bot Apr 18, 2020
700ffb9
chore(deps): update dependency pytz to v2020 [(#3566)](https://github…
renovate-bot Apr 28, 2020
69af221
chore(deps): update dependency ipython to v7.14.0 [(#3670)](https://g…
renovate-bot May 4, 2020
c3c4a0b
[bigquery] opt in to use the build specific projects [(#3883)](https:…
May 26, 2020
4d06dfc
chore(deps): update dependency ipython to v7.15.0 [(#3917)](https://g…
renovate-bot May 30, 2020
863cb32
Replace GCLOUD_PROJECT with GOOGLE_CLOUD_PROJECT. [(#4022)](https://g…
kurtisvg Jun 9, 2020
95091a4
Update dependency google-cloud-bigquery to v1.25.0 [(#4024)](https://…
renovate-bot Jun 11, 2020
1da7353
chore(deps): update dependency matplotlib to v3.2.2 [(#4118)](https:/…
renovate-bot Jun 19, 2020
fdd4121
Merge branch 'snippets-samples' into add-snippets-samples
plamut Jun 25, 2020
353c993
chore: update templates
plamut Jun 26, 2020
6e38740
Merge branch 'master' into add-snippets-samples
plamut Jul 21, 2020
d6b5579
chore: fix samples tests dependencies
plamut Jul 21, 2020
62dc5e8
chore: exclude orphan samples doc files from TOC
plamut Jul 21, 2020
ac1ab92
Change the project env variable to use in tests
plamut Jul 21, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Code owners file.
# This file controls who is tagged for review for any given pull request.
#
# For syntax help see:
# https://help.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners#codeowners-syntax


/samples/ @shollyman @googleapis/python-samples-owners
7 changes: 6 additions & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,12 @@

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ["_build"]
exclude_patterns = [
"_build",
"samples/AUTHORING_GUIDE.md",
"samples/CONTRIBUTING.md",
"samples/snippets/README.rst",
]

# The reST default role (used for this markup: `text`) to use for all
# documents.
Expand Down
1 change: 1 addition & 0 deletions samples/AUTHORING_GUIDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
See https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md
1 change: 1 addition & 0 deletions samples/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
See https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/CONTRIBUTING.md
2 changes: 2 additions & 0 deletions samples/snippets/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
client_secrets.json
service_account.json
166 changes: 166 additions & 0 deletions samples/snippets/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@

.. This file is automatically generated. Do not edit this file directly.

Google BigQuery Python Samples
===============================================================================

.. image:: https://gstatic.com/cloudssh/images/open-btn.png
:target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=bigquery/cloud-client/README.rst


This directory contains samples for Google BigQuery. `Google BigQuery`_ is Google's fully managed, petabyte scale, low cost analytics data warehouse. BigQuery is NoOps—there is no infrastructure to manage and you don't need a database administrator—so you can focus on analyzing data to find meaningful insights, use familiar SQL, and take advantage of our pay-as-you-go model.




.. _Google BigQuery: https://cloud.google.com/bigquery/docs


Setup
-------------------------------------------------------------------------------



Authentication
++++++++++++++

This sample requires you to have authentication setup. Refer to the
`Authentication Getting Started Guide`_ for instructions on setting up
credentials for applications.

.. _Authentication Getting Started Guide:
https://cloud.google.com/docs/authentication/getting-started




Install Dependencies
++++++++++++++++++++

#. Clone python-docs-samples and change directory to the sample directory you want to use.

.. code-block:: bash

$ git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git

#. Install `pip`_ and `virtualenv`_ if you do not already have them. You may want to refer to the `Python Development Environment Setup Guide`_ for Google Cloud Platform for instructions.

.. _Python Development Environment Setup Guide:
https://cloud.google.com/python/setup

#. Create a virtualenv. Samples are compatible with Python 3.6+.

.. code-block:: bash

$ virtualenv env
$ source env/bin/activate

#. Install the dependencies needed to run the samples.

.. code-block:: bash

$ pip install -r requirements.txt

.. _pip: https://pip.pypa.io/
.. _virtualenv: https://virtualenv.pypa.io/






Samples
-------------------------------------------------------------------------------


Quickstart
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

.. image:: https://gstatic.com/cloudssh/images/open-btn.png
:target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=bigquery/cloud-client/quickstart.py,bigquery/cloud-client/README.rst




To run this sample:

.. code-block:: bash

$ python quickstart.py




Simple Application
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

.. image:: https://gstatic.com/cloudssh/images/open-btn.png
:target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=bigquery/cloud-client/simple_app.py,bigquery/cloud-client/README.rst




To run this sample:

.. code-block:: bash

$ python simple_app.py




User Credentials
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

.. image:: https://gstatic.com/cloudssh/images/open-btn.png
:target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=bigquery/cloud-client/user_credentials.py,bigquery/cloud-client/README.rst




To run this sample:

.. code-block:: bash

$ python user_credentials.py


usage: user_credentials.py [-h] [--launch-browser] project

Command-line application to run a query using user credentials.

You must supply a client secrets file, which would normally be bundled with
your application.

positional arguments:
project Project to use for BigQuery billing.

optional arguments:
-h, --help show this help message and exit
--launch-browser Use a local server flow to authenticate.









The client library
-------------------------------------------------------------------------------

This sample uses the `Google Cloud Client Library for Python`_.
You can read the documentation for more details on API usage and use GitHub
to `browse the source`_ and `report issues`_.

.. _Google Cloud Client Library for Python:
https://googlecloudplatform.github.io/google-cloud-python/
.. _browse the source:
https://github.com/GoogleCloudPlatform/google-cloud-python
.. _report issues:
https://github.com/GoogleCloudPlatform/google-cloud-python/issues



.. _Google Cloud SDK: https://cloud.google.com/sdk/
31 changes: 31 additions & 0 deletions samples/snippets/README.rst.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# This file is used to generate README.rst

product:
name: Google BigQuery
short_name: BigQuery
url: https://cloud.google.com/bigquery/docs
description: >
`Google BigQuery`_ is Google's fully managed, petabyte scale, low cost
analytics data warehouse. BigQuery is NoOps—there is no infrastructure to
manage and you don't need a database administrator—so you can focus on
analyzing data to find meaningful insights, use familiar SQL, and take
advantage of our pay-as-you-go model.

required_role: BigQuery Admin

setup:
- auth
- install_deps

samples:
- name: Quickstart
file: quickstart.py
- name: Simple Application
file: simple_app.py
- name: User Credentials
file: user_credentials.py
show_help: true

cloud_client_library: true

folder: bigquery/cloud-client
39 changes: 39 additions & 0 deletions samples/snippets/authenticate_service_account.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import os


def main():
key_path = os.environ.get("GOOGLE_APPLICATION_CREDENTIALS")

# [START bigquery_client_json_credentials]
from google.cloud import bigquery
from google.oauth2 import service_account

# TODO(developer): Set key_path to the path to the service account key
# file.
# key_path = "path/to/service_account.json"

credentials = service_account.Credentials.from_service_account_file(
key_path, scopes=["https://www.googleapis.com/auth/cloud-platform"],
)

client = bigquery.Client(credentials=credentials, project=credentials.project_id,)
# [END bigquery_client_json_credentials]
return client


if __name__ == "__main__":
main()
33 changes: 33 additions & 0 deletions samples/snippets/authenticate_service_account_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import google.auth

import authenticate_service_account


def mock_credentials(*args, **kwargs):
credentials, _ = google.auth.default(
["https://www.googleapis.com/auth/cloud-platform"]
)
return credentials


def test_main(monkeypatch):
monkeypatch.setattr(
"google.oauth2.service_account.Credentials.from_service_account_file",
mock_credentials,
)
client = authenticate_service_account.main()
assert client is not None
Loading