Skip to content

Commit fc3284d

Browse files
frankynengelke
authored andcommitted
Non-client library example of constructing a Signed URL (#1837)
* Humble beginnings * Update * Update * Update * Add test and README for generate_signed_urls.py * Update error message * Fix issues in CLI * Fix region tag and address feedback. * Add spacing * Python 2 and 3 compatible string to hex used * Fixed too long line
1 parent 6928491 commit fc3284d

File tree

5 files changed

+331
-0
lines changed

5 files changed

+331
-0
lines changed

storage/signed_urls/README.rst

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
.. This file is automatically generated. Do not edit this file directly.
2+
3+
Google Cloud Storage Python Samples
4+
===============================================================================
5+
6+
.. image:: https://gstatic.com/cloudssh/images/open-btn.png
7+
:target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=storage/signed_urls/README.rst
8+
9+
10+
This directory contains samples for Google Cloud Storage. `Google Cloud Storage`_ allows world-wide storage and retrieval of any amount of data at any time.
11+
12+
13+
14+
15+
.. _Google Cloud Storage: https://cloud.google.com/storage/docs
16+
17+
Setup
18+
-------------------------------------------------------------------------------
19+
20+
21+
Authentication
22+
++++++++++++++
23+
24+
This sample requires you to have authentication setup. Refer to the
25+
`Authentication Getting Started Guide`_ for instructions on setting up
26+
credentials for applications.
27+
28+
.. _Authentication Getting Started Guide:
29+
https://cloud.google.com/docs/authentication/getting-started
30+
31+
Install Dependencies
32+
++++++++++++++++++++
33+
34+
#. Clone python-docs-samples and change directory to the sample directory you want to use.
35+
36+
.. code-block:: bash
37+
38+
$ git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git
39+
40+
#. Install `pip`_ and `virtualenv`_ if you do not already have them. You may want to refer to the `Python Development Environment Setup Guide`_ for Google Cloud Platform for instructions.
41+
42+
.. _Python Development Environment Setup Guide:
43+
https://cloud.google.com/python/setup
44+
45+
#. Create a virtualenv. Samples are compatible with Python 2.7 and 3.4+.
46+
47+
.. code-block:: bash
48+
49+
$ virtualenv env
50+
$ source env/bin/activate
51+
52+
#. Install the dependencies needed to run the samples.
53+
54+
.. code-block:: bash
55+
56+
$ pip install -r requirements.txt
57+
58+
.. _pip: https://pip.pypa.io/
59+
.. _virtualenv: https://virtualenv.pypa.io/
60+
61+
Samples
62+
-------------------------------------------------------------------------------
63+
64+
Generate Signed URLs in Python
65+
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
66+
67+
.. image:: https://gstatic.com/cloudssh/images/open-btn.png
68+
:target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=storage/signed_urls/generate_signed_urls.py,storage/signed_urls/README.rst
69+
70+
71+
72+
73+
To run this sample:
74+
75+
.. code-block:: bash
76+
77+
$ python generate_signed_urls.py
78+
79+
usage: generate_signed_urls.py [-h]
80+
service_account_file request_method bucket_name
81+
object_name expiration
82+
83+
positional arguments:
84+
service_account_file Path to your Google service account.
85+
request_method A request method, e.g GET, POST.
86+
bucket_name Your Cloud Storage bucket name.
87+
object_name Your Cloud Storage object name.
88+
expiration Expiration Time.
89+
90+
optional arguments:
91+
-h, --help show this help message and exit
92+
93+
94+
95+
96+
97+
.. _Google Cloud SDK: https://cloud.google.com/sdk/

storage/signed_urls/README.rst.in

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# This file is used to generate README.rst
2+
3+
product:
4+
name: Google Cloud Storage
5+
short_name: Cloud Storage
6+
url: https://cloud.google.com/storage/docs
7+
description: >
8+
`Google Cloud Storage`_ allows world-wide storage and retrieval of any
9+
amount of data at any time.
10+
11+
setup:
12+
- auth
13+
- install_deps
14+
15+
samples:
16+
- name: Generate Signed URLs in Python
17+
file: generate_signed_urls.py
18+
show_help: true
19+
20+
cloud_client_library: false
21+
22+
folder: storage/signed_urls
Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
# Copyright 2018 Google, Inc.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
import argparse
16+
17+
"""This application demonstrates how to construct a Signed URL for objects in
18+
Google Cloud Storage.
19+
20+
For more information, see the README.md under /storage and the documentation
21+
at https://cloud.google.com/storage/docs/access-control/signing-urls-manually.
22+
"""
23+
24+
# [START storage_signed_url_all]
25+
# [START storage_signed_url_dependencies]
26+
import binascii
27+
import collections
28+
import datetime
29+
import hashlib
30+
import sys
31+
32+
# pip install six
33+
from six.moves.urllib.parse import quote
34+
35+
# [START storage_signed_url_signer]
36+
# pip install google-auth
37+
from google.oauth2 import service_account
38+
39+
# [END storage_signed_url_signer]
40+
# [END storage_signed_url_dependencies]
41+
42+
43+
def generate_signed_url(service_account_file, bucket_name, object_name,
44+
expiration, http_method='GET', query_parameters=None,
45+
headers=None):
46+
47+
if expiration > 604800:
48+
print('Expiration Time can\'t be longer than 604800 seconds (7 days).')
49+
sys.exit(1)
50+
51+
# [START storage_signed_url_canonical_uri]
52+
escaped_object_name = quote(object_name, safe='')
53+
canonical_uri = '/{}/{}'.format(bucket_name, escaped_object_name)
54+
# [END storage_signed_url_canonical_uri]
55+
56+
# [START storage_signed_url_canonical_datetime]
57+
datetime_now = datetime.datetime.utcnow()
58+
request_timestamp = datetime_now.strftime('%Y%m%dT%H%M%SZ')
59+
datestamp = datetime_now.strftime('%Y%m%d')
60+
# [END storage_signed_url_canonical_datetime]
61+
62+
# [START storage_signed_url_credentials]
63+
# [START storage_signed_url_signer]
64+
google_credentials = service_account.Credentials.from_service_account_file(
65+
service_account_file)
66+
# [END storage_signed_url_signer]
67+
client_email = google_credentials.service_account_email
68+
credential_scope = '{}/auto/gcs/goog4_request'.format(datestamp)
69+
credential = '{}/{}'.format(client_email, credential_scope)
70+
# [END storage_signed_url_credentials]
71+
72+
if headers is None:
73+
headers = dict()
74+
# [START storage_signed_url_canonical_headers]
75+
headers['host'] = 'storage.googleapis.com'
76+
77+
canonical_headers = ''
78+
ordered_headers = collections.OrderedDict(sorted(headers.items()))
79+
for k, v in ordered_headers.items():
80+
lower_k = str(k).lower()
81+
strip_v = str(v).lower()
82+
canonical_headers += '{}:{}\n'.format(lower_k, strip_v)
83+
# [END storage_signed_url_canonical_headers]
84+
85+
# [START storage_signed_url_signed_headers]
86+
signed_headers = ''
87+
for k, _ in ordered_headers.items():
88+
lower_k = str(k).lower()
89+
signed_headers += '{};'.format(lower_k)
90+
signed_headers = signed_headers[:-1] # remove trailing ';'
91+
# [END storage_signed_url_signed_headers]
92+
93+
if query_parameters is None:
94+
query_parameters = dict()
95+
# [START storage_signed_url_canonical_query_parameters]
96+
query_parameters['X-Goog-Algorithm'] = 'GOOG4-RSA-SHA256'
97+
query_parameters['X-Goog-Credential'] = credential
98+
query_parameters['X-Goog-Date'] = request_timestamp
99+
query_parameters['X-Goog-Expires'] = expiration
100+
query_parameters['X-Goog-SignedHeaders'] = signed_headers
101+
102+
canonical_query_string = ''
103+
ordered_query_parameters = collections.OrderedDict(
104+
sorted(query_parameters.items()))
105+
for k, v in ordered_query_parameters.items():
106+
encoded_k = quote(str(k), safe='')
107+
encoded_v = quote(str(v), safe='')
108+
canonical_query_string += '{}={}&'.format(encoded_k, encoded_v)
109+
canonical_query_string = canonical_query_string[:-1] # remove trailing ';'
110+
# [END storage_signed_url_canonical_query_parameters]
111+
112+
# [START storage_signed_url_canonical_request]
113+
canonical_request = '\n'.join([http_method,
114+
canonical_uri,
115+
canonical_query_string,
116+
canonical_headers,
117+
signed_headers,
118+
'UNSIGNED-PAYLOAD'])
119+
# [END storage_signed_url_canonical_request]
120+
121+
# [START storage_signed_url_hash]
122+
canonical_request_hash = hashlib.sha256(
123+
canonical_request.encode()).hexdigest()
124+
# [END storage_signed_url_hash]
125+
126+
# [START storage_signed_url_string_to_sign]
127+
string_to_sign = '\n'.join(['GOOG4-RSA-SHA256',
128+
request_timestamp,
129+
credential_scope,
130+
canonical_request_hash])
131+
# [END storage_signed_url_string_to_sign]
132+
133+
# [START storage_signed_url_signer]
134+
signature = binascii.hexlify(
135+
google_credentials.signer.sign(string_to_sign)
136+
).decode()
137+
# [END storage_signed_url_signer]
138+
139+
# [START storage_signed_url_construction]
140+
host_name = 'https://storage.googleapis.com'
141+
signed_url = '{}{}?{}&x-goog-signature={}'.format(host_name, canonical_uri,
142+
canonical_query_string,
143+
signature)
144+
# [END storage_signed_url_construction]
145+
return signed_url
146+
# [END storage_signed_url_all]
147+
148+
149+
if __name__ == '__main__':
150+
parser = argparse.ArgumentParser(
151+
description=__doc__,
152+
formatter_class=argparse.RawDescriptionHelpFormatter)
153+
parser.add_argument('service_account_file',
154+
help='Path to your Google service account.')
155+
parser.add_argument(
156+
'request_method', help='A request method, e.g GET, POST.')
157+
parser.add_argument('bucket_name', help='Your Cloud Storage bucket name.')
158+
parser.add_argument('object_name', help='Your Cloud Storage object name.')
159+
parser.add_argument('expiration', help='Expiration Time.')
160+
161+
args = parser.parse_args()
162+
signed_url = generate_signed_url(
163+
service_account_file=args.service_account_file,
164+
http_method=args.request_method, bucket_name=args.bucket_name,
165+
object_name=args.object_name, expiration=int(args.expiration))
166+
167+
print(signed_url)
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Copyright 2018 Google, Inc.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
import os
16+
17+
from google.cloud import storage
18+
import pytest
19+
import requests
20+
21+
import generate_signed_urls
22+
23+
BUCKET = os.environ['CLOUD_STORAGE_BUCKET']
24+
GOOGLE_APPLICATION_CREDENTIALS = os.environ['GOOGLE_APPLICATION_CREDENTIALS']
25+
26+
27+
@pytest.fixture
28+
def test_blob():
29+
"""Provides a pre-existing blob in the test bucket."""
30+
bucket = storage.Client().bucket(BUCKET)
31+
blob = bucket.blob('storage_snippets_test_sigil')
32+
blob.upload_from_string('Hello, is it me you\'re looking for?')
33+
return blob
34+
35+
36+
def test_generate_get_signed_url(test_blob, capsys):
37+
get_signed_url = generate_signed_urls.generate_signed_url(
38+
service_account_file=GOOGLE_APPLICATION_CREDENTIALS,
39+
bucket_name=BUCKET, object_name=test_blob.name,
40+
expiration=60)
41+
response = requests.get(get_signed_url)
42+
assert response.ok

storage/signed_urls/requirements.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
google-cloud-storage==1.13.0
2+
google-auth==1.5.1
3+
six==1.11.0

0 commit comments

Comments
 (0)