Skip to content

Applying changes to master #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
4 commits merged into from
May 22, 2019
Merged

Applying changes to master #3

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4ba4d9e
moving print statement to correct place
jfalkenstein May 22, 2019
9ea3554
incrementing version
jfalkenstein May 22, 2019
5c3f4a0
making backoff catch the code correctly
jfalkenstein May 22, 2019
9ce4523
fixing indentation level
jfalkenstein May 22, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 11 additions & 7 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,11 @@
if sys.version_info[0] >= 3:
# Python 3
from urllib.request import urlopen
from urllib.error import HTTPError
else:
# Python 2
from urllib2 import urlopen
from urllib2 import HTTPError

#
# This script modifies the basic setuptools by adding some functionality to the standard
Expand All @@ -52,7 +54,7 @@

PACKAGE_NAME = 'amazon_kclpy'
JAR_DIRECTORY = os.path.join(PACKAGE_NAME, 'jars')
PACKAGE_VERSION = '2.0.2'
PACKAGE_VERSION = '2.0.5'
PYTHON_REQUIREMENTS = [
'boto',
# argparse is part of python2.7 but must be declared for python2.6
Expand Down Expand Up @@ -182,7 +184,6 @@ def download_file(self, url, dest):
"""
Downloads a file at the url to the destination.
"""
print('Attempting to retrieve remote jar {url}'.format(url=url))
try:
response = self.make_request_with_backoff(url)

Expand All @@ -204,11 +205,14 @@ def download_files(self):

def make_request_with_backoff(self, url):
for attempt_number in range(MAX_URL_DOWNLOAD_ATTEMPTS):
response = urlopen(url)
if response.getcode() == 429:
sleep_time = 2 ** attempt_number
print('"429 Too Many Requests" response received. Sleeping {} seconds and trying again.'.format(sleep_time))
sleep(sleep_time)
print('Attempting to retrieve remote jar {url}'.format(url=url))
try:
return urlopen(url)
except HTTPError as e:
if e.code == 429:
sleep_time = 2 ** attempt_number
print('"429 Too Many Requests" response received. Sleeping {} seconds and trying again.'.format(sleep_time))
sleep(sleep_time)
else:
return response
raise Exception('"429 Too Many Requests" responses received.')
Expand Down