Skip to content

Commit 824a5a2

Browse files
authored
Devtools (#5759)
* Initial commit * Initial commit * Initial commit * wip: initial work - incomplete * Copy code from https://github.com/Azure/azure-cli/tree/master/src/azure-cli-testsdk * Add vcrpy version from azure-cli * Add VSCode ignores * Add Azure imports * Start converting CLI commands to SDK calls * Minor proofreading * Add required common and mock modules * Add deps, update pkgs and url * Package structure * Add more editor ignores * Remove azure deps and nonexistent HISTORY ref * Remove Azure mgmt dependencies * Remove azure.common dependency * Add dependency on six * Remove CLI-specific stuff * Remove CLI exception * Remove most CLI-centric patches * Remove older CLI-centric base class * Add sleep patch back in * First steps toward a unified config * Config and no-record headers Add config object to ScenarioTest Add fake header for deactivating recording Align cmdline options with vcr.py options * Move recording-deactivation header detection to right place * Add way to set default config file from scenario test * Remove 'CLI' from env vars * Reinstate patch_long_run_operation_delay * New record disabling mechanism * Modify config options for better backwards compatibility * Ignore more VS stuff * Remove jmespath dependency, update vcrpy * Remove checkers * Remove unuseds, make patches/processors kwargs * Remove checkers, used only for CLI * Update import and instance var name * Remove unused imports * line organization * Update version number and vcrpy version * Add a little more to README, add setup.cfg * Add unit tests for TestConfig * Add travis.yml and update dependencies * Update author email * Dummy push to try to trigger a Travis build * Another dummy commit for Travis * Switch to README.rst * Update Travis tag pointer * Use io to get encoding arg when reading README * Remove unnecessary import * Expose most everything in top-level namespace * ScenarioTest -> ReplayableTest; update version * Update version * Remove unused os.path import * Fix super call * Fix super call in setUp * Update ReplayableTest import * Use earlier vcrpy, update version num * Don't fix vcrpy version * Drop "cli" from default random name prefix * Add deployment to Travis * Specify universal wheel * Bump version to 0.2.2 for testing CI release process * Let preparer subclasses override create_random_name * De-"privatize" mock_in_unit_test and expose it * Bump version to 0.3.0 * Set disable_recording from kwarg * Don't treat mock_in_unit_test as a unittest * Remove skip for mock_in_unit_test, specify test dir * Update version numbers * Remove extraneous requirements * Remove outdated README.md stuff * Update install for new requirements.txt * Add 3.6-dev to pythons * Update code style and run pylint in Travis * Integrate with codecov.io * Add test cover IntegrationTestBase Also check in IntelliJ IDEA workspace configuration just in case someone else is using PyCharm * Add test case for LiveTest constructor * Add tests cover create_random_name * Add test covering get_sha1_hash * Add test cover RecordingProcessor base class * Add test covering SubscriptionRecordingProcessor * Configure code coverage using configuration file * Use unittest instead of nosetests to drive the automation * Fix a few code style issues * Rename coveragerc file * Add AccessTokenProcessor * Split intro and overview into clauses * Back to paragraphs * Update intro, add vcr link * Update intro paragraph to be more general * Remove reference to command modules * Update test and module refs; clarify VCR.py role * Remove outdated reference to class with builtin preparer * Replace intro text Use intro stolen from defunct recording_vcr_tests.md * s/ScenarioTest/ReplayableTest/ * Add note about semantic linefeeds * Make semantic linefeeds note an HTML comment * Remove CLI examples and add links to consumers * Remove doc for legacy test case class * Finish sentence * Add subclass kwarg information * Fix subscription ID removal and test new cases * Fix no-self-use linter complaint * Downgrade the version of vcrpy dependency from 1.11.1 to 1.11.0 due to a sympton similar to this kevin1024/vcrpy#318 * Bump version to 0.5.0 to prepare for release * Fix import on Py3 not-TravisCI * Fix recordmode * Tight config test * url parsing fix * fixed pylint import order * pylint import fix * added record processor for slashes * changed processor name * Bump version 0.5.0 => 0.5.1 * Fix SubscriptionIdReplacer * Update version 0.5.1 => 0.5.2 * Update __init__.py * Detect leaked LRO poller * PyLint happiness * Improve error message * wip * fix error * use base 64 * simplify * add comments * fix tests * fix lint error * use newer pylint to avoid invalid line errors * add a new test * address review feedback * address review feedback * support large response payload * remove useless single quots from the error message * setup: update version * do not use preparer model for allowing large payload * address review feedback * Support Autorest.Python 3.x LRO leak (#42) * Support Autorest.Python 3.x LRO leak * PyLint happyness * Bump version to 0.5.5 * Update the resource removal sequence Also: 1. Adopt py.test over nosetest 2. Cleanup some test code * remove x-ms-authorization-auxiliary header * fix lint error * update version * Add some CI tools to devtools * Readme update * Some tests configuration * Use Pytest as test launcher * PyLint fixes * CI tools tests are Py3.6 only * PyLint CI tools only if 3.6 * CI tools tests recording * Dont't Pylint the tests * Fix incorrect usage of os.environ * Make tests X-platform valid * Fix test dependent of traceback * Robust preparer testing * PyGithub 1.40 * Release 1.1.0 * Allow clone_to_path to have both PR and SHA1 (#49) * Allow clone_to_path to have both PR and SHA1 * Fix test * Update github_tools.py (#55) * SingleValueReplacer.process_request properly decodes request.body if its type is byte. (#56) * Remove pointless files * Move to tools * Update dev_requirement file * Don't record requests to AAD OAuth2 v2.0 endpoint * Use devtools from repo, not from PyPI * Fix mock dep
1 parent e4d9945 commit 824a5a2

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

49 files changed

+3557
-5
lines changed

scripts/dev_setup.py

+11-5
Original file line numberDiff line numberDiff line change
@@ -74,15 +74,20 @@ def pip_command(command, additional_dir=".", error_ok=False):
7474
]
7575
)
7676

77-
# Put azure-common in front
78-
if "azure-common" in content_packages:
79-
content_packages.remove("azure-common")
80-
content_packages.insert(0, "azure-common")
77+
# Install tests dep first
78+
if "azure-devtools" in content_packages:
79+
content_packages.remove("azure-devtools")
80+
content_packages.insert(0, "azure-devtools")
8181

8282
if "azure-sdk-tools" in content_packages:
8383
content_packages.remove("azure-sdk-tools")
8484
content_packages.insert(1, "azure-sdk-tools")
8585

86+
# Put azure-common in front of content package
87+
if "azure-common" in content_packages:
88+
content_packages.remove("azure-common")
89+
content_packages.insert(2, "azure-common")
90+
8691
print("Running dev setup...")
8792
print("Root directory '{}'\n".format(root_dir))
8893

@@ -100,8 +105,9 @@ def pip_command(command, additional_dir=".", error_ok=False):
100105
pip_command("install {}/{}/".format(packages[package_name], package_name))
101106

102107
# install packages
108+
print("Packages to install: {}".format(content_packages))
103109
for package_name in content_packages:
104-
print("Installing {}".format(package_name))
110+
print("\nInstalling {}".format(package_name))
105111
# if we are running dev_setup with no arguments. going after dev_requirements will be a pointless exercise
106112
# and waste of cycles as all the dependencies will be installed regardless.
107113
if os.path.isfile(

tools/azure-devtools/LICENSE

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) Microsoft Corporation. All rights reserved.
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE

tools/azure-devtools/README.rst

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
.. image:: https://travis-ci.org/Azure/azure-python-devtools.svg?branch=master
2+
:target: https://travis-ci.org/Azure/azure-python-devtools
3+
4+
Development tools for Python-based Azure tools
5+
==============================================
6+
7+
This package contains tools to aid in developing Python-based Azure code.
8+
9+
This includes the following components:
10+
11+
scenario_tests
12+
--------------
13+
14+
A testing framework to handle much of the busywork
15+
associated with testing code that interacts with Azure.
16+
17+
ci_tools
18+
--------
19+
20+
Some tooling to help developing CI tools. This includes some Git helpers,
21+
Github RestAPI wrapper and a Bot framework for Github issues.
22+
23+
Contributing
24+
============
25+
26+
This project has adopted the
27+
`Microsoft Open Source Code of Conduct <https://opensource.microsoft.com/codeofconduct/>`__.
28+
For more information see the
29+
`Code of Conduct FAQ <https://opensource.microsoft.com/codeofconduct/faq/>`__
30+
or contact
31+
32+
with any additional questions or comments.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
-e .[ci_tools]
2+
3+
mock;python_version<="2.7"
4+
pytest
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
# How to write ReplayableTest based VCR tests
2+
3+
The `scenario_tests` package uses the [VCR.py](https://pypi.python.org/pypi/vcrpy) library
4+
to record the HTTP messages exchanged during a program run
5+
and play them back at a later time,
6+
making it useful for creating "scenario tests"
7+
that interact with Azure (or other) services.
8+
These tests can be replayed at a later time without any network activity,
9+
allowing us to detect changes in the Python layers
10+
between the code being tested and the underlying REST API.
11+
12+
13+
## Overview
14+
15+
Tests all derive from the `ReplayableTest` class
16+
found in `azure_devtools.scenario_tests.base`.
17+
This class exposes the VCR tests using the standard Python `unittest` framework
18+
and allows the tests to be discovered by and debugged in Visual Studio.
19+
20+
When you run a test,
21+
the test driver will automatically detect the test is unrecorded
22+
and record the HTTP requests and responses in a .yaml file
23+
(referred to by VCR.py as a "cassette").
24+
If the test succeeds, the cassette will be preserved
25+
and future playthroughs of the test will come from the cassette
26+
rather than using actual network communication.
27+
28+
If the tests are run on TravisCI,
29+
any tests which cannot be replayed will automatically fail.
30+
31+
`ReplayableTest` itself derives from `IntegrationTestBase`,
32+
which provides some helpful methods for use in more general unit tests
33+
but no functionality pertaining to network communication.
34+
35+
36+
## Configuring ReplayableTest
37+
38+
The only configuration of `ReplayableTest` that is "exposed"
39+
(in the sense of being accessible other than through subclassing)
40+
is whether tests should be run in "live" or "playback" mode.
41+
This can be set in the following two ways,
42+
of which the first takes precedence:
43+
* Set the environment variable `AZURE_TEST_RUN_LIVE`.
44+
Any value will cause the tests to run in live mode;
45+
if the variable is unset the default of playback mode will be used.
46+
* Specify a boolean value for `live-mode` in a configuration file,
47+
the path to which must be specified by a `ReplayableTest` subclass as described below
48+
(i.e. by default no config file will be read).
49+
True values mean "live" mode; false ones mean "playback."
50+
51+
"Live" and "playback" mode are actually just shorthand for recording modes
52+
in the underlying VCR.py package;
53+
they correspond to "all" and "once"
54+
as described in the [VCR.py documentation](http://vcrpy.readthedocs.io/en/latest/usage.html#record-modes).
55+
56+
### Subclassing ReplayableTest and features
57+
58+
Most customization of `ReplayableTest` is accessible only through subclassing.
59+
60+
The two main users of `ReplayableTest` are
61+
[azure-cli](https://github.com/Azure/azure-cli)
62+
and [azure-sdk-for-python](https://github.com/Azure/azure-sdk-for-python).
63+
Each uses a subclass of `ReplayableTest` to add context-specific functionality
64+
and preserve backward compatibility with test code
65+
prior to the existence of `azure-devtools`.
66+
For example, azure-cli's [compatibility layer](https://github.com/Azure/azure-cli/tree/master/src/azure-cli-testsdk)
67+
adds methods for running CLI commands and evaluating their output.
68+
69+
Subclasses of `ReplayableTest` can configure its behavior
70+
by passing the following keyword arguments when they call
71+
its `__init__` method (probably using `super`):
72+
73+
* `config_file`: Path to a configuration file.
74+
It should be in the format described in Python's
75+
[ConfigParser](https://docs.python.org/3/library/configparser.html) docs
76+
and currently allows only the boolean option `live-mode`.
77+
* `recording_dir` and `recording_name`:
78+
Directory path and file name, respectively,
79+
for the recording that should be used for a given test case.
80+
By default, the directory will be a `recordings` directory
81+
in the same location as the file containing the test case,
82+
and the file name will be the same as the test method name.
83+
A `.yaml` extension will be appended to whatever is used for `recording_name`.
84+
* `recording_processors` and `replay_processors`:
85+
Lists of `RecordingProcessor` instances for making changes to requests and responses
86+
during test recording and test playback, respectively.
87+
See [recording_processors.py](src/azure_devtools/scenario_tests/recording_processors.py)
88+
for some examples and how to implement them.
89+
* `recording_patches` and `replay_patches`:
90+
Lists of patches to apply to functions, methods, etc.
91+
during test recording and playback, respectively.
92+
See [patches.py](src/azure_devtools/scenario_tests/patches.py)
93+
for some examples. Note the `mock_in_unit_test` function
94+
which abstracts out some boilerplate for applying a patch.
95+
96+
97+
<!--
98+
Note: This document's source uses
99+
[semantic linefeeds](http://rhodesmill.org/brandon/2012/one-sentence-per-line/)
100+
to make diffs and updates clearer.
101+
-->

tools/azure-devtools/scripts/ci.sh

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
#!/usr/bin/env bash
2+
3+
pylint src/azure_devtools
4+
pytest src/azure_devtools/scenario_tests/tests --cov=./

tools/azure-devtools/setup.cfg

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
[bdist_wheel]
2+
universal=1

tools/azure-devtools/setup.py

+63
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
#!/usr/bin/env python
2+
3+
# --------------------------------------------------------------------------------------------
4+
# Copyright (c) Microsoft Corporation. All rights reserved.
5+
# Licensed under the MIT License. See License.txt in the project root for license information.
6+
# --------------------------------------------------------------------------------------------
7+
8+
import io
9+
from setuptools import setup
10+
11+
12+
VERSION = "1.1.1"
13+
14+
15+
CLASSIFIERS = [
16+
'Development Status :: 3 - Alpha',
17+
'Intended Audience :: Developers',
18+
'Programming Language :: Python',
19+
'Programming Language :: Python :: 2',
20+
'Programming Language :: Python :: 2.7',
21+
'Programming Language :: Python :: 3',
22+
'Programming Language :: Python :: 3.4',
23+
'Programming Language :: Python :: 3.5',
24+
'Programming Language :: Python :: 3.6',
25+
'License :: OSI Approved :: MIT License',
26+
]
27+
28+
29+
DEPENDENCIES = [
30+
'ConfigArgParse>=0.12.0',
31+
'six>=1.10.0',
32+
'vcrpy>=1.11.0',
33+
]
34+
35+
with io.open('README.rst', 'r', encoding='utf-8') as f:
36+
README = f.read()
37+
38+
setup(
39+
name='azure-devtools',
40+
version=VERSION,
41+
description='Microsoft Azure Development Tools for SDK',
42+
long_description=README,
43+
license='MIT',
44+
author='Microsoft Corporation',
45+
author_email='[email protected]',
46+
url='https://github.com/Azure/azure-python-devtools',
47+
zip_safe=False,
48+
classifiers=CLASSIFIERS,
49+
packages=[
50+
'azure_devtools',
51+
'azure_devtools.scenario_tests',
52+
'azure_devtools.ci_tools',
53+
],
54+
extras_require={
55+
'ci_tools':[
56+
"PyGithub>=1.40", # Can Merge PR after 1.36, "requests" and tests after 1.40
57+
"GitPython",
58+
"requests>=2.0"
59+
]
60+
},
61+
package_dir={'': 'src'},
62+
install_requires=DEPENDENCIES,
63+
)

tools/azure-devtools/src/azure_devtools/__init__.py

Whitespace-only changes.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# --------------------------------------------------------------------------------------------
2+
# Copyright (c) Microsoft Corporation. All rights reserved.
3+
# Licensed under the MIT License. See License.txt in the project root for license information.
4+
# --------------------------------------------------------------------------------------------
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
from collections import namedtuple
2+
from functools import lru_cache
3+
import logging
4+
import os
5+
import re
6+
7+
from github import Github, GithubException, UnknownObjectException
8+
9+
from .github_tools import (
10+
exception_to_github,
11+
)
12+
13+
_LOGGER = logging.getLogger(__name__)
14+
15+
16+
def order(function):
17+
function.bot_order = True
18+
return function
19+
20+
WebhookMetadata = namedtuple(
21+
'WebhookMetadata',
22+
['repo', 'issue', 'text', 'comment']
23+
)
24+
25+
def build_from_issue_comment(gh_token, body):
26+
"""Create a WebhookMetadata from a comment added to an issue.
27+
"""
28+
if body["action"] in ["created", "edited"]:
29+
github_con = Github(gh_token)
30+
repo = github_con.get_repo(body['repository']['full_name'])
31+
issue = repo.get_issue(body['issue']['number'])
32+
text = body['comment']['body']
33+
try:
34+
comment = issue.get_comment(body['comment']['id'])
35+
except UnknownObjectException:
36+
# If the comment has already disapeared, skip the command
37+
return None
38+
return WebhookMetadata(repo, issue, text, comment)
39+
return None
40+
41+
def build_from_issues(gh_token, body):
42+
"""Create a WebhookMetadata from an opening issue text.
43+
"""
44+
if body["action"] in ["opened", "edited"]:
45+
github_con = Github(gh_token)
46+
repo = github_con.get_repo(body['repository']['full_name'])
47+
issue = repo.get_issue(body['issue']['number'])
48+
text = body['issue']['body']
49+
comment = issue # It's where we update the comment: in the issue itself
50+
return WebhookMetadata(repo, issue, text, comment)
51+
return None
52+
53+
@lru_cache()
54+
def robot_name_from_env_variable():
55+
github_con = Github(os.environ["GH_TOKEN"])
56+
return github_con.get_user().login
57+
58+
59+
class BotHandler:
60+
def __init__(self, handler, robot_name=None, gh_token=None):
61+
self.handler = handler
62+
self.gh_token = gh_token or os.environ["GH_TOKEN"]
63+
self.robot_name = robot_name or robot_name_from_env_variable()
64+
65+
def _is_myself(self, body):
66+
return body['sender']['login'].lower() == self.robot_name.lower()
67+
68+
def issue_comment(self, body):
69+
if self._is_myself(body):
70+
return {'message': 'I don\'t talk to myself, I\'m not schizo'}
71+
webhook_data = build_from_issue_comment(self.gh_token, body)
72+
return self.manage_comment(webhook_data)
73+
74+
def issues(self, body):
75+
if self._is_myself(body):
76+
return {'message': 'I don\'t talk to myself, I\'m not schizo'}
77+
webhook_data = build_from_issues(self.gh_token, body)
78+
return self.manage_comment(webhook_data)
79+
80+
def orders(self):
81+
"""Return method tagged "order" in the handler.
82+
"""
83+
return [order_cmd for order_cmd in dir(self.handler)
84+
if getattr(getattr(self.handler, order_cmd), "bot_order", False)]
85+
86+
def manage_comment(self, webhook_data):
87+
if webhook_data is None:
88+
return {'message': 'Nothing for me'}
89+
# Is someone talking to me:
90+
message = re.search("@{} (.*)".format(self.robot_name), webhook_data.text, re.I)
91+
response = None
92+
if message:
93+
command = message.group(1)
94+
split_text = command.lower().split()
95+
orderstr = split_text.pop(0)
96+
if orderstr == "help":
97+
response = self.help_order()
98+
elif orderstr in self.orders():
99+
try: # Reaction is fun, but it's preview not prod.
100+
# Be careful, don't fail the command if we can't thumbs up...
101+
webhook_data.comment.create_reaction("+1")
102+
except GithubException:
103+
pass
104+
with exception_to_github(webhook_data.issue): # Just in case
105+
response = getattr(self.handler, orderstr)(webhook_data.issue, *split_text)
106+
else:
107+
response = "I didn't understand your command:\n```bash\n{}\n```\nin this context, sorry :(\n".format(
108+
command
109+
)
110+
response += self.help_order()
111+
if response:
112+
webhook_data.issue.create_comment(response)
113+
return {'message': response}
114+
return {'message': 'Nothing for me or exception'}
115+
116+
def help_order(self):
117+
orders = ["This is what I can do:"]
118+
for orderstr in self.orders():
119+
orders.append("- `{}`".format(orderstr))
120+
orders.append("- `help` : this help message")
121+
return "\n".join(orders)

0 commit comments

Comments
 (0)