fix!: rename fields that collide with builtins; retrieve job config for risk analysis jobs (#75)

fix: retrieve job config for risk analysis jobs
fix!: rename fields that collide with builtins.
  * `ByteContentItem.type` -> `ByteContentItem.type_`
  * `MetadataLocation.type` -> `MetadataLocation.type_`
  * `Container.type` -> `Container.type_`
  * `Bucket.min` -> `Bucket.min_`
  * `Bucket.max `-> `Bucket.max_`
  * `DlpJob.type` -> `DlpJob.type_`
  * `GetDlpJobRequest.type` -> `GetDlpJobRequest.type_`

Author: yoshi-automation

dlp/snippets/README.rst

@@ -5,7 +5,7 @@ Google Data Loss Prevention Python Samples
 ===============================================================================
 
 .. image:: https://gstatic.com/cloudssh/images/open-btn.png
-   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/python-dlp&page=editor&open_in_editor=samples/snippets/README.rst
+   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=dlp/README.rst
 
 
 This directory contains samples for Google Data Loss Prevention. `Google Data Loss Prevention`_ provides programmatic access to a powerful detection engine for personally identifiable information and other privacy-sensitive data in unstructured data streams.
@@ -37,12 +37,11 @@ credentials for applications.
 Install Dependencies
 ++++++++++++++++++++
 
-#. Clone python-dlp and change directory to the sample directory you want to use.
+#. Clone python-docs-samples and change directory to the sample directory you want to use.
 
     .. code-block:: bash
 
-        $ git clone https://github.com/googleapis/python-dlp.git
-        $ cd samples/snippets
+        $ git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git
 
 #. Install `pip`_ and `virtualenv`_ if you do not already have them. You may want to refer to the `Python Development Environment Setup Guide`_ for Google Cloud Platform for instructions.
 
@@ -78,7 +77,7 @@ Quickstart
 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
 .. image:: https://gstatic.com/cloudssh/images/open-btn.png
-   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/python-dlp&page=editor&open_in_editor=samples/snippets/quickstart.py,samples/snippets/README.rst
+   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=dlp/quickstart.py,dlp/README.rst
 
 
 
@@ -96,7 +95,7 @@ Inspect Content
 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
 .. image:: https://gstatic.com/cloudssh/images/open-btn.png
-   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/python-dlp&page=editor&open_in_editor=samples/snippets/inspect_content.py,samples/snippets/README.rst
+   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=dlp/inspect_content.py,dlp/README.rst
 
 
 
@@ -134,7 +133,7 @@ Redact Content
 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
 .. image:: https://gstatic.com/cloudssh/images/open-btn.png
-   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/python-dlp&page=editor&open_in_editor=samples/snippets/redact.py,samples/snippets/README.rst
+   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=dlp/redact.py,dlp/README.rst
 
 
 
@@ -170,7 +169,7 @@ Metadata
 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
 .. image:: https://gstatic.com/cloudssh/images/open-btn.png
-   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/python-dlp&page=editor&open_in_editor=samples/snippets/metadata.py,samples/snippets/README.rst
+   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=dlp/metadata.py,dlp/README.rst
 
 
 
@@ -203,7 +202,7 @@ Jobs
 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
 .. image:: https://gstatic.com/cloudssh/images/open-btn.png
-   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/python-dlp&page=editor&open_in_editor=samples/snippets/jobs.py,samples/snippets/README.rst
+   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=dlp/jobs.py,dlp/README.rst
 
 
 
@@ -236,7 +235,7 @@ Templates
 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
 .. image:: https://gstatic.com/cloudssh/images/open-btn.png
-   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/python-dlp&page=editor&open_in_editor=samples/snippets/templates.py,samples/snippets/README.rst
+   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=dlp/templates.py,dlp/README.rst
 
 
 
@@ -269,7 +268,7 @@ Triggers
 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
 .. image:: https://gstatic.com/cloudssh/images/open-btn.png
-   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/python-dlp&page=editor&open_in_editor=samples/snippets/triggers.py,samples/snippets/README.rst
+   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=dlp/triggers.py,dlp/README.rst
 
 
 
@@ -302,7 +301,7 @@ Risk Analysis
 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
 .. image:: https://gstatic.com/cloudssh/images/open-btn.png
-   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/python-dlp&page=editor&open_in_editor=samples/snippets/risk.py,samples/snippets/README.rst
+   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=dlp/risk.py,dlp/README.rst
 
 
 
@@ -341,7 +340,7 @@ DeID
 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
 .. image:: https://gstatic.com/cloudssh/images/open-btn.png
-   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/python-dlp&page=editor&open_in_editor=samples/snippets/deid.py,samples/snippets/README.rst
+   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=dlp/deid.py,dlp/README.rst
 
 
 

dlp/snippets/inspect_content.py

@@ -401,7 +401,7 @@ def inspect_file(
 
     # Construct the item, containing the file's byte data.
     with open(filename, mode="rb") as f:
-        item = {"byte_item": {"type": content_type_index, "data": f.read()}}
+        item = {"byte_item": {"type_": content_type_index, "data": f.read()}}
 
     # Convert the project id into a full resource id.
     parent = f"projects/{project}"

dlp/snippets/jobs.py

@@ -75,7 +75,7 @@ def list_dlp_jobs(project, filter_string=None, job_type=None):
 
     # Call the API to get a list of jobs.
     response = dlp.list_dlp_jobs(
-        request={"parent": parent, "filter": filter_string, "type": job_type}
+        request={"parent": parent, "filter": filter_string, "type_": job_type}
     )
 
     # Iterate over results.

dlp/snippets/noxfile.py

@@ -37,22 +37,28 @@
 
 TEST_CONFIG = {
     # You can opt out from the test for specific Python versions.
-    "ignored_versions": ["2.7"],
+    'ignored_versions': ["2.7"],
+
+    # Old samples are opted out of enforcing Python type hints
+    # All new samples should feature them
+    'enforce_type_hints': False,
+
     # An envvar key for determining the project id to use. Change it
     # to 'BUILD_SPECIFIC_GCLOUD_PROJECT' if you want to opt in using a
     # build specific Cloud project. You can also use your own string
     # to use your own Cloud project.
-    "gcloud_project_env": "GOOGLE_CLOUD_PROJECT",
+    'gcloud_project_env': 'GOOGLE_CLOUD_PROJECT',
     # 'gcloud_project_env': 'BUILD_SPECIFIC_GCLOUD_PROJECT',
+
     # A dictionary you want to inject into your test. Don't put any
     # secrets here. These values will override predefined values.
-    "envs": {},
+    'envs': {},
 }
 
 
 try:
     # Ensure we can import noxfile_config in the project's directory.
-    sys.path.append(".")
+    sys.path.append('.')
     from noxfile_config import TEST_CONFIG_OVERRIDE
 except ImportError as e:
     print("No user noxfile_config found: detail: {}".format(e))
@@ -67,12 +73,12 @@ def get_pytest_env_vars():
     ret = {}
 
     # Override the GCLOUD_PROJECT and the alias.
-    env_key = TEST_CONFIG["gcloud_project_env"]
+    env_key = TEST_CONFIG['gcloud_project_env']
     # This should error out if not set.
-    ret["GOOGLE_CLOUD_PROJECT"] = os.environ[env_key]
+    ret['GOOGLE_CLOUD_PROJECT'] = os.environ[env_key]
 
     # Apply user supplied envs.
-    ret.update(TEST_CONFIG["envs"])
+    ret.update(TEST_CONFIG['envs'])
     return ret
 
 
@@ -81,7 +87,7 @@ def get_pytest_env_vars():
 ALL_VERSIONS = ["2.7", "3.6", "3.7", "3.8"]
 
 # Any default versions that should be ignored.
-IGNORED_VERSIONS = TEST_CONFIG["ignored_versions"]
+IGNORED_VERSIONS = TEST_CONFIG['ignored_versions']
 
 TESTED_VERSIONS = sorted([v for v in ALL_VERSIONS if v not in IGNORED_VERSIONS])
 
@@ -130,16 +136,29 @@ def _determine_local_import_names(start_dir):
 
 @nox.session
 def lint(session):
-    session.install("flake8", "flake8-import-order")
+    if not TEST_CONFIG['enforce_type_hints']:
+        session.install("flake8", "flake8-import-order")
+    else:
+        session.install("flake8", "flake8-import-order", "flake8-annotations")
 
     local_names = _determine_local_import_names(".")
     args = FLAKE8_COMMON_ARGS + [
         "--application-import-names",
         ",".join(local_names),
-        ".",
+        "."
     ]
     session.run("flake8", *args)
+#
+# Black
+#
+
 
+@nox.session
+def blacken(session):
+    session.install("black")
+    python_files = [path for path in os.listdir(".") if path.endswith(".py")]
+
+    session.run("black", *python_files)
 
 #
 # Sample Tests
@@ -180,9 +199,9 @@ def py(session):
     if session.python in TESTED_VERSIONS:
         _session_tests(session)
     else:
-        session.skip(
-            "SKIPPED: {} tests are disabled for this sample.".format(session.python)
-        )
+        session.skip("SKIPPED: {} tests are disabled for this sample.".format(
+            session.python
+        ))
 
 
 #
@@ -199,6 +218,11 @@ def _get_repo_root():
             break
         if Path(p / ".git").exists():
             return str(p)
+        # .git is not available in repos cloned via Cloud Build
+        # setup.py is always in the library's root, so use that instead
+        # https://github.com/googleapis/synthtool/issues/792
+        if Path(p / "setup.py").exists():
+            return str(p)
         p = p.parent
     raise Exception("Unable to detect repository root.")
 

dlp/snippets/redact.py

@@ -91,7 +91,7 @@ def redact_image(
 
     # Construct the byte_item, containing the file's byte data.
     with open(filename, mode="rb") as f:
-        byte_item = {"type": content_type_index, "data": f.read()}
+        byte_item = {"type_": content_type_index, "data": f.read()}
 
     # Convert the project id into a full resource id.
     parent = f"projects/{project}"
@@ -146,7 +146,7 @@ def redact_image_all_text(
 
     # Construct the byte_item, containing the file's byte data.
     with open(filename, mode="rb") as f:
-        byte_item = {"type": google.cloud.dlp_v2.FileType.IMAGE, "data": f.read()}
+        byte_item = {"type_": google.cloud.dlp_v2.FileType.IMAGE, "data": f.read()}
 
     # Convert the project id into a full resource id.
     parent = f"projects/{project}"