Skip to content

[text analytics] PII updates for v5.1.0b6 #17038

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Mar 5, 2021
Merged

[text analytics] PII updates for v5.1.0b6 #17038

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
9941478
Implemented change for PII categories
Mar 3, 2021
76b5dd6
Some renames
Mar 3, 2021
7cc73f3
Fixed merge conflicts
Mar 3, 2021
a136c4d
Added tests for pii categories
Mar 4, 2021
0b1a714
Removed pii categories from Analyze task; updated test recordings
Mar 5, 2021
6e33c35
Fixed merge conflict
Mar 5, 2021
fbc5bdc
PR comments
Mar 5, 2021
1aa4c76
Updated tests
Mar 5, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions sdk/textanalytics/azure-ai-textanalytics/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@
- Renamed properties `aspect` and `opinions` to `target` and `assessments` respectively in class `MinedOpinion`.
- Renamed classes `AspectSentiment` and `OpinionSentiment` to `TargetSentiment` and `AssessmentSentiment` respectively.

**New Features**

- Added parameter `pii_entity_categories` to the `recognize_pii_entities` client method.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey @abhahn I actually made a gist for this design, here it is, sorry I should've 100% clarified this with you. js also did a pii entity pr that aligns with this (idt they added an enum though)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I've update things to align with the gist above.

- Added parameter `pii_entity_categries` property to class `RecognizePiiEntitiesAction`.
- Added enum `PiiEntityCategory`.

## 5.1.0b5 (2021-02-10)

**Breaking Changes**
Expand Down
2 changes: 2 additions & 0 deletions sdk/textanalytics/azure-ai-textanalytics/azure/ai/textanalytics/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@
AnalyzeBatchActionsError,
)
from ._paging import AnalyzeHealthcareEntitiesResult
from ._generated.v3_1_preview_4.models import PiiCategory as PiiEntityCategory

__all__ = [
'TextAnalyticsApiVersion',
Expand Down Expand Up @@ -83,6 +84,7 @@
'RequestStatistics',
'AnalyzeBatchActionsType',
"AnalyzeBatchActionsError",
"PiiEntityCategory"
]

__version__ = VERSION
63 changes: 22 additions & 41 deletions ...xtanalytics/azure-ai-textanalytics/azure/ai/textanalytics/_generated/_operations_mixin.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,13 +54,11 @@ def analyze_status(
:type skip: int
:keyword callable cls: A custom type or function that will be passed the direct response
:return: AnalyzeJobState, or the result of cls(response)
:rtype: ~azure.ai.textanalytics.v3_1_preview_3.models.AnalyzeJobState
:rtype: ~azure.ai.textanalytics.v3_1_preview_4.models.AnalyzeJobState
:raises: ~azure.core.exceptions.HttpResponseError
"""
api_version = self._get_api_version('analyze_status')
if api_version == 'v3.1-preview.3':
from .v3_1_preview_3.operations import TextAnalyticsClientOperationsMixin as OperationClass
elif api_version == 'v3.1-preview.4':
if api_version == 'v3.1-preview.4':
from .v3_1_preview_4.operations import TextAnalyticsClientOperationsMixin as OperationClass
else:
raise ValueError("API version {} does not have operation 'analyze_status'".format(api_version))
Expand All @@ -83,21 +81,19 @@ def begin_analyze(
executed.

:param body: Collection of documents to analyze and tasks to execute.
:type body: ~azure.ai.textanalytics.v3_1_preview_3.models.AnalyzeBatchInput
:type body: ~azure.ai.textanalytics.v3_1_preview_4.models.AnalyzeBatchInput
:keyword callable cls: A custom type or function that will be passed the direct response
:keyword str continuation_token: A continuation token to restart a poller from a saved state.
:keyword polling: Pass in True if you'd like the AnalyzeBatchActionsLROPollingMethod polling method,
False for no polling, or your own initialized polling object for a personal polling strategy.
:paramtype polling: bool or ~azure.core.polling.PollingMethod
:keyword int polling_interval: Default waiting time between two polls for LRO operations if no Retry-After header is present.
:return: An instance of AnalyzeBatchActionsLROPoller that returns either AnalyzeJobState or the result of cls(response)
:rtype: ~...._lro.AnalyzeBatchActionsLROPoller[~azure.ai.textanalytics.v3_1_preview_3.models.AnalyzeJobState]
:rtype: ~...._lro.AnalyzeBatchActionsLROPoller[~azure.ai.textanalytics.v3_1_preview_4.models.AnalyzeJobState]
:raises ~azure.core.exceptions.HttpResponseError:
"""
api_version = self._get_api_version('begin_analyze')
if api_version == 'v3.1-preview.3':
from .v3_1_preview_3.operations import TextAnalyticsClientOperationsMixin as OperationClass
elif api_version == 'v3.1-preview.4':
if api_version == 'v3.1-preview.4':
from .v3_1_preview_4.operations import TextAnalyticsClientOperationsMixin as OperationClass
else:
raise ValueError("API version {} does not have operation 'begin_analyze'".format(api_version))
Expand Down Expand Up @@ -131,9 +127,7 @@ def begin_cancel_health_job(
:raises ~azure.core.exceptions.HttpResponseError:
"""
api_version = self._get_api_version('begin_cancel_health_job')
if api_version == 'v3.1-preview.3':
from .v3_1_preview_3.operations import TextAnalyticsClientOperationsMixin as OperationClass
elif api_version == 'v3.1-preview.4':
if api_version == 'v3.1-preview.4':
from .v3_1_preview_4.operations import TextAnalyticsClientOperationsMixin as OperationClass
else:
raise ValueError("API version {} does not have operation 'begin_cancel_health_job'".format(api_version))
Expand All @@ -149,7 +143,7 @@ def begin_health(
self,
documents, # type: List["_models.MultiLanguageInput"]
model_version=None, # type: Optional[str]
string_index_type="TextElements_v8", # type: Optional[Union[str, "_models.StringIndexType"]]
string_index_type=None, # type: Optional[Union[str, "_models.StringIndexType"]]
**kwargs # type: Any
):
"""Submit healthcare analysis job.
Expand All @@ -158,28 +152,26 @@ def begin_health(
symptoms, etc) and their relations.

:param documents: The set of documents to process as part of this batch.
:type documents: list[~azure.ai.textanalytics.v3_1_preview_3.models.MultiLanguageInput]
:type documents: list[~azure.ai.textanalytics.v3_1_preview_4.models.MultiLanguageInput]
:param model_version: (Optional) This value indicates which model will be used for scoring. If
a model-version is not specified, the API should default to the latest, non-preview version.
:type model_version: str
:param string_index_type: (Optional) Specifies the method used to interpret string offsets.
Defaults to Text Elements (Graphemes) according to Unicode v8.0.0. For additional information
see https://aka.ms/text-analytics-offsets.
:type string_index_type: str or ~azure.ai.textanalytics.v3_1_preview_3.models.StringIndexType
:type string_index_type: str or ~azure.ai.textanalytics.v3_1_preview_4.models.StringIndexType
:keyword callable cls: A custom type or function that will be passed the direct response
:keyword str continuation_token: A continuation token to restart a poller from a saved state.
:keyword polling: Pass in True if you'd like the AnalyzeHealthcareEntitiesLROPollingMethod polling method,
False for no polling, or your own initialized polling object for a personal polling strategy.
:paramtype polling: bool or ~azure.core.polling.PollingMethod
:keyword int polling_interval: Default waiting time between two polls for LRO operations if no Retry-After header is present.
:return: An instance of AnalyzeHealthcareEntitiesLROPoller that returns either HealthcareJobState or the result of cls(response)
:rtype: ~...._lro.AnalyzeHealthcareEntitiesLROPoller[~azure.ai.textanalytics.v3_1_preview_3.models.HealthcareJobState]
:rtype: ~...._lro.AnalyzeHealthcareEntitiesLROPoller[~azure.ai.textanalytics.v3_1_preview_4.models.HealthcareJobState]
:raises ~azure.core.exceptions.HttpResponseError:
"""
api_version = self._get_api_version('begin_health')
if api_version == 'v3.1-preview.3':
from .v3_1_preview_3.operations import TextAnalyticsClientOperationsMixin as OperationClass
elif api_version == 'v3.1-preview.4':
if api_version == 'v3.1-preview.4':
from .v3_1_preview_4.operations import TextAnalyticsClientOperationsMixin as OperationClass
else:
raise ValueError("API version {} does not have operation 'begin_health'".format(api_version))
Expand Down Expand Up @@ -220,8 +212,6 @@ def entities_linking(
api_version = self._get_api_version('entities_linking')
if api_version == 'v3.0':
from .v3_0.operations import TextAnalyticsClientOperationsMixin as OperationClass
elif api_version == 'v3.1-preview.3':
from .v3_1_preview_3.operations import TextAnalyticsClientOperationsMixin as OperationClass
elif api_version == 'v3.1-preview.4':
from .v3_1_preview_4.operations import TextAnalyticsClientOperationsMixin as OperationClass
else:
Expand Down Expand Up @@ -264,8 +254,6 @@ def entities_recognition_general(
api_version = self._get_api_version('entities_recognition_general')
if api_version == 'v3.0':
from .v3_0.operations import TextAnalyticsClientOperationsMixin as OperationClass
elif api_version == 'v3.1-preview.3':
from .v3_1_preview_3.operations import TextAnalyticsClientOperationsMixin as OperationClass
elif api_version == 'v3.1-preview.4':
from .v3_1_preview_4.operations import TextAnalyticsClientOperationsMixin as OperationClass
else:
Expand All @@ -284,7 +272,8 @@ def entities_recognition_pii(
model_version=None, # type: Optional[str]
show_stats=None, # type: Optional[bool]
domain=None, # type: Optional[str]
string_index_type="TextElements_v8", # type: Optional[Union[str, "_models.StringIndexType"]]
string_index_type=None, # type: Optional[Union[str, "_models.StringIndexType"]]
pii_categories=None, # type: Optional[List[Union[str, "_models.PiiCategory"]]]
**kwargs # type: Any
):
"""Entities containing personal information.
Expand All @@ -296,7 +285,7 @@ def entities_recognition_pii(
list of enabled languages.

:param documents: The set of documents to process as part of this batch.
:type documents: list[~azure.ai.textanalytics.v3_1_preview_3.models.MultiLanguageInput]
:type documents: list[~azure.ai.textanalytics.v3_1_preview_4.models.MultiLanguageInput]
:param model_version: (Optional) This value indicates which model will be used for scoring. If
a model-version is not specified, the API should default to the latest, non-preview version.
:type model_version: str
Expand All @@ -309,16 +298,16 @@ def entities_recognition_pii(
:param string_index_type: (Optional) Specifies the method used to interpret string offsets.
Defaults to Text Elements (Graphemes) according to Unicode v8.0.0. For additional information
see https://aka.ms/text-analytics-offsets.
:type string_index_type: str or ~azure.ai.textanalytics.v3_1_preview_3.models.StringIndexType
:type string_index_type: str or ~azure.ai.textanalytics.v3_1_preview_4.models.StringIndexType
:param pii_categories: (Optional) describes the PII categories to return.
:type pii_categories: list[str or ~azure.ai.textanalytics.v3_1_preview_4.models.PiiCategory]
:keyword callable cls: A custom type or function that will be passed the direct response
:return: PiiResult, or the result of cls(response)
:rtype: ~azure.ai.textanalytics.v3_1_preview_3.models.PiiResult
:rtype: ~azure.ai.textanalytics.v3_1_preview_4.models.PiiResult
:raises: ~azure.core.exceptions.HttpResponseError
"""
api_version = self._get_api_version('entities_recognition_pii')
if api_version == 'v3.1-preview.3':
from .v3_1_preview_3.operations import TextAnalyticsClientOperationsMixin as OperationClass
elif api_version == 'v3.1-preview.4':
if api_version == 'v3.1-preview.4':
from .v3_1_preview_4.operations import TextAnalyticsClientOperationsMixin as OperationClass
else:
raise ValueError("API version {} does not have operation 'entities_recognition_pii'".format(api_version))
Expand All @@ -328,7 +317,7 @@ def entities_recognition_pii(
mixin_instance._serialize = Serializer(self._models_dict(api_version))
mixin_instance._serialize.client_side_validation = False
mixin_instance._deserialize = Deserializer(self._models_dict(api_version))
return mixin_instance.entities_recognition_pii(documents, model_version, show_stats, domain, string_index_type, **kwargs)
return mixin_instance.entities_recognition_pii(documents, model_version, show_stats, domain, string_index_type, pii_categories, **kwargs)

def health_status(
self,
Expand All @@ -355,13 +344,11 @@ def health_status(
:type show_stats: bool
:keyword callable cls: A custom type or function that will be passed the direct response
:return: HealthcareJobState, or the result of cls(response)
:rtype: ~azure.ai.textanalytics.v3_1_preview_3.models.HealthcareJobState
:rtype: ~azure.ai.textanalytics.v3_1_preview_4.models.HealthcareJobState
:raises: ~azure.core.exceptions.HttpResponseError
"""
api_version = self._get_api_version('health_status')
if api_version == 'v3.1-preview.3':
from .v3_1_preview_3.operations import TextAnalyticsClientOperationsMixin as OperationClass
elif api_version == 'v3.1-preview.4':
if api_version == 'v3.1-preview.4':
from .v3_1_preview_4.operations import TextAnalyticsClientOperationsMixin as OperationClass
else:
raise ValueError("API version {} does not have operation 'health_status'".format(api_version))
Expand Down Expand Up @@ -402,8 +389,6 @@ def key_phrases(
api_version = self._get_api_version('key_phrases')
if api_version == 'v3.0':
from .v3_0.operations import TextAnalyticsClientOperationsMixin as OperationClass
elif api_version == 'v3.1-preview.3':
from .v3_1_preview_3.operations import TextAnalyticsClientOperationsMixin as OperationClass
elif api_version == 'v3.1-preview.4':
from .v3_1_preview_4.operations import TextAnalyticsClientOperationsMixin as OperationClass
else:
Expand Down Expand Up @@ -446,8 +431,6 @@ def languages(
api_version = self._get_api_version('languages')
if api_version == 'v3.0':
from .v3_0.operations import TextAnalyticsClientOperationsMixin as OperationClass
elif api_version == 'v3.1-preview.3':
from .v3_1_preview_3.operations import TextAnalyticsClientOperationsMixin as OperationClass
elif api_version == 'v3.1-preview.4':
from .v3_1_preview_4.operations import TextAnalyticsClientOperationsMixin as OperationClass
else:
Expand Down Expand Up @@ -490,8 +473,6 @@ def sentiment(
api_version = self._get_api_version('sentiment')
if api_version == 'v3.0':
from .v3_0.operations import TextAnalyticsClientOperationsMixin as OperationClass
elif api_version == 'v3.1-preview.3':
from .v3_1_preview_3.operations import TextAnalyticsClientOperationsMixin as OperationClass
elif api_version == 'v3.1-preview.4':
from .v3_1_preview_4.operations import TextAnalyticsClientOperationsMixin as OperationClass
else:
Expand Down
7 changes: 1 addition & 6 deletions ...lytics/azure-ai-textanalytics/azure/ai/textanalytics/_generated/_text_analytics_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
from typing import Any, Optional

from azure.core.credentials import TokenCredential
from azure.core.pipeline.transport import HttpRequest, HttpResponse

class _SDKClient(object):
def __init__(self, *args, **kwargs):
Expand Down Expand Up @@ -73,8 +74,6 @@ def __init__(
):
if api_version == 'v3.0':
base_url = '{Endpoint}/text/analytics/v3.0'
elif api_version == 'v3.1-preview.3':
base_url = '{Endpoint}/text/analytics/v3.1-preview.3'
elif api_version == 'v3.1-preview.4':
base_url = '{Endpoint}/text/analytics/v3.1-preview.4'
else:
Expand All @@ -95,15 +94,11 @@ def models(cls, api_version=DEFAULT_API_VERSION):
"""Module depends on the API version:

* v3.0: :mod:`v3_0.models<azure.ai.textanalytics.v3_0.models>`
* v3.1-preview.3: :mod:`v3_1_preview_3.models<azure.ai.textanalytics.v3_1_preview_3.models>`
* v3.1-preview.4: :mod:`v3_1_preview_4.models<azure.ai.textanalytics.v3_1_preview_4.models>`
"""
if api_version == 'v3.0':
from .v3_0 import models
return models
elif api_version == 'v3.1-preview.3':
from .v3_1_preview_3 import models
return models
elif api_version == 'v3.1-preview.4':
from .v3_1_preview_4 import models
return models
Expand Down
Loading