Skip to content

[Text Analytics] Add piiCategories to recognizePiiEntities #14043

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Mar 2, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions sdk/textanalytics/ai-text-analytics/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
- `beginAnalyzeHealthcareEntities` returns `entityRelations` per document, a list of relations between healthcare entities.
- `beginAnalyzeHealthcareEntities` entities now include `assertions` instead of `isNegated` which gives more context about the respective entity.
- [Breaking] `beginAnalyzeHealthcareEntities` no longer returns `relatedEntities`.
- `recognizePiiEntities` takes a new option, `categoriesFilter`, that specifies a list of Pii categories to return.

## 5.1.0-beta.4 (2021-02-10)

Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -333,13 +333,13 @@ export type PagedAsyncIterableAnalyzeBatchActionsResult = PagedAsyncIterableIter
// @public
export type PagedAsyncIterableAnalyzeHealthcareEntitiesResult = PagedAsyncIterableIterator<AnalyzeHealthcareEntitiesResult, AnalyzeHealthcareEntitiesResultArray>;

// @public
export type PiiCategory = string;

// @public
export interface PiiEntity extends Entity {
}

// @public
export type PiiEntityCategory = string;

// @public
export enum PiiEntityDomainType {
PROTECTED_HEALTH_INFORMATION = "PHI"
Expand Down Expand Up @@ -428,7 +428,7 @@ export type RecognizePiiEntitiesAction = {
domain?: PiiEntityDomainType;
modelVersion?: string;
stringIndexType?: StringIndexType;
piiCategories?: PiiCategory[];
categoriesFilter?: PiiEntityCategory[];
};

// @public
Expand All @@ -447,6 +447,7 @@ export type RecognizePiiEntitiesErrorResult = TextAnalyticsErrorResult;

// @public
export interface RecognizePiiEntitiesOptions extends TextAnalyticsOperationOptions {
categoriesFilter?: PiiEntityCategory[];
domainFilter?: PiiEntityDomainType;
stringIndexType?: StringIndexType;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,32 +24,47 @@ async function main() {

const client = new TextAnalyticsClient(endpoint, new AzureKeyCredential(apiKey));

const [result] = await client.recognizePiiEntities(["My phone number is 555-5555"]);
const textOnePii = "My phone number is 555-5555";
const textNoPHI =
"FIFA is a non-profit organization which describes itself as an international governing body of association football.";
const textMultiplePIIs = "Patient name is Joe and SSN is 859-98-0987";

const [result] = await client.recognizePiiEntities([textOnePii]);

if (!result.error) {
console.log(`The redacted text: ${result.redactedText}`);
console.log(
`The redacted text is "${result.redactedText}" and found the following PII entities`
);
for (const entity of result.entities) {
console.log(`Found PII entity ${entity.text} of type ${entity.category}`);
console.log(`\t- "${entity.text}" of type ${entity.category}`);
}
}

const textNoPHI =
"FIFA is a non-profit organization which describes itself as an international governing body of association football.";
console.log(`There are no PHI entities in this text: ${textNoPHI}`);
const [resultWithPHI] = await client.recognizePiiEntities(
[{ id: "0", text: textNoPHI, language: "en" }],
{ domainFilter: PiiEntityDomainType.PROTECTED_HEALTH_INFORMATION }
);
if (!resultWithPHI.error) {
console.log(`Also there is nothing to redact: ${resultWithPHI.redactedText}`);
assert(resultWithPHI.entities.length === 0);
assert(resultWithPHI.entities.length === 0, "did not expect any entities but got some");
}

console.log(`But there are other entities in that text`);
console.log(`But there are other PII entities in that text:`);
const [resultWithoutPHI] = await client.recognizePiiEntities([textNoPHI]);
if (!resultWithoutPHI.error) {
for (const entity of resultWithoutPHI.entities) {
console.log(`Found PII entity ${entity.text} of type ${entity.category}`);
console.log(`\t- "${entity.text}" of type ${entity.category}`);
}
}
const [resultWithSSNPII] = await client.recognizePiiEntities([textMultiplePIIs], "en", {
categoriesFilter: ["USSocialSecurityNumber"]
});
if (!resultWithSSNPII.error) {
console.log("The service was asked to return just SSN entities:");
for (const entity of resultWithSSNPII.entities) {
console.log(`\t- "${entity.text}"`);
assert(entity.category === "USSocialSecurityNumber");
}
}
}
Expand Down
Loading