Multi-Modal-Content-Safety-Evaluators #38002
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Export the subscription data from the service connection * Update deploy-test-resources.yml --------- Co-authored-by: Wes Haggard <[email protected]> Co-authored-by: Wes Haggard <[email protected]>
…re#37709) * Update task_query_response.prompty remove required keys * Update task_simulate.prompty * Update task_query_response.prompty * Update task_simulate.prompty * Remove private variable and use kwargs * Add experimental tag to adv sim --------- Co-authored-by: Nagkumar Arkalgud <[email protected]>
* Initial draft * Adding tests * Renaming parameter * Update container.py * Renaming test file * Fixing LINT issues * Update container.py * Update _base.py * Update _base.py * Fixing tests * Fixing tests * Adding support to disable response payload on write for AIO * Update CHANGELOG.md * Update _cosmos_client.py * Reacting to code review comments * Addressing code review feedback * Addressed CR feedback * Fixing pyLint errors * Fixing pylint errors * Update test_crud.py * Fixing svc regression * Update sdk/cosmos/azure-cosmos/azure/cosmos/aio/_container.py Co-authored-by: Anna Tisch <[email protected]> * Reacting to code review feedback. * Update container.py * Update test_query_vector_similarity.py --------- Co-authored-by: Anna Tisch <[email protected]>
* deprecate azure_germany * update * update * Update sdk/identity/azure-identity/azure/identity/_constants.py Co-authored-by: Paul Van Eck <[email protected]> * update --------- Co-authored-by: Paul Van Eck <[email protected]>
* Add default impl to handle token challenges * update version * update * update * update * update * Update sdk/core/azure-core/azure/core/pipeline/policies/_utils.py Co-authored-by: Paul Van Eck <[email protected]> * Update sdk/core/azure-core/azure/core/pipeline/policies/_utils.py Co-authored-by: Paul Van Eck <[email protected]> * update * Update sdk/core/azure-core/tests/test_utils.py Co-authored-by: Paul Van Eck <[email protected]> * Update sdk/core/azure-core/azure/core/pipeline/policies/_utils.py Co-authored-by: Paul Van Eck <[email protected]> * update --------- Co-authored-by: Paul Van Eck <[email protected]>
…Evaluators (Azure#37707) * Make Credentials Required for Content Safety Evaluators * fix a typo * lint, fix content safety evaluator * revert test change * remove credential from rai_service
* Add getFeedRanges API * Add feedRange support in query changeFeed Co-authored-by: annie-mac <[email protected]>
…ure#37669) * during mindependency runs, dev_requirements on local relative paths are now checked for conflict with the targeted set of minimum dependencies * multiple type clarifications within azure-sdk-tools * added tests for new conflict resolution logic --------- Co-authored-by: McCoy Patiño <[email protected]>
Co-authored-by: Wes Haggard <[email protected]>
needuv
reviewed
Oct 25, 2024
...ai-evaluation/azure/ai/evaluation/_evaluators/_multimodal/_content_safety_multimodal_base.py
Show resolved
Hide resolved
needuv
reviewed
Oct 25, 2024
...ai-evaluation/azure/ai/evaluation/_evaluators/_multimodal/_content_safety_multimodal_base.py
Outdated
Show resolved
Hide resolved
needuv
reviewed
Oct 25, 2024
...zure-ai-evaluation/azure/ai/evaluation/_evaluators/_multimodal/_content_safety_multimodal.py
Outdated
Show resolved
Hide resolved
w-javed
changed the title
nagkumar91
reviewed
Oct 28, 2024
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/utils.py
Outdated
Show resolved
Hide resolved
nagkumar91
reviewed
Oct 28, 2024
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/utils.py
Outdated
Show resolved
Hide resolved
nagkumar91
reviewed
Oct 28, 2024
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/rai_service.py
Outdated
Show resolved
Hide resolved
nagkumar91
reviewed
Oct 28, 2024
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/rai_service.py
Outdated
Show resolved
Hide resolved
nagkumar91
reviewed
Oct 28, 2024
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluate/_utils.py
Outdated
Show resolved
Hide resolved
nagkumar91
reviewed
Oct 28, 2024
.../azure-ai-evaluation/azure/ai/evaluation/_evaluators/_content_safety/_content_safety_chat.py
Show resolved
Hide resolved
nagkumar91
reviewed
Oct 28, 2024
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/simulator/_adversarial_scenario.py
Outdated
Show resolved
Hide resolved
needuv
approved these changes
Oct 28, 2024
l0lawrence
pushed a commit
to l0lawrence/azure-sdk-for-python
that referenced
this pull request
Feb 19, 2025
* Initial-Commit-multimodal * Fix * Sync eng/common directory with azure-sdk-tools for PR 9092 (Azure#37713) * Export the subscription data from the service connection * Update deploy-test-resources.yml --------- Co-authored-by: Wes Haggard <[email protected]> Co-authored-by: Wes Haggard <[email protected]> * Removing private parameter from __call__ of AdversarialSimulator (Azure#37709) * Update task_query_response.prompty remove required keys * Update task_simulate.prompty * Update task_query_response.prompty * Update task_simulate.prompty * Remove private variable and use kwargs * Add experimental tag to adv sim --------- Co-authored-by: Nagkumar Arkalgud <[email protected]> * Enabling option to disable response payload on writes (Azure#37365) * Initial draft * Adding tests * Renaming parameter * Update container.py * Renaming test file * Fixing LINT issues * Update container.py * Update _base.py * Update _base.py * Fixing tests * Fixing tests * Adding support to disable response payload on write for AIO * Update CHANGELOG.md * Update _cosmos_client.py * Reacting to code review comments * Addressing code review feedback * Addressed CR feedback * Fixing pyLint errors * Fixing pylint errors * Update test_crud.py * Fixing svc regression * Update sdk/cosmos/azure-cosmos/azure/cosmos/aio/_container.py Co-authored-by: Anna Tisch <[email protected]> * Reacting to code review feedback. * Update container.py * Update test_query_vector_similarity.py --------- Co-authored-by: Anna Tisch <[email protected]> * deprecate azure_germany (Azure#37654) * deprecate azure_germany * update * update * Update sdk/identity/azure-identity/azure/identity/_constants.py Co-authored-by: Paul Van Eck <[email protected]> * update --------- Co-authored-by: Paul Van Eck <[email protected]> * Add default impl to handle token challenges (Azure#37652) * Add default impl to handle token challenges * update version * update * update * update * update * Update sdk/core/azure-core/azure/core/pipeline/policies/_utils.py Co-authored-by: Paul Van Eck <[email protected]> * Update sdk/core/azure-core/azure/core/pipeline/policies/_utils.py Co-authored-by: Paul Van Eck <[email protected]> * update * Update sdk/core/azure-core/tests/test_utils.py Co-authored-by: Paul Van Eck <[email protected]> * Update sdk/core/azure-core/azure/core/pipeline/policies/_utils.py Co-authored-by: Paul Van Eck <[email protected]> * update --------- Co-authored-by: Paul Van Eck <[email protected]> * Make Credentials Required for Content Safety and Protected Materials Evaluators (Azure#37707) * Make Credentials Required for Content Safety Evaluators * fix a typo * lint, fix content safety evaluator * revert test change * remove credential from rai_service * addFeedRangesAndUseFeedRangeInQueryChangeFeed (Azure#37687) * Add getFeedRanges API * Add feedRange support in query changeFeed Co-authored-by: annie-mac <[email protected]> * Update release date for core (Azure#37723) * Improvements to mindependency dev_requirement conflict resolution (Azure#37669) * during mindependency runs, dev_requirements on local relative paths are now checked for conflict with the targeted set of minimum dependencies * multiple type clarifications within azure-sdk-tools * added tests for new conflict resolution logic --------- Co-authored-by: McCoy Patiño <[email protected]> * Need to add environment to subscription configuration (Azure#37726) Co-authored-by: Wes Haggard <[email protected]> * Enable samples for formrecognizer (Azure#37676) * multi-modal-changes * fixes * Fix with latest * dict-fix * adding-protected-material * adding-protected-material * adding-protected-material * bumping-version * adding assets * Added image in simulator * Added image in simulator * bumping-version * push-asset * assets * pushing asset * remove-containt-on-key * asset * asset2 * asset3 * asset4 * adding conftest * conftest * cred fix * asset-new * fix * asset * adding multi-modal-without-tests * asset-from-main * asset-from-main * fix * adding one test only * new asset * tests,fix: Sanitizer should replace with enum value not enum name * test-asset * [AutoRelease] t2-containerservicefleet-2024-09-24-42036(can only be merged by SDK owner) (Azure#37538) * code and test * Update CHANGELOG.md * update-testcase --------- Co-authored-by: azure-sdk <PythonSdkPipelines> Co-authored-by: ChenxiJiang333 <[email protected]> Co-authored-by: ChenxiJiang333 <[email protected]> * [AutoRelease] t2-dns-2024-09-25-81486(can only be merged by SDK owner) (Azure#37560) * code and test * update-testcase * Update CHANGELOG.md * Update test_mgmt_dns_test.py --------- Co-authored-by: azure-sdk <PythonSdkPipelines> Co-authored-by: ChenxiJiang333 <[email protected]> Co-authored-by: ChenxiJiang333 <[email protected]> * [AutoRelease] t2-appconfiguration-2024-10-09-68726(can only be merged by SDK owner) (Azure#37800) * code and test * update-testcase * Update pyproject.toml --------- Co-authored-by: azure-sdk <PythonSdkPipelines> Co-authored-by: ChenxiJiang333 <[email protected]> Co-authored-by: Yuchao Yan <[email protected]> * code and test (Azure#37855) Co-authored-by: azure-sdk <PythonSdkPipelines> * [AutoRelease] t2-servicefabricmanagedclusters-2024-10-08-57405(can only be merged by SDK owner) (Azure#37768) * code and test * update-testcase * update-testcases --------- Co-authored-by: azure-sdk <PythonSdkPipelines> Co-authored-by: ChenxiJiang333 <[email protected]> * [AutoRelease] t2-containerinstance-2024-10-21-66631(can only be merged by SDK owner) (Azure#38005) * code and test * update-testcase * Update CHANGELOG.md * Update CHANGELOG.md * Update CHANGELOG.md --------- Co-authored-by: azure-sdk <PythonSdkPipelines> Co-authored-by: ChenxiJiang333 <[email protected]> Co-authored-by: ChenxiJiang333 <[email protected]> * [sdk generation pipeline] bump typespec-python 0.36.1 (Azure#38008) * update version * update package.json * [AutoRelease] t2-dnsresolver-2024-10-12-16936(can only be merged by SDK owner) (Azure#37864) * code and test * update-testcase * Update CHANGELOG.md * Update CHANGELOG.md --------- Co-authored-by: azure-sdk <PythonSdkPipelines> Co-authored-by: ChenxiJiang333 <[email protected]> Co-authored-by: ChenxiJiang333 <[email protected]> Co-authored-by: Yuchao Yan <[email protected]> * new asset after fix in conftest * asset * chore: Update assets.json * Move perf pipelines to TME subscription (Azure#38020) Co-authored-by: Wes Haggard <[email protected]> * fix * after-comments * fix * asset * new asset with 1 test recording only * chore: Update assets.json * conftest fix * assets change * new test * few changes * removing proxy start * added all tests * asset * fixes * fixes with asset * asset-after-tax * enabling 2 more tests * unit test fix * asset * new asset * fixes per comments * changes by black * merge fix * pylint fix * pylint fix * ground test fix * fixes - pylint, black, mypy * more tests * docstring fixes * doc string fix * asset * few updates after Nagkumar review --------- Co-authored-by: Azure SDK Bot <[email protected]> Co-authored-by: Wes Haggard <[email protected]> Co-authored-by: Wes Haggard <[email protected]> Co-authored-by: Nagkumar Arkalgud <[email protected]> Co-authored-by: Nagkumar Arkalgud <[email protected]> Co-authored-by: Fabian Meiswinkel <[email protected]> Co-authored-by: Anna Tisch <[email protected]> Co-authored-by: Xiang Yan <[email protected]> Co-authored-by: Paul Van Eck <[email protected]> Co-authored-by: Neehar Duvvuri <[email protected]> Co-authored-by: Annie Liang <[email protected]> Co-authored-by: annie-mac <[email protected]> Co-authored-by: Scott Beddall <[email protected]> Co-authored-by: McCoy Patiño <[email protected]> Co-authored-by: kdestin <[email protected]> Co-authored-by: ChenxiJiang333 <[email protected]> Co-authored-by: ChenxiJiang333 <[email protected]> Co-authored-by: Yuchao Yan <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We are adding new set of evaluators under Content Safety for multi modal use cases supporting images along with text.