From c87107e4d0ed9d2a7ad0f21d407679da77f05f5d Mon Sep 17 00:00:00 2001 From: Ankit Singhal <30610298+singankit@users.noreply.github.com> Date: Thu, 10 Apr 2025 13:25:06 -0700 Subject: [PATCH 1/5] Update coherence.prompty --- .../ai/evaluation/_evaluators/_coherence/coherence.prompty | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_coherence/coherence.prompty b/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_coherence/coherence.prompty index 716f217ad901..f34dcf4bad95 100644 --- a/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_coherence/coherence.prompty +++ b/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_coherence/coherence.prompty @@ -82,6 +82,8 @@ user: **Query:** Discuss the significance of the Monroe Doctrine in shaping U.S. foreign policy. **Response:** The Monroe Doctrine was a pivotal policy declared in 1823 that asserted U.S. opposition to European colonization in the Americas. By stating that any intervention by external powers in the Western Hemisphere would be viewed as a hostile act, it established the U.S. as a protector of the region. This doctrine shaped U.S. foreign policy by promoting isolation from European conflicts while justifying American influence and expansion in the hemisphere. Its long-term significance lies in its enduring influence on international relations and its role in defining the U.S. position in global affairs. +Note that the QUERY can either be a string with a user request or an entire conversation history including previous requests and responses from the assistant. +In this case, the assistant's response should be evaluated in the context of the entire conversation but the focus should be on the last query and response. # Data QUERY: {{query}} @@ -96,4 +98,4 @@ RESPONSE: {{response}} ## Please provide your answers between the tags: your chain of thoughts, your explanation, your Score. -# Output \ No newline at end of file +# Output From 90a5abafdcab36d156223f29f4f45f6aec44049b Mon Sep 17 00:00:00 2001 From: Ankit Singhal <30610298+singankit@users.noreply.github.com> Date: Thu, 10 Apr 2025 13:25:27 -0700 Subject: [PATCH 2/5] Update fluency.prompty --- .../azure/ai/evaluation/_evaluators/_fluency/fluency.prompty | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_fluency/fluency.prompty b/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_fluency/fluency.prompty index a937994c4ec4..ffb6b9b663cc 100644 --- a/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_fluency/fluency.prompty +++ b/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_fluency/fluency.prompty @@ -70,6 +70,8 @@ user: **Response:** Technology revolutionizes modern education by providing interactive learning platforms, enabling personalized learning experiences, and connecting students worldwide, thereby transforming how knowledge is acquired and shared. +Note that the QUERY can either be a string with a user request or an entire conversation history including previous requests and responses from the assistant. +In this case, the assistant's response should be evaluated in the context of the entire conversation but the focus should be on the last query and response. # Data RESPONSE: {{response}} @@ -83,4 +85,4 @@ RESPONSE: {{response}} ## Please provide your answers between the tags: your chain of thoughts, your explanation, your Score. -# Output \ No newline at end of file +# Output From c6a14e8bfeac65ecf5fcfa8ec30d1dae88fe018d Mon Sep 17 00:00:00 2001 From: Ankit Singhal <30610298+singankit@users.noreply.github.com> Date: Thu, 10 Apr 2025 13:26:00 -0700 Subject: [PATCH 3/5] Update groundedness_with_query.prompty --- .../_evaluators/_groundedness/groundedness_with_query.prompty | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_groundedness/groundedness_with_query.prompty b/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_groundedness/groundedness_with_query.prompty index dfe6bf0d0bac..408dd4b8129d 100644 --- a/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_groundedness/groundedness_with_query.prompty +++ b/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_groundedness/groundedness_with_query.prompty @@ -95,6 +95,8 @@ user: **Query:** By what date must participants register to receive early bird pricing? **Response:** Participants must register by May 31st to receive early bird pricing. +Note that the QUERY can either be a string with a user request or an entire conversation history including previous requests and responses from the assistant. +In this case, the assistant's response should be evaluated in the context of the entire conversation but the focus should be on the last query and response. # Data CONTEXT: {{context}} @@ -110,4 +112,4 @@ RESPONSE: {{response}} ## Please provide your answers between the tags: your chain of thoughts, your explanation, your Score. -# Output \ No newline at end of file +# Output From e33a04b3f7321928b11fc499283c9869d38f9ee0 Mon Sep 17 00:00:00 2001 From: Ankit Singhal <30610298+singankit@users.noreply.github.com> Date: Thu, 10 Apr 2025 13:26:16 -0700 Subject: [PATCH 4/5] Update groundedness_without_query.prompty --- .../_groundedness/groundedness_without_query.prompty | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_groundedness/groundedness_without_query.prompty b/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_groundedness/groundedness_without_query.prompty index d11e42a447ac..83a2486c32d3 100644 --- a/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_groundedness/groundedness_without_query.prompty +++ b/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_groundedness/groundedness_without_query.prompty @@ -82,6 +82,8 @@ user: **Context:** The new smartphone model features a larger display, improved battery life, and an upgraded camera system. **Response:** The new smartphone model features a larger display, improved battery life, and an upgraded camera system. +Note that the QUERY can either be a string with a user request or an entire conversation history including previous requests and responses from the assistant. +In this case, the assistant's response should be evaluated in the context of the entire conversation but the focus should be on the last query and response. # Data CONTEXT: {{context}} @@ -96,4 +98,4 @@ RESPONSE: {{response}} ## Please provide your answers between the tags: your chain of thoughts, your explanation, your Score. -# Output \ No newline at end of file +# Output From 09a78dd0525cd687bd869367e06b5afc8df49e2e Mon Sep 17 00:00:00 2001 From: Ankit Singhal <30610298+singankit@users.noreply.github.com> Date: Thu, 10 Apr 2025 13:26:38 -0700 Subject: [PATCH 5/5] Update relevance.prompty --- .../ai/evaluation/_evaluators/_relevance/relevance.prompty | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_relevance/relevance.prompty b/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_relevance/relevance.prompty index 4000f0ef5a82..6a3d95b1ad83 100644 --- a/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_relevance/relevance.prompty +++ b/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_relevance/relevance.prompty @@ -82,7 +82,8 @@ user: **Query:** What topics will the conference cover? **Response:** The conference will cover renewable energy, climate change, and sustainability practices, bringing together global experts to discuss these critical issues. - +Note that the QUERY can either be a string with a user request or an entire conversation history including previous requests and responses from the assistant. +In this case, the assistant's response should be evaluated in the context of the entire conversation but the focus should be on the last query and response. # Data QUERY: {{query}} @@ -97,4 +98,4 @@ RESPONSE: {{response}} ## Please provide your answers between the tags: your chain of thoughts, your explanation, your Score. -# Output \ No newline at end of file +# Output