Skip to content

Commit 6fdbeb5

Browse files
sabrennersabrenner
authored
chore(llmobs): re-enable langchain llmobs tests (#13181)
Re-enables langchain LLMObs tests entirely. This required a small refactor and cleanup of fixtures, and some new cassettes. Will be using this PR to check for flakiness. ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
1 parent a05cdab commit 6fdbeb5

14 files changed

+2431
-176
lines changed

‎tests/contrib/langchain/cassettes/anthropic_chat_completion_sync.yaml

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
interactions:
2+
- request:
3+
body: '{"max_tokens": 15, "messages": [{"role": "user", "content": "When do you
4+
use ''whom'' instead of ''who''?"}], "model": "claude-3-opus-20240229", "temperature":
5+
0.0}'
6+
headers:
7+
accept:
8+
- application/json
9+
accept-encoding:
10+
- gzip, deflate
11+
anthropic-version:
12+
- '2023-06-01'
13+
connection:
14+
- keep-alive
15+
content-length:
16+
- '160'
17+
content-type:
18+
- application/json
19+
host:
20+
- api.anthropic.com
21+
user-agent:
22+
- Anthropic/Python 0.26.0
23+
x-stainless-arch:
24+
- arm64
25+
x-stainless-async:
26+
- 'false'
27+
x-stainless-lang:
28+
- python
29+
x-stainless-os:
30+
- MacOS
31+
x-stainless-package-version:
32+
- 0.26.0
33+
x-stainless-runtime:
34+
- CPython
35+
x-stainless-runtime-version:
36+
- 3.10.13
37+
method: POST
38+
uri: https://api.anthropic.com/v1/messages
39+
response:
40+
body:
41+
string: !!binary |
42+
H4sIAAAAAAAAA2SPTU/DMAyG/0rkcyp14UOQ8wQnLlyYxFCVpd4aaOMQOzBW9b+jTlQCcbL0Po9f
43+
2SOEFiwMfGjq1eYx3/PNPiW+jp9+c3daPxxPa9AgXwlnC5ndAUFDpn4OHHNgcVFAw0At9mDB9660
44+
WF1UlApXpjaXtTG3oMFTFIwC9nlcCgWP8+p5WNjCU0fDFlRgVRhb5VhJh4p2r+hF0V459YF5pyir
45+
lDERBwkUYXrRwEKpyeiY4nyoOzZCbxgZfhDje8HoEWwsfa+hnB+xI4SYiiyyNUaDd77Dxmd0c3vz
46+
V6gXntG1/xkV+Z2srqbpGwAA//8DAEUIphliAQAA
47+
headers:
48+
CF-RAY:
49+
- 92eaa288e8b55728-IAD
50+
Connection:
51+
- keep-alive
52+
Content-Encoding:
53+
- gzip
54+
Content-Type:
55+
- application/json
56+
Date:
57+
- Fri, 11 Apr 2025 12:54:51 GMT
58+
Server:
59+
- cloudflare
60+
Transfer-Encoding:
61+
- chunked
62+
X-Robots-Tag:
63+
- none
64+
anthropic-organization-id:
65+
- 4257e925-ee99-4ee8-9c62-8e53716d5203
66+
anthropic-ratelimit-input-tokens-limit:
67+
- '400000'
68+
anthropic-ratelimit-input-tokens-remaining:
69+
- '400000'
70+
anthropic-ratelimit-input-tokens-reset:
71+
- '2025-04-11T12:54:51Z'
72+
anthropic-ratelimit-output-tokens-limit:
73+
- '80000'
74+
anthropic-ratelimit-output-tokens-remaining:
75+
- '80000'
76+
anthropic-ratelimit-output-tokens-reset:
77+
- '2025-04-11T12:54:51Z'
78+
anthropic-ratelimit-requests-limit:
79+
- '4000'
80+
anthropic-ratelimit-requests-remaining:
81+
- '3999'
82+
anthropic-ratelimit-requests-reset:
83+
- '2025-04-11T12:54:51Z'
84+
anthropic-ratelimit-tokens-limit:
85+
- '480000'
86+
anthropic-ratelimit-tokens-remaining:
87+
- '480000'
88+
anthropic-ratelimit-tokens-reset:
89+
- '2025-04-11T12:54:51Z'
90+
cf-cache-status:
91+
- DYNAMIC
92+
request-id:
93+
- req_013K59Ezk31wKPP7sKpyLGZP
94+
via:
95+
- 1.1 google
96+
status:
97+
code: 200
98+
message: OK
99+
version: 1

‎tests/contrib/langchain/cassettes/bedrock_amazon_chat_invoke.yaml

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
interactions:
2+
- request:
3+
body: '{"inputText": "\n\nUser: summarize the plot to the lord of the rings in
4+
a dozen words\n\nBot:", "textGenerationConfig": {"maxTokenCount": 50, "temperature":
5+
0}}'
6+
headers:
7+
Accept:
8+
- !!binary |
9+
YXBwbGljYXRpb24vanNvbg==
10+
Content-Length:
11+
- '160'
12+
Content-Type:
13+
- !!binary |
14+
YXBwbGljYXRpb24vanNvbg==
15+
User-Agent:
16+
- !!binary |
17+
Qm90bzMvMS4zNC41MSBtZC9Cb3RvY29yZSMxLjM0LjUxIHVhLzIuMCBvcy9tYWNvcyMyMy42LjAg
18+
bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjEwLjEzIG1kL3B5aW1wbCNDUHl0aG9uIGNmZy9y
19+
ZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjM0LjUx
20+
X-Amz-Date:
21+
- !!binary |
22+
MjAyNTA0MTFUMTQxMDIwWg==
23+
X-Amz-Security-Token:
24+
- !!binary |
25+
SVFvSmIzSnBaMmx1WDJWakVFY2FDWFZ6TFdWaGMzUXRNU0pHTUVRQ0lFVUtlU1lHVHd4b2JrVEk0
26+
VnNUUm5Wa25XVGd6b1VvOUJVM1hJM0owdXRBQWlCNmgyaDRCT2VJK3lpb3gzcFZ5aE9TTVFDR2M0
27+
MGFoUEVSS2tqK2tsTVlKaXFaQXdpLy8vLy8vLy8vLy84QkVBTWFERFl3TVRReU56STNPVGs1TUNJ
28+
TXNwTEI5T1dwRitBQUdhcDFLdTBDUmR4dWpidmsxaFpOMllmcXliSno2TE5ON3UzRGFWbWZzZU1t
29+
eEZEK1kyUTlleTJTTEpUU00xTUJ2MjRreW04Snd2aU0yK0dwMmdFMWJ2SkJQMm8wK2Y5U0l3anln
30+
K0xZQ2FIejNsNmt2bzQzTlZkZWJ3WUVoYm9mWU1TM3dwN1VEOElnemhianpJN2JIRGQwT2hBWDZD
31+
Ym9QSjVWRU5wQzZpL3pLVE1KLzJHQmhrNXpUdmtOMFZYRkhqYWh2TUN6bUpKR2VOYVUvRDY4aTVV
32+
L3R6dVJwTDVKV0V4dzNZajR4bk01bkR1Y3ZqL25PV3VzaExuMzdKbStYVzkvc0RXTy9SeGhGUjVx
33+
UG5RdUIvdkZWcEZGSkRwbjEzZTRKSGNnbHR4Zng5ejdVWFk5aENCOVIxZWJCRkpuMDBlN29wYlhE
34+
V3RFbnVMWjV4QUEwdFh6dEYrRHpGbExXd0VRVmRJbmZJYmdLcXlUMEZGVWNCNlFlQU5IZTlrTWM1
35+
NEdOV0cxeHVueGxUY2xwNXZLOWtoN1lGYzh5citqWk5paWZUdEgvRVJvODY5Y2ltT3dTU0NRWndN
36+
OUlYeGd0amZNZ1N3dGQxL053T0k0bjRtSklvU1NCcWFseFdqUVNabGVLTm40cFZ2TjkxQXd0TVRr
37+
dndZNnB3RXMybnNMMmhlbHBMTDJRQzU1UmdlMno3aFhiTzhNS2hmUmRJMWdKL3NVZUdiQmhqckJp
38+
bUFBSkdqN2MrcWpJR3lxQTI4WHZqRUE3Vm11U1ZVM1hDYm0xa3lWZGlqaXpyN2l1dVlxVUlnSFRE
39+
ZTU1Tm14QVM5eVF2eXI3dHEvelNjdGNTTjNRekJmK3I3MkZWUU1nUDhNR1dmY0s0Z2ZnUnFkN2E5
40+
TzhnQnBvcTZ5NzV3NWFZMVh6UWV0dkpQcVd0M3dLRTBjazVqMi9yd0xTd3VyM1lFUkVBUmlINXk5
41+
akE9PQ==
42+
amz-sdk-invocation-id:
43+
- !!binary |
44+
MTcyYmNlNWUtYWMwNS00MDZmLTlmOTUtZDhlYjMxNTFlODIx
45+
amz-sdk-request:
46+
- !!binary |
47+
YXR0ZW1wdD0x
48+
method: POST
49+
uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/amazon.titan-tg1-large/invoke
50+
response:
51+
body:
52+
string: '{"inputTextTokenCount":19,"results":[{"tokenCount":26,"outputText":"\nThe
53+
Lord of the Rings follows a hobbit on a quest to destroy a ring that belongs
54+
to an evil dark lord.","completionReason":"FINISH"}]}'
55+
headers:
56+
Connection:
57+
- keep-alive
58+
Content-Length:
59+
- '204'
60+
Content-Type:
61+
- application/json
62+
Date:
63+
- Fri, 11 Apr 2025 14:10:22 GMT
64+
X-Amzn-Bedrock-Input-Token-Count:
65+
- '19'
66+
X-Amzn-Bedrock-Invocation-Latency:
67+
- '1306'
68+
X-Amzn-Bedrock-Output-Token-Count:
69+
- '26'
70+
x-amzn-RequestId:
71+
- f90ce49e-62d8-4821-b519-d73aca0df34b
72+
status:
73+
code: 200
74+
message: OK
75+
version: 1

‎tests/contrib/langchain/cassettes/bedrock_amazon_invoke.yaml

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
interactions:
2+
- request:
3+
body: '{"inputText": "can you explain what Datadog is to someone not in the tech
4+
industry?", "textGenerationConfig": {"temperature": 0, "topP": 0.9, "stopSequences":
5+
[], "maxTokenCount": 50}}'
6+
headers:
7+
Accept:
8+
- !!binary |
9+
YXBwbGljYXRpb24vanNvbg==
10+
Content-Length:
11+
- '184'
12+
Content-Type:
13+
- !!binary |
14+
YXBwbGljYXRpb24vanNvbg==
15+
User-Agent:
16+
- !!binary |
17+
Qm90bzMvMS4zNC41MSBtZC9Cb3RvY29yZSMxLjM0LjUxIHVhLzIuMCBvcy9tYWNvcyMyMy42LjAg
18+
bWQvYXJjaCNhcm02NCBsYW5nL3B5dGhvbiMzLjEwLjEzIG1kL3B5aW1wbCNDUHl0aG9uIGNmZy9y
19+
ZXRyeS1tb2RlI2xlZ2FjeSBCb3RvY29yZS8xLjM0LjUx
20+
X-Amz-Date:
21+
- !!binary |
22+
MjAyNTA0MTFUMTQxMjM4Wg==
23+
X-Amz-Security-Token:
24+
- !!binary |
25+
SVFvSmIzSnBaMmx1WDJWakVFY2FDWFZ6TFdWaGMzUXRNU0pHTUVRQ0lFVUtlU1lHVHd4b2JrVEk0
26+
VnNUUm5Wa25XVGd6b1VvOUJVM1hJM0owdXRBQWlCNmgyaDRCT2VJK3lpb3gzcFZ5aE9TTVFDR2M0
27+
MGFoUEVSS2tqK2tsTVlKaXFaQXdpLy8vLy8vLy8vLy84QkVBTWFERFl3TVRReU56STNPVGs1TUNJ
28+
TXNwTEI5T1dwRitBQUdhcDFLdTBDUmR4dWpidmsxaFpOMllmcXliSno2TE5ON3UzRGFWbWZzZU1t
29+
eEZEK1kyUTlleTJTTEpUU00xTUJ2MjRreW04Snd2aU0yK0dwMmdFMWJ2SkJQMm8wK2Y5U0l3anln
30+
K0xZQ2FIejNsNmt2bzQzTlZkZWJ3WUVoYm9mWU1TM3dwN1VEOElnemhianpJN2JIRGQwT2hBWDZD
31+
Ym9QSjVWRU5wQzZpL3pLVE1KLzJHQmhrNXpUdmtOMFZYRkhqYWh2TUN6bUpKR2VOYVUvRDY4aTVV
32+
L3R6dVJwTDVKV0V4dzNZajR4bk01bkR1Y3ZqL25PV3VzaExuMzdKbStYVzkvc0RXTy9SeGhGUjVx
33+
UG5RdUIvdkZWcEZGSkRwbjEzZTRKSGNnbHR4Zng5ejdVWFk5aENCOVIxZWJCRkpuMDBlN29wYlhE
34+
V3RFbnVMWjV4QUEwdFh6dEYrRHpGbExXd0VRVmRJbmZJYmdLcXlUMEZGVWNCNlFlQU5IZTlrTWM1
35+
NEdOV0cxeHVueGxUY2xwNXZLOWtoN1lGYzh5citqWk5paWZUdEgvRVJvODY5Y2ltT3dTU0NRWndN
36+
OUlYeGd0amZNZ1N3dGQxL053T0k0bjRtSklvU1NCcWFseFdqUVNabGVLTm40cFZ2TjkxQXd0TVRr
37+
dndZNnB3RXMybnNMMmhlbHBMTDJRQzU1UmdlMno3aFhiTzhNS2hmUmRJMWdKL3NVZUdiQmhqckJp
38+
bUFBSkdqN2MrcWpJR3lxQTI4WHZqRUE3Vm11U1ZVM1hDYm0xa3lWZGlqaXpyN2l1dVlxVUlnSFRE
39+
ZTU1Tm14QVM5eVF2eXI3dHEvelNjdGNTTjNRekJmK3I3MkZWUU1nUDhNR1dmY0s0Z2ZnUnFkN2E5
40+
TzhnQnBvcTZ5NzV3NWFZMVh6UWV0dkpQcVd0M3dLRTBjazVqMi9yd0xTd3VyM1lFUkVBUmlINXk5
41+
akE9PQ==
42+
amz-sdk-invocation-id:
43+
- !!binary |
44+
ZWMyMWZjN2UtMmU0YS00ZjZmLTk2ZTktMjllYzMwNjIzZTJj
45+
amz-sdk-request:
46+
- !!binary |
47+
YXR0ZW1wdD0x
48+
method: POST
49+
uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/amazon.titan-tg1-large/invoke
50+
response:
51+
body:
52+
string: '{"inputTextTokenCount":16,"results":[{"tokenCount":48,"outputText":"\nDatadog
53+
is a monitoring and analytics platform for IT, DevOps, and business teams.
54+
It collects and analyzes massive amounts of data in real time, providing insights
55+
into the performance and health of applications, infrastructure, and services.","completionReason":"FINISH"}]}'
56+
headers:
57+
Connection:
58+
- keep-alive
59+
Content-Length:
60+
- '345'
61+
Content-Type:
62+
- application/json
63+
Date:
64+
- Fri, 11 Apr 2025 14:12:41 GMT
65+
X-Amzn-Bedrock-Input-Token-Count:
66+
- '16'
67+
X-Amzn-Bedrock-Invocation-Latency:
68+
- '2126'
69+
X-Amzn-Bedrock-Output-Token-Count:
70+
- '48'
71+
x-amzn-RequestId:
72+
- dd3e068e-b7ff-4aba-b2c0-53215c361b52
73+
status:
74+
code: 200
75+
message: OK
76+
version: 1

0 commit comments

Comments
 (0)