Azure OpenAI api 端点设置 failover 配置的话健康检查会报 401，不开的话则正常访问

[Colstuwjx]

If you are reporting any crash or any potential security issue, do not
open an issue in this repo. Please report the issue via ASRC(Alibaba Security Response Center) where the issue will be triaged appropriately.

• I have searched the issues of this repository and believe that this is not a duplicate.

Ⅰ. Issue Description

希望能通过 higress 实现 model 的 failover，但是当开启这个配置时，发送请求会直接报 401

Ⅱ. Describe what happened

当针对一个 azure openai 端点开启 healthcheck 配置时，返回 401 报错：

{"error":{"code":"401","message":"Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource."}}

higress gateway 日志：

2025-02-25T14:03:48.939493Z	debug	envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1392	wasm log: [ai-proxy] [nil] Perform health check for unavailable apiTokens:***	thread=33
2025-02-25T14:03:48.939536Z	error	envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1401	wasm log: get request path failed: error status returned by host: bad argument	thread=33
2025-02-25T14:03:48.939871Z	debug	envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1392	wasm log: http call start, id: ca7fd51e-c855-449e-9dff-c7ef79141e36, cluster: outbound|443||llm-gpt-4o-02.internal.dns, method: POST, url: /openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview, body: {
                      "model": "gpt-4o",
                      "messages": [
                        {
                          "role": "user",
                          "content": "who are you?"
                        }
                      ]
                    }, timeout: 10000	thread=33
2025-02-25T14:03:49.336850Z	debug	envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1392	wasm log: http call end, id: ca7fd51e-c855-449e-9dff-c7ef79141e36, code: 401, normal: true, body: {"error":{"code":"401","message":"Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource."}}	thread=33

Ⅲ. Describe what you expected to happen

正常访问，或者当访问的这个 model endpoint 出现 ratelimit 429 等情况时置为不可用，higress 能够代理到其他可用 的 upstream 实例（当 ingress 配置了多个 upstream 时）

Ⅳ. How to reproduce it (as minimally and precisely as possible)

Ⅴ. Anything else we need to know?

Ⅵ. Environment:

• Higress version: v2.0.7
• OS : Kind + macOS local deployment
• Others:

[Colstuwjx]

复现了，这个报错看上去是根本没带 token header ？ azure openai 是需要带上一个 token 来请求的：

# this works
curl -X POST "https://<YOUR_ENDPOINT_DOMAIN_XXX>.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview" \
-H "Content-Type: application/json" \
-H "api-key:<YOUR_API_TOKEN>" \
-d '{
    "messages": [{
        "role": "user",
        "content": "who are you?"
    }]
}'

如果不带 token 去请求一个 azure openai api 地址就会报这个错误：

curl -X POST "https://<YOUR_ENDPOINT_DOMAIN_XXX>.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview" \
-H "Content-Type: application/json" \
-d '{
    "model": "gpt-4o",
    "messages": [{
        "role": "user",
        "content": "who are you?"
    }]
}'

{"error":{"code":"401","message":"Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource."}}

[Colstuwjx]

貌似不只是 azure openai ？我试了下其他供应商 model endpoint ，也是报 Authorization failed. ，去掉 failover 配置就又 OK 了，cc @cr7258

[cr7258]

@Colstuwjx 我没复现出你这个问题。。我加下你微信，明天方便的话可以共享屏幕一起看一下

[Colstuwjx]

@cr7258 在打印 debug 日志后大概知道问题了，貌似 unavailable token 和 model provider host 不匹配导致的，我这边有配置两个 model provider，如 azure-openai-01, azure-openai-02 ，然后分别有 token-01, token-02 ，但是日志里发现 health check 使用的是 azure-openai-01 + token-02 的组合，这肯定就会报 401 了。

只通过 apitoken 来标识一个 model provider 是否不太完备，不同的 model provider，对应的 api host、api path、api token 可能都不一样。

[cr7258]

@Colstuwjx ok，我晚点修复一下这个问题

[Colstuwjx]

补充下，如果只开一个 azure-openai-01 的 failover 的话，会出现时好时坏的情况，当返回 401 时情况是 failover 里的 apitoken 直接为空了：

2025-03-04T06:00:14.041420Z	debug	envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1392	wasm log higress-system.ai-proxy: [ai-proxy] [nil] [1e32726b-b7ba-40ea-8856-45f1988e432f] apiTokens: [], unavailableApiTokens: []	thread=55
2025-03-04T06:00:14.041427Z	debug	envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1392	wasm log higress-system.ai-proxy: [ai-proxy] [nil] [1e32726b-b7ba-40ea-8856-45f1988e432f] Use apiToken  to send request	thread=55