feat:接入dify #1664

kai2321 · 2025-01-12T13:12:48Z

Ⅰ. Describe what this PR did

Support ai-proxy ai

Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)?

Ⅳ. Describe how to verify it

docker-compose.yaml

version: '3.7'
services:
  envoy:
    image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/gateway:v1.4.0-rc.1
    entrypoint: /usr/local/bin/envoy
    # 开启了 debug 级别日志方便调试
    command: -c /etc/envoy/envoy.yaml --component-log-level wasm:debug
    networks:
      - higress-net
    ports:
      - "10000:10000"
    volumes:
      - ./envoy.yaml:/etc/envoy/envoy.yaml
      - ./plugin.wasm:/etc/envoy/plugin.wasm
networks:
  higress-net: {}

envoy.yaml

# File generated by hgctl. Modify as required.

admin:
  address:
    socket_address:
      protocol: TCP
      address: 0.0.0.0
      port_value: 9901
static_resources:
  listeners:
    - name: listener_0
      address:
        socket_address:
          protocol: TCP
          address: 0.0.0.0
          port_value: 10000
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                scheme_header_transformation:
                  scheme_to_overwrite: https
                stat_prefix: ingress_http
                # Output envoy logs to stdout
                access_log:
                  - name: envoy.access_loggers.stdout
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
                # Modify as required
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: local_service
                      domains: [ "*" ]
                      routes:
                        - match:
                            prefix: "/"
                          route:
                            cluster: dify
                            timeout: 300s
                http_filters:
                  - name: wasmtest
                    typed_config:
                      "@type": type.googleapis.com/udpa.type.v1.TypedStruct
                      type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
                      value:
                        config:
                          name: wasmtest
                          vm_config:
                            runtime: envoy.wasm.runtime.v8
                            code:
                              local:
                                filename: /etc/envoy/plugin.wasm
                          configuration:
                            "@type": "type.googleapis.com/google.protobuf.StringValue"
                            value: |
                              {
                                "provider": {
                                  "type": "dify",                                
                                  "apiTokens": [
                                    "your-api-token"
                                  ],
                                  "modelMapping": {
                                     "*": "dify"
                                  }
                                }
                              }
                  - name: envoy.filters.http.router
  clusters:
    - name: dify
      connect_timeout: 30s
      type: LOGICAL_DNS
      dns_lookup_family: V4_ONLY
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: dify
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: api.dify.ai
                      port_value: 443
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
          "sni": "api.dify.ai"

测试非流式请求：

curl -X POST 'http: //localhost:10000/v1/chat/completions' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gpt-4-turbo",
    "messages": [
        {
            "role": "user",
            "content": "你好，你是谁？"
        }
    ],
    "stream": false
}'

响应：

{
    "id": "7c2310d0-5fd4-46d4-9406-7b837c7f884e",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "你好！我是ChatGPT，一个由OpenAI开发的人工智能语言模型。很高兴能为你提供帮助和回答问题。"
            },
            "finish_reason": "stop"
        }
    ],
    "created": 1736687310,
    "model": "dify",
    "object": "chat.completion",
    "usage": {
        "prompt_tokens": 16,
        "completion_tokens": 296,
        "total_tokens": 312
    }
}

测试流式请求：

curl -X POST 'http: //localhost:10000/v1/chat/completions' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gpt-4-turbo",
    "messages": [
        {
            "role": "user",
            "content": "你好，你是谁？"
        }
    ],
    "stream": true
}'

响应：

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"你好"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"！"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"我是"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"你的"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"智能"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"助手"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"，很"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"高"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"兴"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"为"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"你"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"服务"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"。"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"有什么"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"我"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"可以"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"帮"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"忙"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"的吗"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":"？"}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":""}}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

data: {"id":"fe64c992-bd85-4eb7-84ea-0e91a9e4042d","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":"stop"}],"created":1736687474,"model":"dify","object":"chat.completion.chunk","usage":{}}

Ⅴ. Special notes for reviews

CLAassistant · 2025-01-12T13:12:54Z

All committers have signed the CLA.

CH3CHO

考虑到 dify 也可以私有化部署，是否可以同步支持一下？

kai2321 · 2025-01-13T12:23:26Z

@CH3CHO 支持私有化部署我的理解是加一个支持本地域名的配置项，还是有其他需要更改的呢

CH3CHO · 2025-01-13T12:25:30Z

应该是的，你可以测一下

…

---- 回复的原邮件 ---- | 发件人 | ***@***.***> | | 发送日期 | 2025年01月13日 20:23 | | 收件人 | alibaba/higress ***@***.***> | | 抄送人 | Kent ***@***.***>, Mention ***@***.***> | | 主题 | Re: [alibaba/higress] feat:接入dify (PR #1664) | @CH3CHO 支持私有化部署我的理解是加一个支持本地域名的配置项，还是有其他需要更改的呢 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

codecov-commenter · 2025-01-14T05:46:34Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 43.61%. Comparing base (ef31e09) to head (5abb91c).
Report is 278 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1664      +/-   ##
==========================================
+ Coverage   35.91%   43.61%   +7.70%     
==========================================
  Files          69       76       +7     
  Lines       11576    12358     +782     
==========================================
+ Hits         4157     5390    +1233     
+ Misses       7104     6630     -474     
- Partials      315      338      +23

see 70 files with indirect coverage changes

johnlanni

LGTM, Thanks!

kai2321 requested review from cr7258, CH3CHO and rinfx as code owners January 12, 2025 13:12

CH3CHO requested changes Jan 13, 2025

View reviewed changes

kai2321 added 2 commits January 20, 2025 18:09

feat:接入dify

419776f

增加支持本地部署url, 以及agent、workflow、chat、completion四种模式

5abb91c

kai2321 force-pushed the feat-dify branch from 37fc8ed to 5abb91c Compare January 20, 2025 14:09

johnlanni approved these changes Jan 21, 2025

View reviewed changes

johnlanni merged commit ce94c6e into alibaba:main Jan 21, 2025
11 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat:接入dify #1664

feat:接入dify #1664

kai2321 commented Jan 12, 2025 •

edited

Loading

CLAassistant commented Jan 12, 2025 •

edited

Loading

CH3CHO left a comment

kai2321 commented Jan 13, 2025

CH3CHO commented Jan 13, 2025 via email

codecov-commenter commented Jan 14, 2025 •

edited

Loading

johnlanni left a comment

feat:接入dify #1664

feat:接入dify #1664

Conversation

kai2321 commented Jan 12, 2025 • edited Loading

Ⅰ. Describe what this PR did

Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)?

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

CLAassistant commented Jan 12, 2025 • edited Loading

CH3CHO left a comment

Choose a reason for hiding this comment

kai2321 commented Jan 13, 2025

CH3CHO commented Jan 13, 2025 via email

codecov-commenter commented Jan 14, 2025 • edited Loading

Codecov Report

johnlanni left a comment

Choose a reason for hiding this comment

kai2321 commented Jan 12, 2025 •

edited

Loading

CLAassistant commented Jan 12, 2025 •

edited

Loading

codecov-commenter commented Jan 14, 2025 •

edited

Loading