Skip to content

roadmap: Implement Remote engine #1662

Closed
0 of 1 issue completed
Closed
0 of 1 issue completed
@nguyenhoangthuan99

Description

@nguyenhoangthuan99

Problem

This epic is implementation of Remote engine

Success Criteria

  • Cortex supports remote engine for using remote provider (openai, anthropic, ...)
  • Cortex supports engine managements for remote engine
  • Cortex supports model managements for remote model (gpt-4o, claude, ...)
  • Cortex supports API for Chat Completions, Get models and route request to correct remote provider

Tasklist

  • Implement jinja2 parser and chat completion requests route
  • Implement Get models request route
  • Implement Request Transform
  • Implement Response Transform
  • Implement Engine management
    • Engine table in cortex.db
    • Upsert Remote engine (name & variant would be unique as composite key)
    • Get Remote engine by Id
    • Get Remote engine by name and variant (optional)
    • Delete Remote engine
    • Update API /v1/engines to support remote engine
  • Implement model management (cortex.cpp)
    • Add remote model.yml
    • Update Remote model.yml
    • List, Get remote models
    • load, unload model
    • chat completion with non stream request
    • chat completion with stream request
    • Integrate with new engine implementation
  • Try using transform template with Claude model
  • Support CLI for remote engines
  • Docs

Sub-issues

Metadata

Metadata

Assignees

Labels

type: epicA major feature or initiative

Type

No type

Projects

Status

Completed

Relationships

None yet

Development

No branches or pull requests

Issue actions