|
| 1 | +# LlamaIndex Server |
| 2 | + |
| 3 | +LlamaIndexServer is a Next.js-based application that allows you to quickly launch your [LlamaIndex Workflows](https://ts.llamaindex.ai/docs/llamaindex/modules/agents/workflows) and [Agent Workflows](https://ts.llamaindex.ai/docs/llamaindex/modules/agents/agent_workflow) as an API server with an optional chat UI. It provides a complete environment for running LlamaIndex workflows with both API endpoints and a user interface for interaction. |
| 4 | + |
| 5 | +## Features |
| 6 | + |
| 7 | +- Serving a workflow as a chatbot |
| 8 | +- Built on Next.js for high performance and easy API development |
| 9 | +- Optional built-in chat UI with extendable UI components |
| 10 | +- Prebuilt development code |
| 11 | + |
| 12 | +## Installation |
| 13 | + |
| 14 | +```bash |
| 15 | +npm i @llamaindex/server |
| 16 | +``` |
| 17 | + |
| 18 | +## Quick Start |
| 19 | + |
| 20 | +Create an `index.ts` file and add the following code: |
| 21 | + |
| 22 | +```ts |
| 23 | +import { LlamaIndexServer } from "@llamaindex/server"; |
| 24 | +import { wiki } from "@llamaindex/tools"; // or any other tool |
| 25 | + |
| 26 | +const createWorkflow = () => agent({ tools: [wiki()] }); |
| 27 | + |
| 28 | +new LlamaIndexServer({ |
| 29 | + workflow: createWorkflow, |
| 30 | + uiConfig: { |
| 31 | + appTitle: "LlamaIndex App", |
| 32 | + starterQuestions: ["Who is the first president of the United States?"], |
| 33 | + }, |
| 34 | +}).start(); |
| 35 | +``` |
| 36 | + |
| 37 | +## Running the Server |
| 38 | + |
| 39 | +In the same directory as `index.ts`, run the following command to start the server: |
| 40 | + |
| 41 | +```bash |
| 42 | +tsx index.ts |
| 43 | +``` |
| 44 | + |
| 45 | +The server will start at `http://localhost:3000` |
| 46 | + |
| 47 | +You can also make a request to the server: |
| 48 | + |
| 49 | +```bash |
| 50 | +curl -X POST "http://localhost:3000/api/chat" -H "Content-Type: application/json" -d '{"message": "Who is the first president of the United States?"}' |
| 51 | +``` |
| 52 | + |
| 53 | +## Configuration Options |
| 54 | + |
| 55 | +The `LlamaIndexServer` accepts the following configuration options: |
| 56 | + |
| 57 | +- `workflow`: A callable function that creates a workflow instance for each request |
| 58 | +- `uiConfig`: An object to configure the chat UI containing the following properties: |
| 59 | + - `appTitle`: The title of the application (default: `"LlamaIndex App"`) |
| 60 | + - `starterQuestions`: List of starter questions for the chat UI (default: `[]`) |
| 61 | + - `componentsDir`: The directory for custom UI components rendering events emitted by the workflow. The default is undefined, which does not render custom UI components. |
| 62 | + - `llamaCloudIndexSelector`: Whether to show the LlamaCloud index selector in the chat UI (requires `LLAMA_CLOUD_API_KEY` to be set in the environment variables) (default: `false`) |
| 63 | + |
| 64 | +LlamaIndexServer accepts all the configuration options from Nextjs Custom Server such as `port`, `hostname`, `dev`, etc. |
| 65 | +See all Nextjs Custom Server options [here](https://nextjs.org/docs/app/building-your-application/configuring/custom-server). |
| 66 | + |
| 67 | +## AI-generated UI Components |
| 68 | + |
| 69 | +The LlamaIndex server provides support for rendering workflow events using custom UI components, allowing you to extend and customize the chat interface. |
| 70 | +These components can be auto-generated using an LLM by providing a JSON schema of the workflow event. |
| 71 | + |
| 72 | +### UI Event Schema |
| 73 | + |
| 74 | +To display custom UI components, your workflow needs to emit UI events that have an event type for identification and a data object: |
| 75 | + |
| 76 | +```typescript |
| 77 | +class UIEvent extends WorkflowEvent<{ |
| 78 | + type: "ui_event"; |
| 79 | + data: UIEventData; |
| 80 | +}> {} |
| 81 | +``` |
| 82 | + |
| 83 | +The `data` object can be any JSON object. To enable AI generation of the UI component, you need to provide a schema for that data (here we're using Zod): |
| 84 | + |
| 85 | +```typescript |
| 86 | +const MyEventDataSchema = z |
| 87 | + .object({ |
| 88 | + stage: z |
| 89 | + .enum(["retrieve", "analyze", "answer"]) |
| 90 | + .describe("The current stage the workflow process is in."), |
| 91 | + progress: z |
| 92 | + .number() |
| 93 | + .min(0) |
| 94 | + .max(1) |
| 95 | + .describe("The progress in percent of the current stage"), |
| 96 | + }) |
| 97 | + .describe("WorkflowStageProgress"); |
| 98 | + |
| 99 | +type UIEventData = z.infer<typeof MyEventDataSchema>; |
| 100 | +``` |
| 101 | + |
| 102 | +### Generate UI Components |
| 103 | + |
| 104 | +The `generateEventComponent` function uses an LLM to generate a custom UI component based on the JSON schema of a workflow event. The schema should contain accurate descriptions of each field so that the LLM can generate matching components for your use case. We've done this for you in the example above using the `describe` function from Zod: |
| 105 | + |
| 106 | +```typescript |
| 107 | +import { OpenAI } from "llamaindex"; |
| 108 | +import { generateEventComponent } from "@llamaindex/server"; |
| 109 | +import { MyEventDataSchema } from "./your-workflow"; |
| 110 | + |
| 111 | +// Also works well with Claude 3.5 Sonnet and Google Gemini 2.5 Pro |
| 112 | +const llm = new OpenAI({ model: "gpt-4.1" }); |
| 113 | +const code = generateEventComponent(MyEventDataSchema, llm); |
| 114 | +``` |
| 115 | + |
| 116 | +After generating the code, we need to save it to a file. The file name must match the event type from your workflow (e.g., `ui_event.jsx` for handling events with `ui_event` type): |
| 117 | + |
| 118 | +```ts |
| 119 | +fs.writeFileSync("components/ui_event.jsx", code); |
| 120 | +``` |
| 121 | + |
| 122 | +Feel free to modify the generated code to match your needs. If you're not satisfied with the generated code, we suggest improving the provided JSON schema first or trying another LLM. |
| 123 | + |
| 124 | +> Note that `generateEventComponent` is generating JSX code, but you can also provide a TSX file. |
| 125 | +
|
| 126 | +### Server Setup |
| 127 | + |
| 128 | +To use the generated UI components, you need to initialize the LlamaIndex server with the `componentsDir` that contains your custom UI components: |
| 129 | + |
| 130 | +```ts |
| 131 | +new LlamaIndexServer({ |
| 132 | + workflow: createWorkflow, |
| 133 | + uiConfig: { |
| 134 | + appTitle: "LlamaIndex App", |
| 135 | + componentsDir: "components", |
| 136 | + }, |
| 137 | +}).start(); |
| 138 | +``` |
| 139 | + |
| 140 | +## Default Endpoints and Features |
| 141 | + |
| 142 | +### Chat Endpoint |
| 143 | + |
| 144 | +The server includes a default chat endpoint at `/api/chat` for handling chat interactions. |
| 145 | + |
| 146 | +### Chat UI |
| 147 | + |
| 148 | +The server always provides a chat interface at the root path (`/`) with: |
| 149 | + |
| 150 | +- Configurable starter questions |
| 151 | +- Real-time chat interface |
| 152 | +- API endpoint integration |
| 153 | + |
| 154 | +### Static File Serving |
| 155 | + |
| 156 | +- The server automatically mounts the `data` and `output` folders at `{server_url}{api_prefix}/files/data` (default: `/api/files/data`) and `{server_url}{api_prefix}/files/output` (default: `/api/files/output`) respectively. |
| 157 | +- Your workflows can use both folders to store and access files. By convention, the `data` folder is used for documents that are ingested, and the `output` folder is used for documents generated by the workflow. |
| 158 | + |
| 159 | +## API Reference |
| 160 | + |
| 161 | +- [LlamaIndexServer](https://ts.llamaindex.ai/docs/api/classes/LlamaIndexServer) |
0 commit comments