feat: implement csv upload #96
Conversation
🦋 Changeset detectedLatest commit: 3d3d94c The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
WalkthroughThe recent changes introduce enhancements across various components and functionalities in the codebase. Key updates include the addition of CSV file handling in the chat interface, modifications to URL generation logic, and improvements to data parsing and streaming timeout management. These changes enhance user experience by enabling CSV previews, refining message content processing, and ensuring better resource management in streaming applications. Changes
Sequence Diagram(s) (Beta)sequenceDiagram
participant User
participant ChatInput
participant ChatController
participant StreamHelper
participant LlamaIndexStream
User->>ChatInput: Upload CSV File
ChatInput->>ChatController: Send CSV Data
ChatController->>StreamHelper: Create Stream Timeout
ChatController->>LlamaIndexStream: Start Stream with DataParserOptions
LlamaIndexStream->>ChatController: Stream Data
ChatController->>ChatInput: Display CSV Content
ChatInput->>User: Show CSV Preview
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
templates/types/streaming/nextjs/app/components/ui/chat/chat-input.tsx
Outdated
Show resolved
Hide resolved
templates/types/streaming/nextjs/app/components/ui/chat/chat-input.tsx
Outdated
Show resolved
Hide resolved
| image_url: { | ||
| url: imageUrl, | ||
| console.log({ additionalData }); | ||
| if (additionalData?.imageUrl) { |
There was a problem hiding this comment.
does this also work?
if(!additionalData) return textMessage;
const content = [
{
type: "text",
text: textMessage,
}
];
if (additionalData?.imageUrl) {
content.push(
{
type: "image_url",
image_url: {
url: additionalData?.imageUrl,
},
});
}
if (additionalData?.csvContent) {
content.push({
type: "text",
text: "Use the following CSV data:\n" + "```csv\n" + additionalData.csvContent + "\n```",
});
}
return content;
There was a problem hiding this comment.
It still worked. But we are just changing the payload for llm. Message history in the frontend still doesn't have csv content. So above code won't fix stateless issue.
There was a problem hiding this comment.
but it would allow to send an image together with a CSV, right?
There was a problem hiding this comment.
Ah I got it. Let me update it.
I think we can enhance the UI to support submiting image and csv at the same time (but should be in another PR)
templates/types/streaming/express/src/controllers/chat.controller.ts
Outdated
Show resolved
Hide resolved
|
|
||
| if (additionalData?.csvContent) { | ||
| const csvContent = | ||
| "Use the following CSV data:\n" + |
There was a problem hiding this comment.
as the API is stateless, does this CSV survive multiple requests? e.g. can I ask follow up question on the CSV's content?
There was a problem hiding this comment.
Just tested and it's not survive multiple requests.
The reason is because I am storing csv content inside annotation in the frontend.
And it's not exist inside chat history payload
How about appendding csv content to assistant repsonse instead of appending to annotation ?
There was a problem hiding this comment.
how about converting the CSV in a markdown instead and then inserting the markdown in the chat message?
There was a problem hiding this comment.
just keeping the CSV as markdown in the past messages didn't not work (the LLM was hallucinating).
@thucpn as discussed, we try to send the CSV with every message (if the user selects to do so)
There was a problem hiding this comment.
Yes, I have updated the PR. Now users can decide to use existed csv file or not before they submit the message
thucpn
force-pushed
the
feat/implement-csv-upload
branch
from
2360445 to
2e47af8
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
Outside diff range and nitpick comments (5)
templates/types/streaming/nextjs/app/api/files/[...slug]/route.ts (1)
Line range hint
1-45: Add the 'node:' protocol prefix to Node.js built-in module imports for clarity.- import { readFile } from "fs/promises"; + import { readFile } from "node:fs/promises";templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts (1)
16-20: Ensure proper import usage forappendCsvDataand related functions.Consider using explicit type imports for functions that are only used as types to enhance code clarity and maintainability.
templates/types/streaming/express/src/controllers/chat.controller.ts (1)
[!TIP]
Codebase VerificationThe search results indicate that the
chatfunction is defined intemplates/types/streaming/express/src/controllers/chat.controller.ts:export const chat = async (req: Request, res: Response) => { const chatEngine = await createChatEngine(); // ... rest of the code };However, there is no evidence of error handling within this function, such as
try-catchblocks or other mechanisms to handle potential errors during execution.
- File:
templates/types/streaming/express/src/controllers/chat.controller.ts- Lines: 2-48
Please ensure that error handling is robust in the
chatfunction by adding appropriatetry-catchblocks or other error handling mechanisms.Analysis chain
Line range hint
42-78: Ensure error handling is robust inchatfunction.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Verify that error handling paths are properly tested. rg --type typescript "chat" | grep "catch"Length of output: 75
Script:
#!/bin/bash # Verify that error handling paths are properly tested. rg --type ts "chat" | grep "catch"Length of output: 32
Script:
#!/bin/bash # Locate the definition of the `chat` function and inspect its implementation for error handling. ast-grep --lang typescript --pattern 'function chat($_) { $$$ }'Length of output: 66
Script:
#!/bin/bash # Locate the `chat` method within classes or objects and inspect its implementation for error handling. ast-grep --lang typescript --pattern 'class $_ { $$$ chat($_) { $$$ } $$$ }'Length of output: 78
Script:
#!/bin/bash # Locate the `chat` function in various formats and inspect its implementation for error handling. ast-grep --lang typescript --pattern 'const chat = ($_): $_ => { $$$ }'Length of output: 73
Script:
#!/bin/bash # Search for the `chat` function definition and its usage across the codebase. rg --type ts "chat"Length of output: 6455
templates/types/streaming/nextjs/app/api/chat/route.ts (1)
Line range hint
50-89: Enhance error handling inPOSTfunction.Consider adding more specific error messages and handling potential null values more gracefully to improve the robustness of the API.
templates/types/streaming/nextjs/app/components/ui/chat/chat-input.tsx (1)
Line range hint
30-46: Optimize file handling and state management inChatInput.Consider abstracting file handling logic into separate functions or hooks to reduce complexity and improve reusability.
Also applies to: 62-75
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (15)
- templates/components/engines/typescript/agent/tools/interpreter.ts (1 hunks)
- templates/types/streaming/express/src/controllers/chat.controller.ts (3 hunks)
- templates/types/streaming/express/src/controllers/llamaindex-stream.ts (2 hunks)
- templates/types/streaming/express/src/controllers/stream-helper.ts (1 hunks)
- templates/types/streaming/fastapi/gitignore (1 hunks)
- templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts (2 hunks)
- templates/types/streaming/nextjs/app/api/chat/route.ts (4 hunks)
- templates/types/streaming/nextjs/app/api/chat/stream-helper.ts (1 hunks)
- templates/types/streaming/nextjs/app/api/files/[...slug]/route.ts (1 hunks)
- templates/types/streaming/nextjs/app/components/ui/chat/chat-input.tsx (5 hunks)
- templates/types/streaming/nextjs/app/components/ui/chat/chat-message.tsx (4 hunks)
- templates/types/streaming/nextjs/app/components/ui/chat/csv-content.tsx (1 hunks)
- templates/types/streaming/nextjs/app/components/ui/chat/index.ts (3 hunks)
- templates/types/streaming/nextjs/app/components/ui/lib/url.ts (1 hunks)
- templates/types/streaming/nextjs/app/components/ui/upload-csv-preview.tsx (1 hunks)
Files skipped from review due to trivial changes (1)
- templates/types/streaming/fastapi/gitignore
Additional Context Used
Biome (57)
templates/components/engines/typescript/agent/tools/interpreter.ts (20)
171-171: Expected an identifier, a string literal, a number literal, a private field name, or a computed name but instead found '<<'.
172-172: Expected an identifier, a string literal, a number literal, a private field name, or a computed name but instead found '<<'.
177-177: Expected an identifier, a string literal, a number literal, a private field name, or a computed name but instead found '==='.
178-178: numbers cannot be followed by identifiers directly after
178-178: expected a semicolon to end the class property, but found none
178-178: expected a semicolon to end the class property, but found none
178-178: expected a semicolon to end the class property, but found none
178-178: expected a semicolon to end the class property, but found none
178-178: Expected an identifier, a string literal, a number literal, a private field name, or a computed name but instead found ')'.
184-184: Expected an identifier, a string literal, a number literal, a private field name, or a computed name but instead found '>'.
184-184: numbers cannot be followed by identifiers directly after
184-184: expected a semicolon to end the class property, but found none
184-184: expected a semicolon to end the class property, but found none
184-184: expected a semicolon to end the class property, but found none
184-184: expected a semicolon to end the class property, but found none
184-184: Expected an identifier, a string literal, a number literal, a private field name, or a computed name but instead found ','.
184-184: expected a semicolon to end the class property, but found none
184-184: expected a semicolon to end the class property, but found none
184-184: expected a semicolon to end the class property, but found none
184-184: expected a semicolon to end the class property, but found none
templates/types/streaming/express/src/controllers/chat.controller.ts (5)
31-34: Template literals are preferred over string concatenation.
2-2: Some named imports are only used as types.
3-3: All these imports are only used as types.
4-4: Some named imports are only used as types.
6-6: Some named imports are only used as types.
templates/types/streaming/express/src/controllers/llamaindex-stream.ts (4)
57-57: This variable implicitly has the any type.
2-8: Some named imports are only used as types.
9-14: Some named imports are only used as types.
16-16: All these imports are only used as types.
templates/types/streaming/express/src/controllers/stream-helper.ts (2)
2-2: All these imports are only used as types.
3-9: Some named imports are only used as types.
templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts (4)
57-57: This variable implicitly has the any type.
2-8: Some named imports are only used as types.
9-14: Some named imports are only used as types.
16-16: All these imports are only used as types.
templates/types/streaming/nextjs/app/api/chat/route.ts (5)
38-41: Template literals are preferred over string concatenation.
3-3: Some named imports are only used as types.
4-4: Some named imports are only used as types.
5-5: Some named imports are only used as types.
8-8: Some named imports are only used as types.
templates/types/streaming/nextjs/app/api/chat/stream-helper.ts (2)
2-2: All these imports are only used as types.
3-9: Some named imports are only used as types.
templates/types/streaming/nextjs/app/api/files/[...slug]/route.ts (3)
2-2: A Node.js builtin module should be imported with the node: protocol.
4-4: A Node.js builtin module should be imported with the node: protocol.
3-3: Some named imports are only used as types.
templates/types/streaming/nextjs/app/components/ui/chat/chat-input.tsx (2)
87-87: Unexpected any. Specify a different type.
8-8: All these imports are only used as types.
templates/types/streaming/nextjs/app/components/ui/chat/chat-message.tsx (3)
4-4: All these imports are only used as types.
12-21: Some named imports are only used as types.
104-104: Avoid using the index of an array as key property in an element.
templates/types/streaming/nextjs/app/components/ui/chat/csv-content.tsx (1)
2-2: All these imports are only used as types.
templates/types/streaming/nextjs/app/components/ui/chat/index.ts (1)
2-2: All these imports are only used as types.
templates/types/streaming/nextjs/app/components/ui/lib/url.ts (2)
8-8: Forbidden non-null assertion.
18-18: Forbidden non-null assertion.
templates/types/streaming/nextjs/app/components/ui/upload-csv-preview.tsx (3)
7-14: Alternative text title element cannot be empty
15-15: JSX elements without children should be marked as self-closing. In JSX, it is valid for any element to be self-closing.
16-22: JSX elements without children should be marked as self-closing. In JSX, it is valid for any element to be self-closing.
Additional comments not posted (11)
templates/types/streaming/nextjs/app/components/ui/chat/csv-content.tsx (1)
1-12: LGTM! The component is well-structured for displaying CSV content.templates/types/streaming/nextjs/app/components/ui/chat/index.ts (1)
9-9: The new types and enum additions are appropriate for the described functionality.Also applies to: 20-22, 54-54
templates/types/streaming/express/src/controllers/llamaindex-stream.ts (1)
16-20: The additions and modifications for CSV data handling are correctly implemented.Also applies to: 26-28, 34-34, 42-43, 81-81
templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts (2)
81-81: Ensure consistency in the use ofDataParserOptionsacross different functions.
34-34: Ensureoptsparameter increateParserfunction is properly handled.Verification successful
Ensure
optsparameter increateParserfunction is properly handled.The search results show that the
createParserfunction is called withopts?.parserOptionsin both instances found. This indicates that theoptsparameter is being handled correctly by using optional chaining to accessparserOptions.
templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts: return createParser(response, data, opts?.parserOptions)templates/types/streaming/express/src/controllers/llamaindex-stream.ts: return createParser(response, data, opts?.parserOptions)Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Verify that all calls to `createParser` correctly pass `DataParserOptions`. rg --type typescript "createParser"Length of output: 70
Script:
#!/bin/bash # Verify that all calls to `createParser` correctly pass `DataParserOptions`. rg --type ts "createParser"Length of output: 467
templates/types/streaming/nextjs/app/api/chat/stream-helper.ts (1)
20-28: Duplicate functionality withexpress/src/controllers/stream-helper.ts.Consider abstracting common functionality into a shared library or module to avoid code duplication and ensure consistency.
templates/types/streaming/nextjs/app/components/ui/chat/chat-message.tsx (4)
23-23: EnsureCsvContentis properly utilized in the component.
13-13: LGTM! Good use of structured data types for handling CSV content.
51-54: Properly retrieves CSV data annotations from messages.
81-83: Ensure CSV content is rendered only when available.templates/components/engines/typescript/agent/tools/interpreter.ts (1)
180-182: Update to local file system path enhances security and control over file management.
templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts
Outdated
Show resolved
Hide resolved
templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts
Outdated
Show resolved
Hide resolved
templates/types/streaming/express/src/controllers/chat.controller.ts
Outdated
Show resolved
Hide resolved
templates/types/streaming/nextjs/app/components/ui/upload-csv-preview.tsx
Outdated
Show resolved
Hide resolved
templates/types/streaming/nextjs/app/components/ui/chat/chat-input.tsx
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 0
Outside diff range and nitpick comments (1)
templates/components/engines/typescript/agent/tools/interpreter.ts (1)
Line range hint
6-6: Change the import offsto use thenode:protocol for consistency with Node.js best practices.- import fs from "fs"; + import fs from "node:fs";
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (3)
- templates/components/engines/typescript/agent/tools/interpreter.ts (1 hunks)
- templates/types/streaming/nextjs/app/components/ui/chat/chat-message.tsx (3 hunks)
- templates/types/streaming/nextjs/app/components/ui/chat/index.ts (3 hunks)
Additional Context Used
Biome (7)
templates/components/engines/typescript/agent/tools/interpreter.ts (3)
4-4: A Node.js builtin module should be imported with the node: protocol.
2-2: Some named imports are only used as types.
5-5: All these imports are only used as types.
templates/types/streaming/nextjs/app/components/ui/chat/chat-message.tsx (3)
4-4: All these imports are only used as types.
13-22: Some named imports are only used as types.
104-104: Avoid using the index of an array as key property in an element.
templates/types/streaming/nextjs/app/components/ui/chat/index.ts (1)
2-2: All these imports are only used as types.
Additional comments not posted (4)
templates/types/streaming/nextjs/app/components/ui/chat/index.ts (2)
9-9: Addition ofCSVtoMessageAnnotationTypeenum is appropriate for handling CSV data as part of message annotations.
20-22: Definition ofCsvDatatype is clear and aligns with the requirements for handling CSV content in messages.templates/types/streaming/nextjs/app/components/ui/chat/chat-message.tsx (2)
11-11: Import ofCsvContentis necessary for rendering CSV data within chat messages.
51-54: Correct usage ofCsvDatatype to handle CSV annotations in chat messages.
There was a problem hiding this comment.
Actionable comments posted: 5
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files ignored due to path filters (1)
templates/types/streaming/nextjs/app/components/ui/icons/sheet.svgis excluded by!**/*.svg
Files selected for processing (7)
- .gitignore (1 hunks)
- templates/types/streaming/express/src/controllers/chat.controller.ts (3 hunks)
- templates/types/streaming/express/src/controllers/llamaindex-stream.ts (2 hunks)
- templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts (2 hunks)
- templates/types/streaming/nextjs/app/api/chat/route.ts (4 hunks)
- templates/types/streaming/nextjs/app/components/ui/chat/chat-input.tsx (5 hunks)
- templates/types/streaming/nextjs/app/components/ui/upload-csv-preview.tsx (1 hunks)
Files skipped from review due to trivial changes (1)
- .gitignore
Files skipped from review as they are similar to previous changes (1)
- templates/types/streaming/nextjs/app/components/ui/upload-csv-preview.tsx
Additional Context Used
Biome (20)
templates/types/streaming/express/src/controllers/chat.controller.ts (4)
2-2: Some named imports are only used as types.
3-3: All these imports are only used as types.
4-4: Some named imports are only used as types.
6-10: Some named imports are only used as types.
templates/types/streaming/express/src/controllers/llamaindex-stream.ts (5)
54-57: Template literals are preferred over string concatenation.
89-89: This variable implicitly has the any type.
2-8: Some named imports are only used as types.
9-15: Some named imports are only used as types.
17-17: All these imports are only used as types.
templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts (5)
54-57: Template literals are preferred over string concatenation.
89-89: This variable implicitly has the any type.
2-8: Some named imports are only used as types.
9-15: Some named imports are only used as types.
17-17: All these imports are only used as types.
templates/types/streaming/nextjs/app/api/chat/route.ts (4)
3-3: Some named imports are only used as types.
4-4: Some named imports are only used as types.
5-5: Some named imports are only used as types.
8-12: Some named imports are only used as types.
templates/types/streaming/nextjs/app/components/ui/chat/chat-input.tsx (2)
85-85: Unexpected any. Specify a different type.
8-8: All these imports are only used as types.
Additional comments not posted (5)
templates/types/streaming/express/src/controllers/chat.controller.ts (1)
49-50: Confirm that CSV content is correctly appended to the stream data.Verification successful
The
appendCsvDatafunction is correctly implemented to handle CSV content. It checks ifcsvContentis provided and, if so, appends it to thedataobject with the appropriate annotation.
templates/types/streaming/nextjs/app/api/chat/stream-helper.tstemplates/types/streaming/express/src/controllers/stream-helper.tsBoth implementations are consistent and correctly handle the CSV content.
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify that CSV content is correctly appended to the stream data. # Test: Search for the function `appendCsvData` usage. Expect: Correct handling of CSV content. rg --type typescript 'appendCsvData'Length of output: 71
Script:
#!/bin/bash # Description: Verify that CSV content is correctly appended to the stream data. # Test: Search for the function `appendCsvData` usage. Expect: Correct handling of CSV content. rg --type ts 'appendCsvData'Length of output: 700
Script:
#!/bin/bash # Description: Extract the implementation of the `appendCsvData` function to verify correct handling of CSV content. # Extract the function implementation from the identified files. ast-grep --lang typescript --pattern 'function appendCsvData($_, $_) { $$$ }'Length of output: 1743
templates/types/streaming/nextjs/app/api/chat/route.ts (1)
61-62: Check that CSV content is correctly transformed and appended in the streaming response.Verification successful
The
appendCsvDatafunction instream-helper.tsis responsible for appending CSV content to the data stream. It checks ifcsvContentis present and then appends it as a message annotation. The function is used inllamaindex-stream.tswithin aReadableStreamto append CSV content when the stream starts.
stream-helper.ts:export function appendCsvData(data: StreamData, csvContent?: string) { if (!csvContent) return; data.appendMessageAnnotation({ type: "csv", data: { content: csvContent, }, }); }
llamaindex-stream.ts:import { appendCsvData } from "./stream-helper"; return new ReadableStream<string>({ start() { appendImageData(data, opts?.imageUrl); appendCsvData(data, opts?.csvContent); }, async pull(controller): Promise<void> { const { value, done } = await it.next(); if (done) { if (sourceNodes) {The code correctly handles the transformation and appending of CSV content in the streaming response.
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify correct transformation and appending of CSV content in streaming responses. # Test: Search for transformation logic involving `csvContent`. Expect: Correct handling in streaming responses. rg --type typescript 'csvContent'Length of output: 68
Script:
#!/bin/bash # Description: Verify correct transformation and appending of CSV content in streaming responses. # Test: Search for transformation logic involving `csvContent`. Expect: Correct handling in streaming responses. rg --type ts 'csvContent'Length of output: 2229
Script:
#!/bin/bash # Extract the appendCsvData function and its usage in llamaindex-stream.ts for detailed inspection. # Extract appendCsvData function from stream-helper.ts echo "Extracting appendCsvData function from stream-helper.ts" ast-grep --lang typescript --pattern $'export function appendCsvData($_, $_) { $$$ }' templates/types/streaming/nextjs/app/api/chat/stream-helper.ts # Extract usage of appendCsvData in llamaindex-stream.ts echo "Extracting usage of appendCsvData in llamaindex-stream.ts" rg --context 5 'appendCsvData' templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.tsLength of output: 1872
templates/types/streaming/express/src/controllers/llamaindex-stream.ts (1)
74-75: Ensure that CSV data appending functions are optimized for performance.Verification successful
The implementations of
appendImageDataandappendCsvDatahave been successfully extracted. Both functions perform simple checks and append data to theStreamDataobject if the respective content is provided. These operations appear to be straightforward and efficient.
appendImageDatachecks ifimageUrlis provided and appends an image annotation to the data.appendCsvDatachecks ifcsvContentis provided and appends a CSV annotation to the data.Given their simplicity, these functions are already optimized for performance.
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify optimization of data appending functions. # Test: Search for `appendCsvData` and `appendImageData` usage. Expect: Optimized performance. rg --type typescript 'appendCsvData|appendImageData'Length of output: 89
Script:
#!/bin/bash # Description: Verify optimization of data appending functions. # Test: Search for `appendCsvData` and `appendImageData` usage. Expect: Optimized performance. rg --type ts 'appendCsvData|appendImageData'Length of output: 1394
Script:
#!/bin/bash # Description: Extract implementations of appendCsvData and appendImageData for review. # Test: Verify optimization of data appending functions. ast-grep --lang typescript --pattern 'export function appendImageData($_$) { $$$ }' templates/types/streaming/express/src/controllers/stream-helper.ts ast-grep --lang typescript --pattern 'export function appendCsvData($_$) { $$$ }' templates/types/streaming/express/src/controllers/stream-helper.tsLength of output: 303
Script:
#!/bin/bash # Description: Locate exact lines of appendCsvData and appendImageData definitions. # Test: Verify optimization of data appending functions. rg 'export function appendImageData' templates/types/streaming/express/src/controllers/stream-helper.ts rg 'export function appendCsvData' templates/types/streaming/express/src/controllers/stream-helper.tsLength of output: 351
Script:
#!/bin/bash # Description: Extract full implementations of appendCsvData and appendImageData for review. # Test: Verify optimization of data appending functions. rg -A 20 'export function appendImageData' templates/types/streaming/express/src/controllers/stream-helper.ts rg -A 20 'export function appendCsvData' templates/types/streaming/express/src/controllers/stream-helper.tsLength of output: 1168
templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts (1)
74-75: Verify that data appending functions are consistently optimized across both Express and Next.js implementations.Verification successful
The search results indicate that the
appendCsvDataandappendImageDatafunctions are used in both Express and Next.js implementations. The functions are defined instream-helper.tsfiles for both platforms and are used inllamaindex-stream.tsfiles.
templates/types/streaming/express/src/controllers/stream-helper.tstemplates/types/streaming/nextjs/app/api/chat/stream-helper.tstemplates/types/streaming/express/src/controllers/llamaindex-stream.tstemplates/types/streaming/nextjs/app/api/chat/llamaindex-stream.tsThis suggests that the data appending functions are consistently implemented across both platforms. However, to ensure they are optimized similarly, a deeper inspection of the function implementations is necessary.
The implementations of
appendCsvDataandappendImageDatafunctions are identical in both the Express and Next.js platforms. Here are the relevant snippets from both implementations:Express Implementation
export function appendImageData(data: StreamData, imageUrl?: string) { if (!imageUrl) return; data.appendMessageAnnotation({ type: "image", data: { url: imageUrl, }, }); } export function appendCsvData(data: StreamData, csvContent?: string) { if (!csvContent) return; data.appendMessageAnnotation({ type: "csv", data: { content: csvContent, }, }); }Next.js Implementation
export function appendImageData(data: StreamData, imageUrl?: string) { if (!imageUrl) return; data.appendMessageAnnotation({ type: "image", data: { url: imageUrl, }, }); } export function appendCsvData(data: StreamData, csvContent?: string) { if (!csvContent) return; data.appendMessageAnnotation({ type: "csv", data: { content: csvContent, }, }); }Both implementations are consistent and optimized in the same manner. Therefore, the concern about consistent optimization across both platforms is addressed.
templates/types/streaming/express/src/controllers/stream-helper.tstemplates/types/streaming/nextjs/app/api/chat/stream-helper.tsScripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify consistent optimization of data appending functions across platforms. # Test: Search for `appendCsvData` and `appendImageData` usage in both Express and Next.js implementations. Expect: Consistent optimization. rg --type typescript 'appendCsvData|appendImageData'Length of output: 89
Script:
#!/bin/bash # Description: Verify consistent optimization of data appending functions across platforms. # Test: Search for `appendCsvData` and `appendImageData` usage in both Express and Next.js implementations. Expect: Consistent optimization. rg --type ts 'appendCsvData|appendImageData'Length of output: 1394
Script:
#!/bin/bash # Description: Retrieve the implementations of appendCsvData and appendImageData functions for comparison. # Test: Extract the function definitions from the relevant files. ast-grep --lang typescript --pattern 'function appendImageData($_, $_) { $$$ }' templates/types/streaming/express/src/controllers/stream-helper.ts ast-grep --lang typescript --pattern 'function appendCsvData($_, $_) { $$$ }' templates/types/streaming/express/src/controllers/stream-helper.ts ast-grep --lang typescript --pattern 'function appendImageData($_, $_) { $$$ }' templates/types/streaming/nextjs/app/api/chat/stream-helper.ts ast-grep --lang typescript --pattern 'function appendCsvData($_, $_) { $$$ }' templates/types/streaming/nextjs/app/api/chat/stream-helper.tsLength of output: 3827
templates/types/streaming/nextjs/app/components/ui/chat/chat-input.tsx (1)
97-103: Confirm that the CSV file preview component handles state updates correctly.Verification successful
The context around the state updates for
uploadedCsvshows that the state is being set toundefinedand to an object with propertiescontent,filename, andfilesize. Additionally, theonRemovefunction correctly setsuploadedCsvtoundefinedwhen the CSV file is removed.The state updates appear to be handled correctly within the component lifecycle, ensuring that the
uploadedCsvstate is properly managed and updated.
- The state is initialized with
useState<UploadedCsv>().- The state is set to
undefinedafter form submission.- The state is updated with the CSV file details when a file is uploaded.
- The state is reset to
undefinedwhen the file is removed via theonRemovefunction.This confirms that the CSV file preview component handles state updates correctly.
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify correct state handling in CSV file preview component. # Test: Search for state updates related to `uploadedCsv`. Expect: Correct handling in component lifecycle. rg --type typescript 'setUploadedCsv'Length of output: 72
Script:
#!/bin/bash # Description: Verify correct state handling in CSV file preview component. # Test: Search for state updates related to `uploadedCsv`. Expect: Correct handling in component lifecycle. rg --type ts 'setUploadedCsv'Length of output: 485
Script:
#!/bin/bash # Description: Extract context around state updates for `uploadedCsv` to verify correct handling. # Extract 5 lines before and after each occurrence of `setUploadedCsv`. rg --type ts 'setUploadedCsv' -A 5 -B 5Length of output: 4316
templates/types/streaming/express/src/controllers/llamaindex-stream.ts
Outdated
Show resolved
Hide resolved
templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 4
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (6)
- helpers/env-variables.ts (1 hunks)
- templates/types/streaming/express/src/controllers/chat.controller.ts (4 hunks)
- templates/types/streaming/express/src/controllers/stream-helper.ts (2 hunks)
- templates/types/streaming/nextjs/app/api/chat/route.ts (4 hunks)
- templates/types/streaming/nextjs/app/api/chat/stream-helper.ts (2 hunks)
- templates/types/streaming/nextjs/app/components/ui/upload-csv-preview.tsx (1 hunks)
Files skipped from review as they are similar to previous changes (1)
- templates/types/streaming/nextjs/app/components/ui/upload-csv-preview.tsx
Additional Context Used
Biome (21)
helpers/env-variables.ts (9)
2-2: A Node.js builtin module should be imported with the node: protocol.
3-3: A Node.js builtin module should be imported with the node: protocol.
138-151: Other switch clauses can erroneously access this declaration.
Wrap the declaration in a block to restrict its access to the switch clause.
292-302: Prefer for...of instead of forEach.
312-319: Prefer for...of instead of forEach.
317-317: Template literals are preferred over string concatenation.
349-349: Do not use template literals if interpolation and special-character handling are not needed.
4-4: Some named imports are only used as types.
5-10: All these imports are only used as types.
templates/types/streaming/express/src/controllers/chat.controller.ts (4)
2-2: Some named imports are only used as types.
3-3: All these imports are only used as types.
4-4: Some named imports are only used as types.
6-10: Some named imports are only used as types.
templates/types/streaming/express/src/controllers/stream-helper.ts (2)
2-2: All these imports are only used as types.
3-9: Some named imports are only used as types.
templates/types/streaming/nextjs/app/api/chat/route.ts (4)
3-3: Some named imports are only used as types.
4-4: Some named imports are only used as types.
5-5: Some named imports are only used as types.
8-12: Some named imports are only used as types.
templates/types/streaming/nextjs/app/api/chat/stream-helper.ts (2)
2-2: All these imports are only used as types.
3-9: Some named imports are only used as types.
Additional comments not posted (7)
templates/types/streaming/express/src/controllers/chat.controller.ts (2)
20-20: Ensure proper validation ofdatabefore usage.
49-50: Ensure CSV content is handled correctly in the stream.Verification successful
The handling of
csvContentin the stream appears to be correctly implemented. TheappendCsvDatafunction instream-helper.tsensures that the CSV content is appended as a message annotation if it exists. Additionally, the logic inllamaindex-stream.tsformats the CSV content properly before returning it.
stream-helper.ts:
- The
appendCsvDatafunction checks ifcsvContentis present and appends it as a message annotation.llamaindex-stream.ts:
- The logic formats the CSV content and includes it in the returned message.
These implementations indicate that CSV content is handled appropriately in the stream.
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify CSV content handling in the stream. # Test: Search for CSV content handling logic in the project. Expect: Proper handling logic. rg --type typescript 'csvContent'Length of output: 68
Script:
#!/bin/bash # Description: Verify CSV content handling in the stream. # Test: Search for CSV content handling logic in the project. Expect: Proper handling logic. rg --type ts 'csvContent'Length of output: 2229
Script:
#!/bin/bash # Description: Extract relevant functions and logic handling csvContent. # Test: Ensure proper handling of csvContent in the stream. # Extract functions from stream-helper.ts ast-grep --lang ts --pattern 'function appendCsvData($_, $_) { $$$ }' templates/types/streaming/express/src/controllers/stream-helper.ts # Extract logic from llamaindex-stream.ts ast-grep --lang ts --pattern 'if (additionalData?.csvContent) { $$$ }' templates/types/streaming/express/src/controllers/llamaindex-stream.tsLength of output: 1923
templates/types/streaming/nextjs/app/api/chat/route.ts (2)
62-63: Ensure CSV content is handled correctly in the stream.
30-30: Ensure proper validation ofdatabefore usage.templates/types/streaming/express/src/controllers/stream-helper.ts (1)
78-85: Ensure proper handling of stream timeouts.Verification successful
Ensure proper handling of stream timeouts.
The search results indicate that the
createStreamTimeoutfunction intemplates/types/streaming/express/src/controllers/stream-helper.tsis correctly handling stream timeouts by setting a timeout and appending event data when the timeout occurs. The logic appears consistent with other timeout implementations in the codebase.
templates/types/streaming/express/src/controllers/stream-helper.ts: Lines 78-85Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify stream timeout handling in the project. # Test: Search for timeout handling logic in the project. Expect: Proper handling logic. rg --type typescript 'timeout'Length of output: 65
Script:
#!/bin/bash # Description: Verify stream timeout handling in the project. # Test: Search for timeout handling logic in the project. Expect: Proper handling logic. rg --type ts 'timeout'Length of output: 2349
templates/types/streaming/nextjs/app/api/chat/stream-helper.ts (1)
78-85: Ensure proper handling of stream timeouts.Verification successful
Ensure proper handling of stream timeouts.
The search results indicate that the timeout handling logic is present in multiple files, including the
stream-helper.tsfile under review. The timeout is set and handled appropriately in thecreateStreamTimeoutfunction.
templates/types/streaming/nextjs/app/api/chat/stream-helper.ts: Lines 78-85 handle the stream timeout by setting a timeout and appending event data when the timeout occurs.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify stream timeout handling in the project. # Test: Search for timeout handling logic in the project. Expect: Proper handling logic. rg --type typescript 'timeout'Length of output: 65
Script:
#!/bin/bash # Description: Verify stream timeout handling in the project. # Test: Search for timeout handling logic in the project. Expect: Proper handling logic. rg --type ts 'timeout'Length of output: 2349
helpers/env-variables.ts (1)
280-284: Ensure proper handling of theSTREAM_TIMEOUTenvironment variable.Verification successful
The
STREAM_TIMEOUTenvironment variable is properly defined and used in the codebase. It is set with a default value and utilized in the code to determine a timeout duration.
env-variables.ts: DefinesSTREAM_TIMEOUT.- Usage: Converts
STREAM_TIMEOUTto a number with a default value of 5 minutes.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify handling of `STREAM_TIMEOUT` in the project. # Test: Search for usage of `STREAM_TIMEOUT` in the project. Expect: Proper usage and handling. rg --type typescript 'STREAM_TIMEOUT'Length of output: 72
Script:
#!/bin/bash # Description: Verify handling of `STREAM_TIMEOUT` in the project. # Test: Search for usage of `STREAM_TIMEOUT` in the project. Expect: Proper usage and handling. # Find all TypeScript files and search for `STREAM_TIMEOUT` fd --extension ts --exec rg 'STREAM_TIMEOUT' {}Length of output: 269
templates/types/streaming/express/src/controllers/stream-helper.ts
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 0
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- .changeset/bright-turkeys-melt.md (1 hunks)
Files skipped from review due to trivial changes (1)
- .changeset/bright-turkeys-melt.md
| const handleUploadFile = async (file: File) => { | ||
| try { | ||
| if (props.multiModal && file.type.startsWith("image/")) { | ||
| if (file.type.startsWith("image/")) { |
There was a problem hiding this comment.
remove multiModal from props?
There was a problem hiding this comment.
Actionable comments posted: 10
Outside diff range and nitpick comments (4)
templates/types/streaming/express/src/controllers/chat.controller.ts (1)
[!TIP]
Codebase VerificationThe extracted code sections show that
csvContentis directly assigned fromadditionalData.uploadedCsv.contentwithout any validation. This indicates a potential issue as there is no check to ensure the CSV content is valid or safe before usage.
- File:
templates/types/streaming/express/src/controllers/llamaindex-stream.ts- File:
templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.tsBoth files need proper validation for
csvContentbefore it is used.Analysis chain
Line range hint
13-61: Updatedchatfunction to handle CSV data and manage streams with timeouts. Ensure proper error handling and validation for CSV content.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify proper validation of CSV content in all usages within the project. # Test: Search for usages of `csvContent` in the project. Expect: Proper validation checks before usage. rg --type typescript 'csvContent'Length of output: 68
Script:
#!/bin/bash # Description: Verify proper validation of CSV content in all usages within the project. # Test: Search for usages of `csvContent` in the project. Expect: Proper validation checks before usage. rg --type ts 'csvContent'Length of output: 439
Script:
#!/bin/bash # Description: Verify proper validation of `csvContent` in the identified files. # Test: Search for the definition and usage of `csvContent` in the identified files and check for validation logic. rg 'csvContent' templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts templates/types/streaming/express/src/controllers/llamaindex-stream.tsLength of output: 567
Script:
#!/bin/bash # Description: Extract and review the relevant code sections from the identified files to ensure `csvContent` is properly validated. # Test: Display the surrounding code of `csvContent` in the identified files. rg -A 10 -B 10 'csvContent' templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts templates/types/streaming/express/src/controllers/llamaindex-stream.tsLength of output: 4756
templates/types/streaming/nextjs/app/api/chat/route.ts (1)
[!TIP]
Codebase VerificationThe extracted snippets indicate that
uploadedCsvis checked for existence (if (additionalData?.uploadedCsv)) but there is no explicit validation of its content before usage. This could potentially lead to issues if the content is malformed or contains unexpected data.
- Files to review for validation logic:
templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.tstemplates/types/streaming/express/src/controllers/llamaindex-stream.tsPlease ensure that
uploadedCsv.contentis properly validated before being used to constructcsvContent.Analysis chain
Line range hint
21-79: UpdatedPOSTfunction to handle CSV data and manage streams with timeouts. Ensure proper error handling and validation for CSV content.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify proper validation of CSV content in all usages within the project. # Test: Search for usages of `csvContent` in the project. Expect: Proper validation checks before usage. rg --type typescript 'csvContent'Length of output: 68
Script:
#!/bin/bash # Description: Verify proper validation of CSV content in all usages within the project. # Test: Search for usages of `csvContent` in the project. Expect: Proper validation checks before usage. rg 'csvContent' --glob '*.ts'Length of output: 443
Script:
#!/bin/bash # Description: Extract and review the relevant code snippets to verify the validation of `csvContent`. # Extract lines around the usage of `csvContent` in the identified files. rg 'csvContent' --glob '*.ts' -A 5 -B 5Length of output: 2800
Script:
#!/bin/bash # Description: Verify validation of `additionalData.uploadedCsv.content` before usage. # Extract lines around the usage of `additionalData.uploadedCsv.content` in the identified files. rg 'additionalData.uploadedCsv.content' --glob '*.ts' -A 5 -B 5Length of output: 2086
Script:
#!/bin/bash # Description: Verify validation of `uploadedCsv` before usage. # Extract lines around the usage of `uploadedCsv` in the identified files. rg 'uploadedCsv' --glob '*.ts' -A 5 -B 5Length of output: 11591
templates/types/streaming/express/src/controllers/stream-helper.ts (2)
Line range hint
1-1: Consider using TypeScript'stypeimport for types used only in type annotations to optimize bundling.- import { StreamData } from "ai"; + import type { StreamData } from "ai"; - import { - CallbackManager, - Metadata, - NodeWithScore, - ToolCall, - ToolOutput, - } from "llamaindex"; + import type { + CallbackManager, + Metadata, + NodeWith Score, + ToolCall, + ToolOutput, + } from "llamaindex";Also applies to: 26-26
[!TIP]
Codebase VerificationValidation for
uploadedCsv.contentis missingThe
uploadedCsv.contentis used directly without validation in the following files:
templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.tstemplates/types/streaming/express/src/controllers/llamaindex-stream.tsTo ensure robustness, consider adding validation for
uploadedCsv.contentbefore it is used.Analysis chain
Line range hint
20-89: Introduced new types and functions for handling CSV data and managing streams with timeouts. Ensure proper error handling and validation for CSV content.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify proper validation of CSV content in all usages within the project. # Test: Search for usages of `csvContent` in the project. Expect: Proper validation checks before usage. rg --type typescript 'csvContent'Length of output: 68
Script:
#!/bin/bash # Description: Verify proper validation of CSV content in all usages within the project. # Test: Search for usages of `csvContent` in the project. Expect: Proper validation checks before usage. rg --type ts 'csvContent'Length of output: 439
Script:
#!/bin/bash # Description: Extract relevant code snippets to verify validation of `csvContent`. # Extract lines around the usage of `csvContent` in the identified files. rg --type ts --context 10 'csvContent' templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts templates/types/streaming/express/src/controllers/llamaindex-stream.tsLength of output: 4767
Script:
#!/bin/bash # Description: Search for the definition and handling of `additionalData` and `uploadedCsv` in the relevant files. # Extract lines around the definition and usage of `additionalData` and `uploadedCsv`. rg --type ts --context 10 'additionalData' templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts templates/types/streaming/express/src/controllers/llamaindex-stream.tsLength of output: 7703
Script:
#!/bin/bash # Description: Search for the usage and handling of `uploadedCsv` and `UploadedCsv` types in the codebase. # Extract lines around the usage of `uploadedCsv` and `UploadedCsv`. rg --type ts --context 10 'uploadedCsv'Length of output: 27872
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (11)
- templates/types/streaming/express/src/controllers/chat.controller.ts (4 hunks)
- templates/types/streaming/express/src/controllers/llamaindex-stream.ts (2 hunks)
- templates/types/streaming/express/src/controllers/stream-helper.ts (2 hunks)
- templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts (2 hunks)
- templates/types/streaming/nextjs/app/api/chat/route.ts (4 hunks)
- templates/types/streaming/nextjs/app/api/chat/stream-helper.ts (2 hunks)
- templates/types/streaming/nextjs/app/components/chat-section.tsx (2 hunks)
- templates/types/streaming/nextjs/app/components/ui/chat/chat-input.tsx (4 hunks)
- templates/types/streaming/nextjs/app/components/ui/chat/chat-message.tsx (4 hunks)
- templates/types/streaming/nextjs/app/components/ui/chat/chat-resources.tsx (1 hunks)
- templates/types/streaming/nextjs/app/components/ui/chat/index.ts (3 hunks)
Additional Context Used
Biome (34)
templates/types/streaming/express/src/controllers/chat.controller.ts (4)
1-1: Some named imports are only used as types.
1-2: All these imports are only used as types.
2-3: Some named imports are only used as types.
4-9: Some named imports are only used as types.
templates/types/streaming/express/src/controllers/llamaindex-stream.ts (6)
54-57: Template literals are preferred over string concatenation.
89-89: This variable implicitly has the any type.
1-7: Some named imports are only used as types.
7-14: Some named imports are only used as types.
15-16: All these imports are only used as types.
16-22: Some named imports are only used as types.
templates/types/streaming/express/src/controllers/stream-helper.ts (2)
1-1: All these imports are only used as types.
1-8: Some named imports are only used as types.
templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts (6)
54-57: Template literals are preferred over string concatenation.
89-89: This variable implicitly has the any type.
1-7: Some named imports are only used as types.
7-14: Some named imports are only used as types.
15-16: All these imports are only used as types.
16-22: Some named imports are only used as types.
templates/types/streaming/nextjs/app/api/chat/route.ts (4)
1-2: Some named imports are only used as types.
2-3: Some named imports are only used as types.
3-4: Some named imports are only used as types.
6-11: Some named imports are only used as types.
templates/types/streaming/nextjs/app/api/chat/stream-helper.ts (2)
1-1: All these imports are only used as types.
1-8: Some named imports are only used as types.
templates/types/streaming/nextjs/app/components/ui/chat/chat-input.tsx (3)
102-102: Unexpected any. Specify a different type.
1-2: All these imports are only used as types.
8-9: All these imports are only used as types.
templates/types/streaming/nextjs/app/components/ui/chat/chat-message.tsx (3)
2-3: All these imports are only used as types.
11-21: Some named imports are only used as types.
96-96: Avoid using the index of an array as key property in an element.
templates/types/streaming/nextjs/app/components/ui/chat/chat-resources.tsx (1)
2-3: All these imports are only used as types.
templates/types/streaming/nextjs/app/components/ui/chat/index.ts (3)
84-98: Prefer for...of instead of forEach.
90-96: Prefer for...of instead of forEach.
1-1: All these imports are only used as types.
Additional comments not posted (6)
templates/types/streaming/nextjs/app/components/chat-section.tsx (1)
40-40: Passing resources toChatInputusinggetInputResourcesaligns with the enhancements for handling CSV uploads.templates/types/streaming/nextjs/app/components/ui/chat/chat-resources.tsx (1)
1-10: IntroducedChatResourcescomponent and necessary imports to manage and display CSV resources.templates/types/streaming/nextjs/app/components/ui/chat/index.ts (1)
Line range hint
1-102: Introduced new types and functions for handling CSV and other data types in messages. Ensure these types are used consistently across the application.Verification successful
The search results indicate that the new types (
CsvData,ImageData,SourceData,EventData,ToolData, andMessageAnnotation) are consistently used across multiple files in the project. This confirms that the types introduced intemplates/types/streaming/nextjs/app/components/ui/chat/index.tsare being utilized correctly throughout the codebase.
CsvDatais used in various files such aschat-message.tsx,csv-content.tsx,chat-input.tsx,chat-resources.tsx, and others.ImageDatais used inchat-message.tsx,chat-image.tsx, and other files.SourceDatais used inchat-message.tsx,chat-sources.tsx, and other files.EventDatais used inchat-message.tsx,chat-events.tsx, and other files.ToolDatais used inchat-message.tsx,chat-tools.tsx, and other files.MessageAnnotationandMessageAnnotationTypeare used inchat-message.tsx,stream-helper.ts, and other files.Given this consistent usage, the changes appear to be correctly integrated.
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify consistent usage of new types across the application. # Test: Search for usages of `CsvData` and other new types in the project. Expect: Consistent usage across files. rg --type typescript 'CsvData'Length of output: 65
Script:
#!/bin/bash # Description: Verify consistent usage of new types across the application. # Test: Search for usages of `CsvData` and other new types in the project. Expect: Consistent usage across files. rg --type-add 'ts:*.ts' --type ts 'CsvData' rg --type-add 'ts:*.ts' --type ts 'ImageData' rg --type-add 'ts:*.ts' --type ts 'SourceData' rg --type-add 'ts:*.ts' --type ts 'EventData' rg --type-add 'ts:*.ts' --type ts 'ToolData' rg --type-add 'ts:*.ts' --type ts 'MessageAnnotation'Length of output: 10471
templates/types/streaming/nextjs/app/api/chat/stream-helper.ts (1)
20-24: TheUploadedCsvtype definition is clear and well-structured.templates/types/streaming/express/src/controllers/llamaindex-stream.ts (1)
67-76: The implementation ofcreateParseris robust, handling different types of data annotations effectively.templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts (1)
67-76: The implementation ofcreateParseris robust, handling different types of data annotations effectively.
| export function appendCsvData(data: StreamData, uploadedCsv?: UploadedCsv) { | ||
| if (!uploadedCsv) return; | ||
| data.appendMessageAnnotation({ | ||
| type: "csv", | ||
| data: uploadedCsv, | ||
| }); | ||
| } |
There was a problem hiding this comment.
Consider adding a check for empty csvContent to handle cases where the CSV content might be an empty string.
if (!uploadedCsv || !uploadedCsv.content.trim()) return;Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
| export function appendCsvData(data: StreamData, uploadedCsv?: UploadedCsv) { | |
| if (!uploadedCsv) return; | |
| data.appendMessageAnnotation({ | |
| type: "csv", | |
| data: uploadedCsv, | |
| }); | |
| } | |
| export function appendCsvData(data: StreamData, uploadedCsv?: UploadedCsv) { | |
| if (!uploadedCsv || !uploadedCsv.content.trim()) return; | |
| data.appendMessageAnnotation({ | |
| type: "csv", | |
| data: uploadedCsv, | |
| }); | |
| } |
templates/types/streaming/nextjs/app/components/ui/chat/chat-resources.tsx
Show resolved
Hide resolved
| export type DataParserOptions = { | ||
| imageUrl?: string; | ||
| uploadedCsv?: UploadedCsv; | ||
| }; | ||
|
|
||
| export const convertMessageContent = ( | ||
| textMessage: string, | ||
| additionalData?: DataParserOptions, | ||
| ): MessageContent => { | ||
| if (additionalData?.imageUrl) { | ||
| return [ | ||
| { | ||
| type: "text", | ||
| text: textMessage, | ||
| }, | ||
| { | ||
| type: "image_url", | ||
| image_url: { | ||
| url: additionalData?.imageUrl, | ||
| }, | ||
| }, | ||
| ]; | ||
| } | ||
|
|
||
| if (additionalData?.uploadedCsv) { | ||
| const csvContent = | ||
| "Use the following CSV data:\n" + | ||
| "```csv\n" + | ||
| additionalData.uploadedCsv.content + | ||
| "\n```"; | ||
| return `${csvContent}\n\n${textMessage}`; | ||
| } | ||
|
|
||
| return textMessage; |
There was a problem hiding this comment.
The updates to DataParserOptions and convertMessageContent are well-implemented. However, consider using template literals for better readability and performance in string concatenation.
- const csvContent =
- "Use the following CSV data:\n" +
- "```csv\n" +
- additionalData.uploadedCsv.content +
- "\n```";
+ const csvContent = `Use the following CSV data:\n\`\`\`csv\n${additionalData.uploadedCsv.content}\n\`\`\``;
templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts
Outdated
Show resolved
Hide resolved
| order: -2, | ||
| component: csvData[0] ? <CsvContent data={csvData[0]} /> : null, | ||
| }, |
There was a problem hiding this comment.
The integration of CsvContent is well done. However, consider using a more stable identifier than the index as a key in React lists to prevent potential issues with re-rendering.
- <Fragment key={index}>{content.component}</Fragment>
+ <Fragment key={content.order}>{content.component}</Fragment>Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
| order: -2, | |
| component: csvData[0] ? <CsvContent data={csvData[0]} /> : null, | |
| }, | |
| order: -2, | |
| component: csvData[0] ? <CsvContent data={csvData[0]} /> : null, | |
| }, |
There was a problem hiding this comment.
Actionable comments posted: 4
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (3)
- templates/types/streaming/express/src/controllers/stream-helper.ts (2 hunks)
- templates/types/streaming/nextjs/app/api/chat/stream-helper.ts (2 hunks)
- templates/types/streaming/nextjs/app/components/ui/chat/index.ts (3 hunks)
Additional Context Used
Biome (7)
templates/types/streaming/express/src/controllers/stream-helper.ts (2)
1-1: All these imports are only used as types.
1-8: Some named imports are only used as types.
templates/types/streaming/nextjs/app/api/chat/stream-helper.ts (2)
1-1: All these imports are only used as types.
1-8: Some named imports are only used as types.
templates/types/streaming/nextjs/app/components/ui/chat/index.ts (3)
85-99: Prefer for...of instead of forEach.
91-97: Prefer for...of instead of forEach.
1-1: All these imports are only used as types.
Additional comments not posted (7)
templates/types/streaming/nextjs/app/components/ui/chat/index.ts (4)
9-9: Added new enum value for CSV.
20-24: Introduced new typeCsvDatafor handling CSV content.
57-62: UpdatedAnnotationDatatype to includeCsvData.
69-74: Utility function to filter and map annotations by type.templates/types/streaming/express/src/controllers/stream-helper.ts (1)
20-24: Introduced new typeUploadedCsvfor handling CSV data in streams.templates/types/streaming/nextjs/app/api/chat/stream-helper.ts (2)
20-24: Introduced new typeUploadedCsvfor handling CSV data in streams.
83-90: Define the default timeout value as a constant for better readability and maintainability.+ const DEFAULT_TIMEOUT = 1000 * 60 * 5; // 5 minutes const timeout = Number(process.env.STREAM_TIMEOUT ?? DEFAULT_TIMEOUT);Likely invalid or redundant comment.
| export function appendCsvData(data: StreamData, uploadedCsv?: UploadedCsv) { | ||
| if (!uploadedCsv) return; | ||
| data.appendMessageAnnotation({ | ||
| type: "csv", | ||
| data: uploadedCsv, | ||
| }); |
There was a problem hiding this comment.
Function to append CSV data to a stream. Ensure it handles empty or null csvContent gracefully.
+ if (!uploadedCsv || !uploadedCsv.content.trim()) return;Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
| export function appendCsvData(data: StreamData, uploadedCsv?: UploadedCsv) { | |
| if (!uploadedCsv) return; | |
| data.appendMessageAnnotation({ | |
| type: "csv", | |
| data: uploadedCsv, | |
| }); | |
| export function appendCsvData(data: StreamData, uploadedCsv?: UploadedCsv) { | |
| if (!uploadedCsv || !uploadedCsv.content.trim()) return; | |
| data.appendMessageAnnotation({ | |
| type: "csv", | |
| data: uploadedCsv, | |
| }); |
There was a problem hiding this comment.
Actionable comments posted: 4
Outside diff range and nitpick comments (1)
templates/components/ui/html/chat/chat-input.tsx (1)
Line range hint
29-29: Consider removing theautoFocusattribute to improve accessibility.
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (3)
- templates/components/ui/html/chat/chat-input.tsx (2 hunks)
- templates/types/streaming/nextjs/app/components/chat-section.tsx (1 hunks)
- templates/types/streaming/nextjs/app/components/ui/chat/chat-input.tsx (4 hunks)
Files skipped from review due to trivial changes (1)
- templates/types/streaming/nextjs/app/components/chat-section.tsx
Additional Context Used
Biome (5)
templates/components/ui/html/chat/chat-input.tsx (2)
29-29: Avoid the autoFocus attribute.
2-3: All these imports are only used as types.
templates/types/streaming/nextjs/app/components/ui/chat/chat-input.tsx (3)
100-100: Unexpected any. Specify a different type.
1-2: Some named imports are only used as types.
8-9: All these imports are only used as types.
Additional comments not posted (2)
templates/types/streaming/nextjs/app/components/ui/chat/chat-input.tsx (2)
77-89: The CSV file upload functionality is well implemented. Ensure to refine the error handling as suggested.
93-98: The handling of different file types is implemented correctly.
| const removeResource = (index: number) => { | ||
| setInputResources((resources) => { | ||
| const newResources = [...resources]; | ||
| newResources[index].selected = false; | ||
| return newResources; | ||
| }); | ||
| }; |
There was a problem hiding this comment.
Optimize the resource removal logic by directly modifying the selected property.
- const newResources = [...resources];
- newResources[index].selected = false;
+ resources[index].selected = false;Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
| const removeResource = (index: number) => { | |
| setInputResources((resources) => { | |
| const newResources = [...resources]; | |
| newResources[index].selected = false; | |
| return newResources; | |
| }); | |
| }; | |
| const removeResource = (index: number) => { | |
| setInputResources((resources) => { | |
| resources[index].selected = false; | |
| return resources; | |
| }); | |
| }; |
| return await handleUploadImageFile(file); | ||
| } | ||
| if (file.type === "text/csv") { | ||
| return await handleUploadCsvFile(file); | ||
| } |
There was a problem hiding this comment.
Refine the error type handling for better robustness.
- } catch (error: any) {
+ } catch (error: unknown) {
props.onFileError?.(error instanceof Error ? error.message : 'An unknown error occurred');
}Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
| } | |
| } catch (error: unknown) { | |
| props.onFileError?.(error instanceof Error ? error.message : 'An unknown error occurred'); | |
| } |
| import { useEffect, useState } from "react"; | ||
| import { CsvData, getInputResources } from "."; |
There was a problem hiding this comment.
Use explicit type imports for clarity and potential bundling optimizations.
- import { CsvData, getInputResources } from ".";
- import { ChatHandler } from "./chat.interface";
+ import type { CsvData, getInputResources } from ".";
+ import type { ChatHandler } from "./chat.interface";Also applies to: 8-9
Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
| import { useEffect, useState } from "react"; | |
| import { CsvData, getInputResources } from "."; | |
| import { useEffect, useState } from "react"; | |
| import type { CsvData, getInputResources } from "."; | |
| import type { ChatHandler } from "./chat.interface"; |
| @@ -43,25 +74,62 @@ export default function ChatInput( | |||
| setImageUrl(base64); | |||
| }; | |||
|
|
|||
| const handleUploadCsvFile = async (file: File) => { | |||
There was a problem hiding this comment.
This is just an attached CSV in the session, and we won't store it in the backend, right?
There was a problem hiding this comment.
Yes, we don't save it to filesystem in backend
| const handleUploadCsvFile = async (file: File) => { | ||
| const content = await new Promise<string>((resolve, reject) => { | ||
| const reader = new FileReader(); | ||
| reader.readAsText(file); |
There was a problem hiding this comment.
If it's just an attachment, we'll include it every time the file is selected. The app might struggle with large CSV files, so how about we limit the file size?
There was a problem hiding this comment.
Good catch @leehuwuj , we are limitting 50MB for each uploaded file:
The app might struggle with large CSV files
-> Yes, in the long term, we should save these csv files to backend (so we can make the FE lighter)
| return await handleUploadImageFile(file); | ||
| } | ||
| if (file.type === "text/csv") { | ||
| return await handleUploadCsvFile(file); | ||
| } | ||
| props.onFileUpload?.(file); |
There was a problem hiding this comment.
How about other file types (pdf, json, txt,...)? i think we can show a popup error that file type is not supported and list the supported one.
There was a problem hiding this comment.
Yes, we have this props to define file extensions:
There was a problem hiding this comment.
Actionable comments posted: 3
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (5)
- templates/types/streaming/express/src/controllers/llamaindex-stream.ts (2 hunks)
- templates/types/streaming/express/src/controllers/stream-helper.ts (2 hunks)
- templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts (2 hunks)
- templates/types/streaming/nextjs/app/api/chat/stream-helper.ts (2 hunks)
- templates/types/streaming/nextjs/app/components/ui/chat/chat-resources.tsx (1 hunks)
Additional Context Used
Biome (21)
templates/types/streaming/express/src/controllers/llamaindex-stream.ts (6)
56-59: Template literals are preferred over string concatenation.
94-94: This variable implicitly has the any type.
1-7: Some named imports are only used as types.
7-14: Some named imports are only used as types.
15-16: All these imports are only used as types.
16-22: Some named imports are only used as types.
templates/types/streaming/express/src/controllers/stream-helper.ts (4)
21-21: The computed expression can be simplified without the use of a string literal.
23-23: The computed expression can be simplified without the use of a string literal.
1-1: All these imports are only used as types.
1-8: Some named imports are only used as types.
templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts (6)
56-59: Template literals are preferred over string concatenation.
94-94: This variable implicitly has the any type.
1-7: Some named imports are only used as types.
7-14: Some named imports are only used as types.
15-16: All these imports are only used as types.
16-22: Some named imports are only used as types.
templates/types/streaming/nextjs/app/api/chat/stream-helper.ts (4)
21-21: The computed expression can be simplified without the use of a string literal.
23-23: The computed expression can be simplified without the use of a string literal.
1-1: All these imports are only used as types.
1-8: Some named imports are only used as types.
templates/types/streaming/nextjs/app/components/ui/chat/chat-resources.tsx (1)
2-3: All these imports are only used as types.
Additional comments not posted (2)
templates/types/streaming/nextjs/app/components/ui/chat/chat-resources.tsx (1)
2-3: Consider using TypeScript'stypeimport forCsvDataif it's only used for type checking to optimize bundling.- import { CsvData } from "."; + import type { CsvData } from ".";Likely invalid or redundant comment.
templates/types/streaming/nextjs/app/api/chat/stream-helper.ts (1)
85-92: Consider defining the default timeout value as a constant for better readability and maintainability.+ const DEFAULT_TIMEOUT = 1000 * 60 * 5; // 5 minutes const timeout = Number(process.env.STREAM_TIMEOUT ?? DEFAULT_TIMEOUT);Likely invalid or redundant comment.
| "Use the following CSV data:\n" + | ||
| "```csv\n" + | ||
| additionalData.uploadedCsv.content + | ||
| "\n```"; |
There was a problem hiding this comment.
Use template literals for better performance and readability in CSV content handling.
- "Use the following CSV data:\n" +
- "```csv\n" +
- additionalData.uploadedCsv.content +
- "\n```";
+ `Use the following CSV data:\n\`\`\`csv\n${additionalData.uploadedCsv.content}\n\`\`\``| "Use the following CSV data:\n" + | ||
| "```csv\n" + | ||
| additionalData.uploadedCsv.content + | ||
| "\n```"; |
There was a problem hiding this comment.
Use template literals for better performance and readability in CSV content handling.
- "Use the following CSV data:\n" +
- "```csv\n" +
- additionalData.uploadedCsv.content +
- "\n```";
+ `Use the following CSV data:\n\`\`\`csv\n${additionalData.uploadedCsv.content}\n\`\`\``| export function createStreamTimeout(stream: StreamData) { | ||
| const timeout = Number(process.env.STREAM_TIMEOUT ?? 1000 * 60 * 5); // default to 5 minutes | ||
| const t = setTimeout(() => { | ||
| appendEventData(stream, `Stream timed out after ${timeout / 1000} seconds`); | ||
| stream.close(); | ||
| }, timeout); | ||
| return t; | ||
| } |
There was a problem hiding this comment.
Define the default timeout value as a constant for better readability and maintainability.
+ const DEFAULT_TIMEOUT = 1000 * 60 * 5; // 5 minutes
const timeout = Number(process.env.STREAM_TIMEOUT ?? DEFAULT_TIMEOUT);Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
| export function createStreamTimeout(stream: StreamData) { | |
| const timeout = Number(process.env.STREAM_TIMEOUT ?? 1000 * 60 * 5); // default to 5 minutes | |
| const t = setTimeout(() => { | |
| appendEventData(stream, `Stream timed out after ${timeout / 1000} seconds`); | |
| stream.close(); | |
| }, timeout); | |
| return t; | |
| } | |
| export function createStreamTimeout(stream: StreamData) { | |
| const DEFAULT_TIMEOUT = 1000 * 60 * 5; // 5 minutes | |
| const timeout = Number(process.env.STREAM_TIMEOUT ?? DEFAULT_TIMEOUT); | |
| const t = setTimeout(() => { | |
| appendEventData(stream, `Stream timed out after ${timeout / 1000} seconds`); | |
| stream.close(); | |
| }, timeout); | |
| return t; | |
| } |
There was a problem hiding this comment.
Actionable comments posted: 0
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- templates/types/streaming/express/src/controllers/llamaindex-stream.ts (2 hunks)
- templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts (2 hunks)
Additional Context Used
Biome (12)
templates/types/streaming/express/src/controllers/llamaindex-stream.ts (6)
55-58: Template literals are preferred over string concatenation.
93-93: This variable implicitly has the any type.
1-7: Some named imports are only used as types.
7-14: Some named imports are only used as types.
15-16: All these imports are only used as types.
16-22: Some named imports are only used as types.
templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts (6)
55-58: Template literals are preferred over string concatenation.
93-93: This variable implicitly has the any type.
1-7: Some named imports are only used as types.
7-14: Some named imports are only used as types.
15-16: All these imports are only used as types.
16-22: Some named imports are only used as types.
Additional comments not posted (6)
templates/types/streaming/express/src/controllers/llamaindex-stream.ts (3)
55-58: Use template literals for better performance and readability in CSV content handling.
71-71: The integration ofDataParserOptionsincreateParserfunction looks good.
118-118: The updates toLlamaIndexStreamto useDataParserOptionsare correctly implemented.templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts (3)
55-58: Use template literals for better performance and readability in CSV content handling.
71-71: The integration ofDataParserOptionsincreateParserfunction looks good.
118-118: The updates toLlamaIndexStreamto useDataParserOptionsare correctly implemented.
There was a problem hiding this comment.
Actionable comments posted: 1
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- templates/types/streaming/nextjs/app/components/ui/chat/csv-content.tsx (1 hunks)
Additional Context Used
Biome (2)
templates/types/streaming/nextjs/app/components/ui/chat/csv-content.tsx (2)
6-6: Template literals are preferred over string concatenation.
1-1: All these imports are only used as types.
Additional comments not posted (1)
templates/types/streaming/nextjs/app/components/ui/chat/csv-content.tsx (1)
1-1: The import is used correctly as a type, which is a common practice in TypeScript.
| @@ -13,6 +15,7 @@ export interface ChatInputProps { | |||
| handleSubmit: (e: React.FormEvent<HTMLFormElement>) => void; | |||
| isLoading: boolean; | |||
| multiModal?: boolean; | |||
Summary by CodeRabbit
New Features
Improvements
Chores
.gitignoreto include additional directories and environment files.STREAM_TIMEOUTto manage stream response times.