feat: Add template for structured extraction #185
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🦋 Changeset detectedLatest commit: a4d6145 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
WalkthroughThe recent changes introduce a new structured extraction template and expand the functionality of the existing codebase to accommodate this addition. Key modifications include the introduction of new types and enhancements to functions that handle template installation and querying. This update aims to improve data management practices, streamline workflows, and enrich user interaction with a new API built on FastAPI. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant API
participant QueryEngine
participant Output
User->>API: Send query request
API->>QueryEngine: Process request data
QueryEngine->>Output: Generate response
Output-->>API: Return structured data
API-->>User: Provide response
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Outside diff range, codebase verification and nitpick comments (4)
templates/types/extractor/fastapi/app/api/routers/output.py (1)
5-5: Consider using a more specific logger name.Using a more specific logger name can help differentiate logs from different modules.
- logger = logging.getLogger("uvicorn") + logger = logging.getLogger(__name__)templates/types/extractor/fastapi/main.py (1)
19-20: Consider using a more specific logger name.Using a more specific logger name can help differentiate logs from different modules.
- logger = logging.getLogger("uvicorn") + logger = logging.getLogger(__name__)templates/types/extractor/fastapi/app/api/routers/extractor.py (1)
40-50: Clarify the error message.The error message should be more user-friendly and avoid referencing specific commands.
- detail=str( - "StorageContext is empty - call 'poetry run generate' to generate the storage first" - ) + detail="Storage context is empty. Please generate the storage first."templates/types/extractor/fastapi/README-template.md (1)
58-60: Specify the language for the fenced code block.To comply with markdown linting rules, specify the language for the fenced code block.
-``` +```shell ENVIRONMENT=prod python main.py<details> <summary>Tools</summary> <details> <summary>Markdownlint</summary><blockquote> 58-58: null Fenced code blocks should have a language specified (MD040, fenced-code-language) </blockquote></details> </details> </blockquote></details> </blockquote></details> <details> <summary>Review details</summary> **Configuration used: CodeRabbit UI** **Review profile: CHILL** <details> <summary>Commits</summary> Files that changed from the base of the PR and between a553d5051e1d158f0f94f619c0320d797f9034bb and a4d6145bfbaca594397d3133515b685fb6c50681. </details> <details> <summary>Files selected for processing (9)</summary> * .changeset/proud-seals-yell.md (1 hunks) * helpers/index.ts (1 hunks) * helpers/types.ts (1 hunks) * questions.ts (2 hunks) * templates/types/extractor/fastapi/README-template.md (1 hunks) * templates/types/extractor/fastapi/app/api/routers/extractor.py (1 hunks) * templates/types/extractor/fastapi/app/api/routers/output.py (1 hunks) * templates/types/extractor/fastapi/main.py (1 hunks) * templates/types/extractor/fastapi/pyproject.toml (1 hunks) </details> <details> <summary>Files skipped from review due to trivial changes (2)</summary> * .changeset/proud-seals-yell.md * templates/types/extractor/fastapi/pyproject.toml </details> <details> <summary>Additional context used</summary> <details> <summary>Ruff</summary><blockquote> <details> <summary>templates/types/extractor/fastapi/main.py</summary><blockquote> 43-43: Remove unnecessary `True if ... else False` Remove unnecessary `True if ... else False` (SIM210) </blockquote></details> </blockquote></details> <details> <summary>Markdownlint</summary><blockquote> <details> <summary>templates/types/extractor/fastapi/README-template.md</summary><blockquote> 58-58: null Fenced code blocks should have a language specified (MD040, fenced-code-language) </blockquote></details> </blockquote></details> </details> <details> <summary>Additional comments not posted (21)</summary><blockquote> <details> <summary>templates/types/extractor/fastapi/app/api/routers/output.py (2)</summary><blockquote> `8-22`: **LGTM!** The `Output` class is well-defined with appropriate fields and validation. --- `24-32`: **LGTM!** The `Config` class provides a useful example for the JSON schema. </blockquote></details> <details> <summary>templates/types/extractor/fastapi/main.py (7)</summary><blockquote> `1-3`: **LGTM!** Environment variables are loaded using `dotenv`. --- `5-10`: **LGTM!** Imports and initial setup for FastAPI and logging are correct. --- `11-13`: **LGTM!** The extractor router and settings are correctly imported. --- `15-18`: **LGTM!** FastAPI app instance and settings initialization are correctly done. --- `22-30`: **LGTM!** CORS middleware is correctly configured for development mode. --- `32-35`: **LGTM!** Redirection to the documentation page is correctly implemented. --- `38-38`: **LGTM!** The extractor router is correctly included. </blockquote></details> <details> <summary>templates/types/extractor/fastapi/app/api/routers/extractor.py (7)</summary><blockquote> `1-3`: **LGTM!** Imports for logging and environment variables are correct. --- `4-7`: **LGTM!** Imports for FastAPI, HTTPException, and Pydantic are correct. --- `8-10`: **LGTM!** Imports for the output model and index function are correct. --- `11-13`: **LGTM!** API router instance and logger are correctly initialized. --- `16-24`: **LGTM!** The `RequestData` class is well-defined with appropriate fields and validation. --- `27-37`: **LGTM!** The `query_request` endpoint is well-defined and correctly processes the request. --- `52-58`: **LGTM!** The `get_query_engine` function is well-defined and correctly configures the query engine. </blockquote></details> <details> <summary>helpers/types.ts (1)</summary><blockquote> `21-21`: **LGTM! The new template type "extractor" enhances flexibility.** The addition of the "extractor" type to `TemplateType` looks good and enhances the flexibility of the type definition. </blockquote></details> <details> <summary>templates/types/extractor/fastapi/README-template.md (1)</summary><blockquote> `1-68`: **LGTM! The README provides clear and comprehensive instructions.** The instructions for setting up and running the FastAPI project with LlamaIndex are clear and comprehensive. <details> <summary>Tools</summary> <details> <summary>Markdownlint</summary><blockquote> 58-58: null Fenced code blocks should have a language specified (MD040, fenced-code-language) </blockquote></details> </details> </blockquote></details> <details> <summary>helpers/index.ts (1)</summary><blockquote> `166-170`: **LGTM! The conditional logic now supports the "extractor" template type.** The modification to include the "extractor" template type in the conditional logic of the `installTemplate` function looks good and broadens the functionality. </blockquote></details> <details> <summary>questions.ts (2)</summary><blockquote> `345-348`: **Addition of new template option "Structured Extractor".** The addition of the new option "Structured Extractor" is well-integrated into the existing list of template choices. --- `409-411`: **Modification of conditional logic to handle "extractor" template.** The conditional logic now appropriately includes a check for the "extractor" template, setting the framework to "fastapi" if the template is either "multiagent" or "extractor". </blockquote></details> </blockquote></details> </details> <!-- This is an auto-generated comment by CodeRabbit for review status -->
This was referenced Sep 24, 2024
This was referenced Oct 3, 2024
This was referenced Oct 17, 2024
Merged
This was referenced Nov 20, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit
New Features
Documentation
Bug Fixes