Skip to content

chore: create-llama monorepo #581

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Apr 25, 2025
Merged

chore: create-llama monorepo #581

merged 21 commits into from
Apr 25, 2025

Conversation

thucpn
Copy link
Collaborator

@thucpn thucpn commented Apr 25, 2025

Summary by CodeRabbit

  • Chores
    • Updated repository structure to a monorepo, introducing a new root configuration and workspace setup.
    • Added and refined ignore files to manage generated and environment-specific files for both the monorepo and subpackages.
    • Adjusted project scripts and workflows to align with the new monorepo structure and directory organization.
  • New Features
    • Introduced a dedicated package manifest for the "create-llama" package, including scripts and dependency management.
  • Style
    • Simplified and reorganized ignore patterns for improved clarity and maintainability.

@thucpn thucpn requested a review from marcusschiesser April 25, 2025 05:10
Copy link

changeset-bot bot commented Apr 25, 2025

🦋 Changeset detected

Latest commit: 1eb52e6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
create-llama Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link

coderabbitai bot commented Apr 25, 2025

Walkthrough

This update restructures the repository into a monorepo centered around the "create-llama" package. The root package.json is refactored for monorepo management, and a dedicated packages/create-llama directory is introduced with its own configuration and ignore rules. Multiple GitHub Actions workflows are updated to reflect new directory structures, particularly for Python and TypeScript packages. The .gitignore is simplified at the root and a new, more targeted ignore file is added for the package. End-to-end and linting workflows are scoped to the appropriate subdirectories, and a changeset documents the monorepo creation.

Changes

File(s) Change Summary
.changeset/plenty-spies-tickle.md Added changeset documenting patch-level chore for "create-llama" monorepo creation.
.github/workflows/e2e.yml
.github/workflows/lint_on_push_or_pull.yml
.github/workflows/release_llama_index_server.yml
.github/workflows/test_llama_index_server.yml
Updated workflow paths, working directories, and artifact locations to reflect new monorepo and subdirectory structure.
.gitignore Simplified by removing various ignore patterns for testing, build, and Python artifacts.
packages/create-llama/.gitignore Added comprehensive ignore file for dependencies, build outputs, test artifacts, and environment files specific to the package.
.husky/pre-commit Updated Python formatting check path for pre-commit hook to target new package subdirectory.
package.json Refactored as monorepo root manifest: changed name, made private, removed direct dependencies, added workspaces, updated scripts.
packages/create-llama/package.json Added new package manifest for "create-llama" with metadata, scripts, dependencies, and configuration.
packages/create-llama/e2e/typescript/resolve_dependencies.spec.ts Modified pnpm install command in test to use --ignore-workspace flag.

Sequence Diagram(s)

sequenceDiagram
    participant Developer
    participant Monorepo Root
    participant CreateLlamaPkg
    participant GitHub Actions

    Developer->>Monorepo Root: Runs workspace script (e.g., build, lint)
    Monorepo Root->>CreateLlamaPkg: Delegates command to package via pnpm workspace
    CreateLlamaPkg-->>Monorepo Root: Executes package-specific logic
    Developer->>GitHub Actions: Pushes code / opens PR
    GitHub Actions->>Monorepo Root: Triggers workflow
    GitHub Actions->>CreateLlamaPkg: Runs tests/lint/build in subdirectory
    CreateLlamaPkg-->>GitHub Actions: Returns results/artifacts
Loading

Possibly related PRs

Suggested reviewers

  • marcusschiesser

Poem

In the garden of code, a new home we sow,
Monorepo magic begins to grow.
Packages gather, scripts align,
Workflows updated, everything fine.
With each commit, the structure’s strong—
🐇 Hopping forward, we all belong!

✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@thucpn thucpn requested a review from marcusschiesser April 25, 2025 11:17
@thucpn thucpn marked this pull request as ready for review April 25, 2025 11:17
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (6)
packages/create-llama/.gitignore (2)

54-56: Fix duplicate and unusual Python cache patterns

There are a couple of issues with the Python ignore patterns:

  1. dist/ is duplicated (already appears on line 35)
  2. .__pycache__ (line 55) has an unusual leading dot - the standard pattern is just __pycache__
# Python
.mypy_cache/
venv/
.venv/
-dist/
-.__pycache__
__pycache__

1-62: Add newline at end of file

The file is missing a trailing newline at the end, which is a common best practice for text files.

Make sure the file ends with a newline character.

packages/create-llama/package.json (2)

51-57: Consider pinning dependency versions for better stability

Several dependencies use caret ranges (^) which could introduce breaking changes in minor version updates:

  • global-agent: ^3.0.0
  • ollama: ^0.5.0
  • ora: ^8.0.1
  • smol-toml: ^1.1.4
  • terminal-link: ^3.0.0

For a production CLI tool, consider pinning to exact versions for better stability.


22-34: Consider adding a general 'test' script

The scripts section is comprehensive with specific e2e test scripts, but there's no general test script that users can run with npm test. Consider adding one that runs all tests.

"scripts": {
  "build": "bash ./scripts/build.sh",
  "build:ncc": "pnpm run clean && ncc build ./index.ts -o ./dist/ --minify --no-cache --no-source-map-register",
  "clean": "rimraf --glob ./dist ./templates/**/__pycache__ ./templates/**/node_modules ./templates/**/poetry.lock",
  "dev": "ncc build ./index.ts -w -o dist/",
  "e2e": "playwright test",
  "e2e:python": "playwright test e2e/shared e2e/python",
  "e2e:typescript": "playwright test e2e/shared e2e/typescript",
+  "test": "pnpm run e2e",
  "format": "prettier --ignore-unknown --cache --check .",
  "format:write": "prettier --ignore-unknown --write .",
  "lint": "eslint . --ignore-pattern dist --ignore-pattern e2e/cache",
  "pack-install": "bash ./scripts/pack.sh"
},
.github/workflows/e2e.yml (1)

54-73: Suggest DRY-up repeated working-directory settings
You could reduce duplication by leveraging:

jobs:
  e2e-python:
    defaults:
      run:
        shell: bash
        working-directory: packages/create-llama
    # ...

Then omit working-directory in individual steps. This simplifies the workflow and eases future maintenance.

Also applies to: 124-142

package.json (1)

6-9: Keywords set; consider adding more
The tags "rag" and "llamaindex" fit the context. Optionally, you might include "monorepo" or "pnpm-workspace" to improve discoverability.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 53e1cd5 and 1eb52e6.

⛔ Files ignored due to path filters (3)
  • packages/create-llama/templates/components/agents/python/form_filling/sec_10k_template.csv is excluded by !**/*.csv
  • packages/create-llama/templates/components/agents/typescript/form_filling/sec_10k_template.csv is excluded by !**/*.csv
  • packages/create-llama/templates/components/data/101.pdf is excluded by !**/*.pdf
📒 Files selected for processing (11)
  • .changeset/plenty-spies-tickle.md (1 hunks)
  • .github/workflows/e2e.yml (5 hunks)
  • .github/workflows/lint_on_push_or_pull.yml (1 hunks)
  • .github/workflows/release_llama_index_server.yml (4 hunks)
  • .github/workflows/test_llama_index_server.yml (4 hunks)
  • .gitignore (0 hunks)
  • .husky/pre-commit (1 hunks)
  • package.json (1 hunks)
  • packages/create-llama/.gitignore (1 hunks)
  • packages/create-llama/e2e/typescript/resolve_dependencies.spec.ts (1 hunks)
  • packages/create-llama/package.json (1 hunks)
💤 Files with no reviewable changes (1)
  • .gitignore
⏰ Context from checks skipped due to timeout of 90000ms (25)
  • GitHub Check: typescript (22, 3.11, ubuntu-22.04, nextjs, --llamacloud)
  • GitHub Check: typescript (22, 3.11, ubuntu-22.04, nextjs, --example-file)
  • GitHub Check: typescript (22, 3.11, ubuntu-22.04, nextjs, --no-files)
  • GitHub Check: typescript (22, 3.11, windows-latest, nextjs, --llamacloud)
  • GitHub Check: typescript (22, 3.11, windows-latest, nextjs, --example-file)
  • GitHub Check: typescript (22, 3.11, windows-latest, nextjs, --no-files)
  • GitHub Check: typescript (22, 3.11, macos-latest, nextjs, --llamacloud)
  • GitHub Check: typescript (22, 3.11, macos-latest, nextjs, --example-file)
  • GitHub Check: typescript (22, 3.11, macos-latest, nextjs, --no-files)
  • GitHub Check: typescript (20, 3.11, ubuntu-22.04, nextjs, --llamacloud)
  • GitHub Check: python (20, 3.11, ubuntu-22.04, fastapi, --llamacloud)
  • GitHub Check: typescript (20, 3.11, ubuntu-22.04, nextjs, --example-file)
  • GitHub Check: python (20, 3.11, ubuntu-22.04, fastapi, --example-file)
  • GitHub Check: typescript (20, 3.11, ubuntu-22.04, nextjs, --no-files)
  • GitHub Check: typescript (20, 3.11, windows-latest, nextjs, --llamacloud)
  • GitHub Check: python (20, 3.11, windows-latest, fastapi, --llamacloud)
  • GitHub Check: typescript (20, 3.11, windows-latest, nextjs, --example-file)
  • GitHub Check: python (20, 3.11, windows-latest, fastapi, --example-file)
  • GitHub Check: typescript (20, 3.11, windows-latest, nextjs, --no-files)
  • GitHub Check: python (20, 3.11, windows-latest, fastapi, --no-files)
  • GitHub Check: typescript (20, 3.11, macos-latest, nextjs, --llamacloud)
  • GitHub Check: typescript (20, 3.11, macos-latest, nextjs, --example-file)
  • GitHub Check: Unit Tests (windows-latest, 3.9)
  • GitHub Check: python (20, 3.11, macos-latest, fastapi, --example-file)
  • GitHub Check: typescript (20, 3.11, macos-latest, nextjs, --no-files)
🔇 Additional comments (36)
packages/create-llama/.gitignore (1)

1-62: Overall gitignore structure is well-organized

The gitignore file comprehensively covers all necessary patterns for a Node.js/Next.js project with Python components, organized with clear section headers.

packages/create-llama/package.json (1)

1-79: Well-structured package.json for monorepo setup

The package.json is well-organized with all necessary metadata, scripts, and dependencies for the package within the monorepo structure.

.changeset/plenty-spies-tickle.md (1)

1-6: Properly formatted changeset for the monorepo restructuring

The changeset correctly documents the patch-level chore for creating the monorepo structure. This is appropriate for tracking the non-breaking structural change.

.husky/pre-commit (1)

3-3: Path correctly updated for monorepo structure

The path update for the uvx ruff format command properly reflects the new location of templates in the monorepo structure.

packages/create-llama/e2e/typescript/resolve_dependencies.spec.ts (1)

77-77: Good addition of the --ignore-workspace flag for monorepo compatibility

This change correctly adds the --ignore-workspace flag to the pnpm install command, which is essential in a monorepo setup. The flag ensures that the e2e tests install dependencies directly from the registry rather than using workspace references, providing proper isolation for testing.

.github/workflows/lint_on_push_or_pull.yml (2)

38-38: Correctly updated Python linting path

The source path has been properly updated to target Python code in its new monorepo location.


44-44: Correctly updated Python format check path

The source path has been properly updated to target Python code in its new monorepo location.

.github/workflows/test_llama_index_server.yml (4)

16-16: Correctly updated unit-test working directory path

The working directory has been properly updated to reflect the new monorepo structure.


53-53: Correctly updated type-check working directory path

The working directory has been properly updated to reflect the new monorepo structure.


85-85: Correctly updated build working directory path

The working directory has been properly updated to reflect the new monorepo structure.


111-111: Correctly updated artifact upload path

The artifact upload path has been properly updated to reflect the new monorepo structure.

.github/workflows/release_llama_index_server.yml (4)

8-8: Correctly updated path filter for workflow trigger

The path filter has been properly updated to reflect the new location of the llama-index-server package in the monorepo.


22-22: Correctly updated working directory for release job

The working directory has been properly updated to reflect the new monorepo structure.


83-83: Correctly updated working directory for publish job

The working directory has been properly updated to reflect the new monorepo structure.


117-117: Correctly updated package directory for PyPI publishing

The package directory path has been properly updated to reflect the new monorepo structure.

.github/workflows/e2e.yml (12)

1-1: Workflow name updated to reflect package context
Good job renaming the workflow to "E2E Tests for create-llama package", which clearly indicates its scope.


6-6: Paths-ignore updated to bypass Python server changes
Ignoring python/llama-index-server/** prevents unnecessary E2E runs when only the Python server code changes.

Also applies to: 10-10


54-55: Ensure Playwright installation runs in the package directory
Using working-directory: packages/create-llama here correctly scopes the Playwright browser setup to the create-llama package.


58-59: Build step scoped to the package
Great use of working-directory: packages/create-llama to execute the build within the correct workspace.


62-63: Pack-install step scoped to the package
The pack-install script is now correctly invoked inside the create-llama package directory.


73-73: Execute Python E2E tests within the package
The test runner is now correctly pointed at packages/create-llama, ensuring isolation.


79-79: Upload artifact path updated
The report upload path now matches the package structure at packages/create-llama/playwright-report/.


124-125: Install Playwright for TypeScript scoped to package
Good catch adding working-directory: packages/create-llama for the TypeScript browser install step.


128-129: TypeScript build step correctly scoped
The build command is now properly run inside the package directory.


132-133: Pack-install for TypeScript in correct context
Ensuring pack-install runs within packages/create-llama is spot on.


141-142: Execute TypeScript E2E tests in the package
Using the package directory for the E2E test run aligns with the monorepo layout.


147-147: Artifact upload path for TypeScript updated
The artifacts are now uploaded from packages/create-llama/playwright-report/, matching the workspace structure.

package.json (9)

2-5: Monorepo root manifest correctly defined
The name, version, private, and description fields reflect the new monorepo structure.


10-13: Repository information updated
The repository block points to the correct Git URL and type.


14-14: License declared as MIT
The open-source licensing is clearly stated.


15-17: Workspace globs configured
["packages/*"] correctly scopes your workspaces.


19-23: Verify Husky prepare script
The prepare script invokes husky, but Husky v7+ recommends husky install to initialize Git hooks. Please confirm your current setup or consider updating to:

"prepare": "husky install"

24-29: Confirm workspace filter targets
The --filter create-llama flag assumes your workspace’s name is exactly "create-llama". Please verify in packages/create-llama/package.json that this matches, or adjust to:

"build": "pnpm -r --filter ./packages/create-llama build"

31-34: DevDependencies trimmed for root
Only @changesets/cli and husky remain at the root, delegating other tools to workspace packages. This aligns well with your monorepo approach.


35-35: Pinning pnpm version
packageManager is set to [email protected], ensuring consistent installs across environments.


36-38: Node engine requirement
Specifying node >=16.14.0 ensures compatibility with modern Node features.

@thucpn thucpn merged commit 3960618 into main Apr 25, 2025
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants