Skip to content

fetch secrets from <workspace>/.continue/.env folder before <workspace>/.env #5461

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jun 2, 2025
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 96 additions & 0 deletions core/config/yaml/LocalPlatformClient.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
import { FQSN, SecretResult, SecretType } from "@continuedev/config-yaml";
import { testControlPlaneClient, testIde } from "../../test/fixtures";
import { LocalPlatformClient } from "./LocalPlatformClient";

jest.mock("../../util/paths.ts", () => ({
...jest.requireActual("../../util/paths"),
getContinueDotEnv: jest.fn(),
}));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey @tomasz-stefaniak

there is some problem when am trying to mock a local file as jest is unable to resolve it. it seems like a problem with how jest is setup. I tried various ways to get it working. Currently none of these work.

Any idea what should I do to a mock?

(I need to mock utils/paths for imported functions like getContinueDotEnv())

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@uinstinct have you checked some existing instances of jest.mock we have in the codebase?
image

I understand most of them are for node packages but maybe they can point you in the right direction. If you are still stuck, let me know and I can try to pull this into the sprint next week.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for replying Tomasz!
I had tried these actually but for some reason jest is unable to resolve it.

For alternative options, we can:

  1. use the filesystem env (problem: the test might affect the local users .env)
  2. use a conditional (process.env.TEST flag) inside getContinueDotEnv to produce a key value pair for testing

Copy link
Contributor Author

@uinstinct uinstinct May 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spent some more time debugging this. Looks like the module resolution was starting from core/test folder since jest was declared as global in core/test/jest.setup-after-env.js

So, ../util/paths would work but not ../../util/paths.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay. Tried both unstable_mockModule and jest.mock('../util/paths') both which resolve correctly but do not mock implementations for some reason.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: this is a huge pain, I'm exploring migrating to Vitest 😆

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥲 Agree!!!
BTW, can i help in some way? I have some experience with migrating jest to vitest.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be great, actually! Do you think you'd be able to create a basic vitest setup? It could be on this branch, and it would be used to run LocalPlatformClient.test.ts. Once this proof of concept works we can migrate other tests to vitest.

This would also require adding an extra step here: https://github.com/continuedev/continue/blob/main/.github/workflows/pr_checks.yaml

Let me know what you think!

Copy link
Contributor Author

@uinstinct uinstinct May 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

implemented!

  • vitest works nicely (and with mocks) 🥳
  • added vitest running for LocalPlatformClient only in pr_check.yaml

Edit: Should I draft another PR to migrate core tests to vitest?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edit: Should I draft another PR to migrate core tests to vitest?

Yeah, that would be great. It's one of our more important tasks at the moment.


describe("LocalPlatformClient", () => {
const testFQSN: FQSN = {
packageSlugs: [
{
ownerSlug: "test-owner-slug",
packageSlug: "test-package-slug",
},
],
secretName: "TEST_CONTINUE_SECRET_KEY",
};
const testResolvedFQSN: SecretResult = {
found: true,
fqsn: testFQSN,
secretLocation: {
secretName: testFQSN.secretName,
secretType: SecretType.Organization,
orgSlug: "test-org-slug",
},
};

afterEach(() => jest.restoreAllMocks());

test("should not be able to resolve FQSNs if they do not exist", async () => {
testControlPlaneClient.resolveFQSNs = jest.fn(async () => [
testResolvedFQSN,
]);
const localPlatformClient = new LocalPlatformClient(
null,
testControlPlaneClient,
testIde,
);
const resolvedFQSNs = await localPlatformClient.resolveFQSNs([testFQSN]);
expect(testControlPlaneClient.resolveFQSNs).toHaveBeenCalled();
expect(resolvedFQSNs).toEqual([testResolvedFQSN]);
});

test("should not be able to resolve FQSNs if they do not exist", async () => {
const localPlatformClient = new LocalPlatformClient(
null,
testControlPlaneClient,
testIde,
);
const resolvedFQSNs = await localPlatformClient.resolveFQSNs([testFQSN]);

expect(resolvedFQSNs.length).toBeGreaterThan(0);
expect(resolvedFQSNs[0]?.found).toBe(false);
});

describe.only("searches for secrets in local .env files", () => {
const secretValue = Math.floor(Math.random() * 100) + "";
const envKeyValues = {
TEST_CONTINUE_SECRET_KEY: secretValue,
};
const getContinueDotEnv = jest.fn(() => envKeyValues);

beforeEach(async () => {
// const directoryExists = fs.existsSync("./_tests");
// if (directoryExists) {
// fs.rmSync("./_tests", { force: true, recursive: true });
// }
// const envFileContent = `TEST_CONTINUE_SECRET_KEY=${secretValue}`;
// fs.writeFileSync("./_tests/.env", envFileContent);
// utilPaths.getContinueDotEnv = getContinueDotEnv
// const spy = jest.spyOn(utilPaths, 'getContinueDotEnv')
// Object.defineProperty(utilPaths, 'getContinueDotEnv', {
// writable: true,
// configurable: true,
// value: spy.mockImplementation(() => envKeyValues),
// });
// jest.mock(utilPaths, () => ({
// ...jest.requireActual(utilPaths),
// getContinueDotEnv: jest.fn().mockReturnValue(envKeyValues)
// }))
});

test("should be able to get secrets from global ~/.continue/.env files", async () => {
const localPlatformClient = new LocalPlatformClient(
null,
testControlPlaneClient,
testIde,
);
const resolvedFQSNs = await localPlatformClient.resolveFQSNs([testFQSN]);
expect(getContinueDotEnv).toHaveBeenCalled();
console.log("resolved->", resolvedFQSNs);
});
});
});
13 changes: 11 additions & 2 deletions core/config/yaml/LocalPlatformClient.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,16 @@ export class LocalPlatformClient implements PlatformClient {
private readonly ide: IDE,
) {}

/**
* searches for the first valid secret file in order of ~/.continue/.env, <workspace>/.continue/.env, <workspace>/.env
*/
private async findSecretInEnvFiles(
fqsn: FQSN,
): Promise<SecretResult | undefined> {
const secretValue =
this.findSecretInLocalEnvFile(fqsn) ??
(await this.findSecretInWorkspaceEnvFiles(fqsn));
(await this.findSecretInWorkspaceEnvFiles(fqsn, true)) ??
(await this.findSecretInWorkspaceEnvFiles(fqsn, false));

if (secretValue) {
return {
Expand Down Expand Up @@ -52,12 +56,17 @@ export class LocalPlatformClient implements PlatformClient {

private async findSecretInWorkspaceEnvFiles(
fqsn: FQSN,
insideContinue: boolean,
): Promise<string | undefined> {
try {
const workspaceDirs = await this.ide.getWorkspaceDirs();

for (const folder of workspaceDirs) {
const envFilePath = joinPathsToUri(folder, ".env");
const envFilePath = joinPathsToUri(
folder,
insideContinue ? ".continue" : "",
".env",
);
try {
const fileExists = await this.ide.fileExists(envFilePath);
if (fileExists) {
Expand Down
Loading