Skip to content

Restore the prefix option #2982

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
gajus opened this issue May 31, 2025 · 0 comments
Open

Restore the prefix option #2982

gajus opened this issue May 31, 2025 · 0 comments

Comments

@gajus
Copy link

gajus commented May 31, 2025

Motivation

Having regrets after migrating from ioredis because of the lack of prefix option which is causing a ton of issues using Redis in CI.

The problem is that (in the CI) it is desirable to isolate tests, which we do in PostgreSQL by creating a dedicated database, and in Redis, we used to do it with a prefix.

In the absence of Redis prefix, we've resolved to using workarounds like the one that I documented in #1657:

/**
 * https://github.com/redis/node-redis/issues/1657#issuecomment-2873493460
 */
const createRedisDatabaseAssigner = async ({ url }: { url: string }) => {
  const redisMain = (await createClient({
    // We use database 0 as a main database for test run id storage.
    database: 0,
    url,
  }).connect()) as RedisClientType;

  return {
    assignDatabase: async ({ testRunId }: { testRunId: string }) => {
      let assignedDatabase = (await redisMain.get(
        `assigned-database:${testRunId}`,
      ))
        ? Number(await redisMain.get(`assigned-database:${testRunId}`))
        : null;

      let newAssignedDatabase = false;

      if (!assignedDatabase) {
        const lastDatabase = (await redisMain.get('last-database'))
          ? Number(await redisMain.get('last-database'))
          : 0;

        // By default there are 16 databases (indexed from 0 to 15) and
        // you can navigate between them using select command.
        // Beware that this means that this concurrency control will break
        // if we run more than 16 tests concurrently.
        assignedDatabase = lastDatabase > 14 ? 1 : lastDatabase + 1;

        await redisMain.set('last-database', assignedDatabase);
        await redisMain.set(`assigned-database:${testRunId}`, assignedDatabase);

        newAssignedDatabase = true;
      }

      const redis = (await createClient({
        database: assignedDatabase,
        url,
      }).connect()) as RedisClientType;

      if (newAssignedDatabase) {
        await redis.flushDb();
      }

      return redis;
    },
  };
};

However, this means that our test runners are limited to running at most 15 tests in parallel, which is far from ideal.

Meanwhile, having the ability to configure a prefix provides a very elegant way to isolate these environments.

Additionally, prefixes are necessary to clearly separate concerns of different parts of the application, e.g. web app from various background workers, etc. and the absence of prefix makes it extremely cumbersome and dependent on whatever libraries that we use.

Related issues:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant