NoOp DB Migrations to allow for DB backports

[techknowlogick]

Due to our migrations table only keeping a count of the highest migration that has been run, this means that PRs with migrations cannot be backported as then the migration count would conflict if the administrator ever tries to upgrade to a newer major release.

The purpose of this PR is to add several (15 in this case) no-op migrations, so that the count would be artificially inflated to allow for migration backports.

Pretend Example:
v1.A - max migration = 100
v1.B - migrations start at 101 + 15 noops
An index is added to a large table in v1.B to improve speed of queries, as there are 15 noops this can become migration 101 in v1.A (as when upgrading to v1.B migration 101 would be skipped)

Note: This has a failure case of when a migration may attempted to be run twice on upgrade (the migration that is backported would have a different number, and so on upgrade it would be attempted to be run again), as well as a few others. There are some good suggestions below on improvements to this PR of tracking all migrations that are run to ensure they aren't run in duplicate, as well as a few others.

Alternative: #10625

[wxiaoguang]

It will output a lot of logs?

		log.Info("Migration[%d]: %s", v+int64(i), m.Description())

How about just use nil to fill the slice.

[lunny]

If I'm not wrong, these migrations are prepared for the future. But I don't think it will work well.

[wxiaoguang]

If I'm not wrong, these migrations are prepared for the future. But I don't think it will work well.

Agree, it needs a complete document to clarify all the cases. Otherwise bugs come ....

[lunny]

If I'm not wrong, these migrations are prepared for the future. But I don't think it will work well.

Agree, it needs a complete document to clarify all the cases. Otherwise bugs come ....

I think the problems is when user upgrade from an old version which have executed some placeholder migrations, it may encountered problems. Assume users are in v1.19.3 which will add a new column, but for v1.20.0 has no migration for this column but v1.20.3 has, when user upgrade from v1.19.3 to v1.20.0 not v1.20.3, it will encounter problems.

[techknowlogick]

Blocking per @lunny to ensure this is the correct approach.

[wxiaoguang]

The migration number has finished its job. To make the migration more stable, each migration task should has its own ID and the ID should be saved into database.

Then a migration task in 1.20 can be skipped if the same task has been executed in 1.19.

[lunny]

The migration number has finished its job. To make the migration more stable, each migration task should has its own ID and the ID should be saved into database.

Then a migration task in 1.20 can be skipped if the same task has been executed in 1.19.

Maybe a UUID generated manually? But it doesn't resolve my concern.

[wxiaoguang]

The migration number has finished its job. To make the migration more stable, each migration task should has its own ID and the ID should be saved into database.
Then a migration task in 1.20 can be skipped if the same task has been executed in 1.19.

Maybe a UUID generated manually? But it doesn't resolve my concern.

Make backported migrations also have the "minor target version" (or use binary build/release date, etc). For example: if a migration backported to 1.19.3 should declare that the next minor version should be not less than 1.20.3 for 1.20.x and not less than 1.21.1 for 1.21.x (suppose 1.22 is under development).

[wxiaoguang]

As the first step of improving the migration system, I think current number approach still works for most cases.

There are 2 kinds of migrations:

1. idempotent migration (or can be idempotent-like)
   • increase column size
   • add a column (skip it if the column already exists)
   • fix column value which is compatible for all versions
2. one-time migration
   • change column type to an incompatible one
   • update column value which causes incompatible data between versions
   • rename/drop column

I think most migrations are kind 1. So current simple approach could do its job.