Bad error message when 00-index exists but 01-index does not

[ezyang]

Summary. To understand this problem, it's necessary to know a little bit about how the Hackage index (the directory of all Hackage packages you may be able to install via cabal install) works. Previously, cabal downloaded from Hackage 00-index.tar, which basically was a giant tarball with all the Cabal files on Hackage. This required us to download the ENTIRE tarball every time we did a Hackage update, which mean cabal update got slower, and slower, as the Hackage index increased in size.

Two years ago, Hackage gained a new format, 01-index.tar, which is append-only. It's described in this post: https://www.well-typed.com/blog/2015/08/hackage-security-beta/ The upshot is (1) now you can incrementally download updates to the Hackage index, and (2) it's a backwards incompatible format. Last year (during ICFP, no less!) cabal-install turned on Hackage security by default, which means that it switched to using 01-index.tar; e.g., when you run cabal update, you only update 01-index.tar. The new support worked well, but there are some rough edges when switching between cabal which read/writes 00-index.tar, and a cabal which reads/writes 01-index.tar. This is one example of such an infelicity.

Steps to reproduce:

1. You'll need two ways to invoke cabal, one where Hackage security is on and another where it is off. One easy way to make this be the case is to use HEAD (Hackage security is ON) and 1.24 (Hackage security is OFF). For all subsequent steps, specify a HOME directory with no .cabal
2. Run cabal update with the security turned off. This will give you 00-index
3. Try to run the dep solver with security turned on.

Expected result: Either dep solver will work, OR it tells me to run cabal update. (Even better if it tells me that it needs to redownload index because security was enabled.)

Actual result: I get:

cabal: Could not read index. Did you call 'checkForUpdates'?

How to fix. It is instructive to compare the bug above with what happens normally when an index does not exist:

ezyang@sabre:~/Dev/labs/f$ HOME=`pwd` cabal install happy
Config file path source is default config file.
Config file /home/ezyang/Dev/labs/f/.cabal/config not found.
Writing default configuration to /home/ezyang/Dev/labs/f/.cabal/config
Warning: The package list for 'hackage.haskell.org' does not exist. Run 'cabal
update' to download it.
cabal: There is no package named 'happy'.
You may need to run 'cabal update' to get the latest list of available
packages.

By grepping, you can find the code responsible for emitting these warnings.

Unlike many of the bugs here, fixing this bug involves making modifications to hackage-security, so what you will first need to do is setup your dev environment to let you make modifications to hackage-security and have them show up in Cabal. This is pretty easy to do with new-build: clone hackage-security to a directory of your choice (inside your Cabal checkout is a good pick) and then edit cabal.project.local and add packages: path/to/hackage-security.

Things to follow up. What happens if I run cabal update with HEAD cabal-install, and then try to use an older HEAD?

[ezyang]

CC @dcoutts and @edsko.

[ezyang]

@dcoutts: @bollu expressed interest in trying to fix this, so put it on the back burner of your priority queue ;)

[ezyang]

I don't think anyone is actively working on this atm.

[ezyang]

Apparently, this is fixed, since I can't reproduce anymore.