A fatal error occurred: Extractors for 'go' are found in several same-priority locations

[tconnolly11]

I am following the directions listed here.

I am doing this on Mac, running 10.15.7.

These are the steps I've taken:

1. Downloaded CodeQL prebuilt binary (2.5.4) from here
2. Extracted CodeQL into ~/code-ql
3. Verified codeql CLI is working:
   codeql % codeql version CodeQL command-line toolchain release 2.5.4. Copyright (C) 2019-2021 GitHub, Inc. Unpacked in: /Users/tc/codeql Analysis results depend critically on separately distributed query and extractor modules. To list modules that are visible to the toolchain, use 'codeql resolve qlpacks' and 'codeql resolve languages'.
4. Obtained local copies of both codeql-repo & codeql-go and installed them in ~/codeql/codeql-repo ~/codeql/codeql-go respectiveley
5. Ran make inside of ~/codeql-go
6. Ran codeql resolve qlpacks which resulted in legacy-upgrades (/Users/tc/codeql/legacy-upgrades)
7. Ran codeql resolve languages which resulted in:

cpp (/Users/tc/codeql/cpp)
csharp (/Users/tc/codeql/csharp)
csv (/Users/tc/codeql/csv)
go is found in 2 same-priority locations, so attempts to resolve it will fail:
   /Users/tc/codeql/go
   /Users/tc/codeql/codeql-go
html (/Users/tc/codeql/html)
java (/Users/tc/codeql/java)
javascript (/Users/tc/codeql/javascript)
properties (/Users/tc/codeql/properties)
python (/Users/tc/codeql/python)
xml (/Users/tc/codeql/xml)

8. Attempted to do codeql database create appdb --language=go within the root directory of my Go project which resulted in:

Initializing database at /Users/tc/go/horizon/appdb
A fatal error occurred: Extractors for 'go' are found in several same-priority locations. It is unclear which to use:
   /Users/tc/codeql/go
   /Users/tc/codeql/codeql-go

Is there something I'm missing? At a minimum, I think the documentation here could be updated to make it more clear. Thanks!

[dbartol]

If I understand correctly, you copied the codeql and codeql-go repos into a directory that's underneath the directory in which you installed the CodeQL CLI? If that's the case, that's probably what's causing the problem. The repos can be in peer directories of the CLI, but not in a subdirectory of the CLI.

[tconnolly11]

Thanks Dave - that worked! The documentation here should probably be clarified to relay that. The docs there instruct you to put the codeql-repo in codeql-home where you are instructed to put the codeql CLI binary in section #2 (https://codeql.github.com/docs/codeql-cli/getting-started-with-the-codeql-cli/#create-a-new-codeql-directory) 

[smowton]

The docs currently say

The cloned repositories should have a sibling relationship. For example, if the root of the cloned CodeQL repository is $HOME/codeql-home/codeql-repo, then the root of the cloned CodeQL for Go repository should be $HOME/codeql-home/codeql-go.

Can you suggest an improvement to the wording?

[tconnolly11]

I'd suggest the following, since in Section 2, you instruct the user to put the CLI in $HOME/codeql-home:

The cloned repositories should have a sibling relationship, and must not be in the same directory as the CLI. For example, if the root of the codeql-cli is $HOME/codeql-home and the root of the cloned CodeQL repository is $HOME/codeql-repo, then the root of the cloned CodeQL for Go repository should be $HOME/codeql-go.

[smowton]

That step is supposed to specify creating a new, empty directory, not unpacking the CLI.

Currently it says

Create a new directory where you can place the CLI and any queries and libraries you want to use. For example, $HOME/codeql-home.

How about changing that to

Create a new, empty directory where, in steps 3 and 4, you will place the CLI and any queries and libraries you want to use. For example, mkdir $HOME/codeql-home.

[tconnolly11]

From what @dbartol said the whole issue that I ran into is because the codeql-repo & codeql-go query repositories cannot exist within the same directory as the CLI. So I'm confused as to why you still want the documentation to instruct the user to do that?

[intrigus-lgtm]

This is the intended structure as far as I know.

[smowton]

That's correct. "Where you will place the CLI" is supposed to mean you're going to do cd $CODEQL_HOME; unzip cli.zip, which will create the structure you see there (actually the CLI dir will be called codeql not codeql-cli)

How about amending step 4 to say:

For example, if the path to your copy of the CodeQL repository is $HOME/codeql-home/codeql-repo, then extract the CLI zip file into $HOME/codeql-home/. This should result in a subdirectory $HOME/codeql-home/codeql containing the CLI binary (also named codeql) and its support files.

[adityasharad]

Closing: I believe the docs now explain the intended layout, and since we have merged codeql-go into codeql there will soon be no need to check out both repos.