telemetry breaks in GH CI

[aguschin]

reported in discord by @rocco-fortuna

Hi, I'm having issues loading a .mlem model from a cml github actions pipeline, even though the exact same code/data/model works perfectly locally (edited)

Traceback (most recent call last):
  File "./programming-language-classification/run_eval.py", line 16, in <module>
    mlem_model = load(model_path)
  File "/usr/local/lib/python3.8/dist-packages/mlem/core/metadata.py", line 156, in load
    meta = load_meta(
  File "/usr/local/lib/python3.8/dist-packages/mlem/telemetry.py", line 56, in inner
    telemetry.send_event(
  File "/usr/local/lib/python3.8/dist-packages/iterative_telemetry/__init__.py", line 125, in send_event
    self.send(
  File "/usr/local/lib/python3.8/dist-packages/iterative_telemetry/__init__.py", line 151, in send
    payload.update(self._runtime_info())
  File "/usr/local/lib/python3.8/dist-packages/iterative_telemetry/__init__.py", line 213, in _runtime_info
    ci_id = _generate_ci_id()
  File "/usr/local/lib/python3.8/dist-packages/iterative_telemetry/__init__.py", line 335, in _generate_ci_id
    res = generator()
  File "/usr/local/lib/python3.8/dist-packages/iterative_telemetry/__init__.py", line 286, in _generate_github_id
    user_id = subprocess.check_output(  # nosec B603, B607
  File "/usr/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 493, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'gh'
Error: Process completed with exit code 1.

[aguschin]

This fails on https://github.com/iterative/telemetry-python/blob/main/src/iterative_telemetry/__init__.py#L287

The line calls gh CLI but it can't be found. Judging by https://docs.github.com/en/actions/using-workflows/using-github-cli-in-workflows, it must be pre-installed in standard docker images like ubuntu-latest, etc.

@rocco-fortuna, are you using a custom docker image to run the workflow? Curios to see if this is the problem here.

The second thing: if fails, I see it returns None, but it could at least return group_id, which doesn't require running gh. This way we will have some way to differentiate between local and CI runs in such case. Minor though, I can it can be neglected.

[aguschin]

Fixing this in #56, once merged and released, it should catch that error as expected.

[rocco-fortuna]

I'm using cml:

jobs:
  deploy_runner:
    ...
    run:
      cml runner launch --cloud=gcp --cloud-region=us-west1-b --cloud-type=m+k80
  run-eval:
    needs: deploy-runner
    runs-on: [self-hosted, cml-runner]

    container:
      image: docker://iterativeai/cml:latest-gpu
    ...

So I'm guessing cml:latest-gpu does not have gh installed?

[aguschin]

Hey @0x2b3bfa0, is this expected?

[aguschin]

trying to release this, but waiting for pypa/gh-action-pypi-publish#130 to be fixed

[skshetry]

@aguschin, meanwhile you can try uploading yourself:

rm -rf dist
nox -s build
twine upload dist/*

[aguschin]

Nice @skshetry 🙌🏻 ! Just did that, https://pypi.org/project/iterative-telemetry/0.0.8/