Implement index server

[GeekArthur]

Signed-off-by: jingfu wang [email protected]

Please specify the area for this PR

What does does this PR do / why we need it:
This PR implements the new index server, it also integrates with nginx server to serve non-registry API requests.

Which issue(s) this PR fixes:

Fixes devfile/api#188

PR acceptance criteria:

• Test (WIP)

• Documentation (WIP)

How to test changes / Special notes to the reviewer:
Deploy index server and registry as before, send GET request to endpoint /index or /devfiles/{devfile name}

[johnmcollier]

Left some review comments.

Another suggestion I had is that since this is called index-server, does it make sense to move it under the index/ folder in this repo?

e.g.

index/
   generator/
   server/

[GeekArthur]

Left some review comments.

Another suggestion I had is that since this is called index-server, does it make sense to move it under the index/ folder in this repo?

e.g.

index/
   generator/
   server/

I though about the same thing before, will do that.

[GeekArthur]

@johnmcollier comments addressed, please review again, thanks!

[johnmcollier]

Every time the index server starts for the first time, it exits with an error:

^CJohns-MacBook-Pro-3:kubernetes johncollier$ oc get po
NAME                                                   READY   STATUS    RESTARTS   AGE
devfile-registry-8467d84c6d-zrwgw                      2/2     Running   1          4m20s
registry-operator-controller-manager-9b6b557b4-79hpz   2/2     Running   0          15m

I think it's happening because of the log.Fatal we do if the http.Get to the registry server returns an error. Probably the registry server isn't quite ready yet and isn't giving back an http response code?)

I think it might be better if we don't exit when an http.Get returns an error, but rather keep trying up to a given timeout?

This is what I did in the registry operator when I needed to poll a given URL (see https://github.com/johnmcollier/registry-operator/blob/main/pkg/util/util.go#L25)

func WaitForServer(url string, timeout time.Duration) error {
	return wait.PollImmediate(time.Second, timeout, func() (bool, error) {
		tr := &http.Transport{
			TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
		}
		client := &http.Client{Transport: tr}
		resp, err := client.Get(url)
		if err != nil {
			return false, err
		}
		if resp.StatusCode == 200 {
			return true, nil
		}
		return false, nil
	})
}

I'd recommend setting the timeout to no more than 30 seconds, since any more and the Kube liveness probe will fail and kill the container. You may also want to change the time.Second in wait.PollImmediate to some multiple of time.Millisecond.

[GeekArthur]

@johnmcollier Can you help review again? Thanks

[GeekArthur]

@johnmcollier changes are pushed, please review.

[johnmcollier]

LGTM

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: GeekArthur, johnmcollier

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [GeekArthur,johnmcollier]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment