Skip to content

Migrate/mirror to Gitlab #238

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
1 of 3 tasks
MartinDelille opened this issue Jun 6, 2018 · 35 comments
Closed
1 of 3 tasks

Migrate/mirror to Gitlab #238

MartinDelille opened this issue Jun 6, 2018 · 35 comments

Comments

@MartinDelille
Copy link
Contributor

MartinDelille commented Jun 6, 2018

Since Microsoft is buying Github, it might be the time to think about a migration (or at least a mirror) to https://about.gitlab.com/

I use the community edition at work and it is awesome! (continuous integration in particular).

Any thought about this?

  • Mirror to Gitlab
  • Automate Gitlab mirroring
  • Set-up Gitlab CI
@wing328
Copy link
Member

wing328 commented Jun 6, 2018

Thanks for bringing it up. I've used gitlab before and the experience was pretty good. Here are some thoughts:

  • If the majority of Github users (or openapi-generator users) have switched to Gitlab, we should also host our repo in Gitlab (network effect)
  • Even though hosting the repo in Gitlab (or other service providers), there may be chances one day Gitlab (or other service providers) gets acquired by another company
  • We may want to see what's MS going to do with Github first. If let's say we start seeing advertisements promoting competing tech to OpenAPI Generator in our repo, we should definitely consider moving to Gitlab, Bitbucket or other alternatives

PS: I did work with a MS engineer to create the R generator (currently included in the OpenAPI Generator project) but I'm not sure whether that engineer did it on MS behalf or using his free time.

@jmini
Copy link
Member

jmini commented Jun 6, 2018

As long as the service is good and free for open-source I do not see a reason to move away from GitHub.

Microsoft as company did a lot to support Open-Source in the last years (typescript, vscode, linux kernel, ...). It is no longer the same company as in the 90ties.

@macjohnny
Copy link
Member

I am strongly in favor of using GitHub as a platform, since most developers already have an account and it greatly enhances chances for initial contributions.

@jimschubert
Copy link
Member

I don't know why nobody has focused on the mirroring recommendation. I mirror projects that I publish to maven from GitHub to GitLab and I use GitLab's pipelines for automated testing and deployment. Not only do they have a significantly better issue tracker, project management, permissions structure, ability to lock files, email-based "Service Desk", integrated pipelines, container registry, and work out in the open when troubleshooting issues… they also turn around fixes and new features much faster than GitHub.

While it's true that remaining on GitHub means a much bigger community, not mirroring to GitLab means we're excluding all engineers who have jumped ship because they don't want their codebases housed on a system that is owned and operated by Microsoft (the "bring no $ to Microsoft" argument). I understand the irony in the fact that GitLab is hosted on Azure, but I'm also not jumping ship… I have accounts on both systems.

GitLab announced yesterday that their Ultimate and Gold plans would be free for open source projects which don't bring in profit from the resulting software. This isn't something we have with GitHub. See the blog post: https://about.gitlab.com/2018/06/05/gitlab-ultimate-and-gold-free-for-education-and-open-source/

I'm not advocating for migrating to GitLab, I just want to point out the positives to having a mirror. I personally prefer GitLab and I think it would be a much better platform if there were more people using it.

I think the "Since Microsoft is buying Github" may have been taken out of the intended context of this issue; Martin is French and a direct translation to "Since" in English has some negative connotations that I don't think it has in French. If you read it as "Let's ditch GitHub, cuz Microsoft sucks" then that is faulty reasoning. But if you read it as "Now that Microsoft is buying GitHub and GitLab is making a switch very enticing", then I think not considering at least a mirror would be short-sighted. For instance, if they find it unsustainable to continue offering Ultimate/Gold plans for open source projects… not opening a mirror means missing out on some excellent features.

@jimschubert
Copy link
Member

Another potential positive of mirroring to GitLab is that our Travis pipeline currently spends about 25 minutes doing simple machine prep. On GitLab, we could easily refer to a pre-configured build container. I've looked into how to set this up on Travis CI and it doesn't seem to be as clear how to do this (if it is even possible).

If we could move some or all of the pipeline to GitLab, I think it would speed builds and simplify the operational overhead.

@MartinDelille
Copy link
Contributor Author

Gitlab CI is awesome !

@philsturgeon
Copy link
Contributor

There is absolutely no reason to do this other than odd personal preferences. 🤷‍♂️

@philsturgeon
Copy link
Contributor

There are alternatives to Travis if Travis is slow for your pipeline.

@jimschubert
Copy link
Member

@philsturgeon you're saying there's absolutely no reason to migrate, or absolutely no reason to mirror?

I agree there's currently no reason to migrate. However, possible reasons to migrate in the future have already been mentioned in this thread. Most other comments in this thread are either agreeing that migration isn't currently necessary or are focused on potentially mirroring. I think I'm the only one who has expressed a personal preference of GitLab over GitHub, but I wouldn't call it an odd personal preference… GitLab has offered innovative productivity tools way before GitHub (they had team discussions, projects, and various other things only available in newer versions of GitHub Enterprise more than a year before GitHub considered these; GitLab offers them for free).

Travis isn't the only slowness in our pipeline. We have 4 services running our pipeline: Travis, CircleCI, Appveyor, and Shippable. The fact that we have to fan out to have somewhat acceptable build times of about 45 minutes is a pretty good indicator that another platform like GitLab or Bitbucket, both of which offer extensible pipelines for free, would be worth evaluating.

@philsturgeon
Copy link
Contributor

@jimschubert only the original message mentioned potentially mirroring, and your comment. 🤔

Most folks here are saying hard pass on migration, and that’s definitely what I’m trying to say too.

To consider mirroring, I have serious concerns about what you’re proposing. If the conversation was “Can we set up a mirror with everything else turned off just so we can speed up maven” the.mn that’s something sure, but pure proposing duplicating issue trackers, project management, and fracturing development between two things. That’s gonna be a royal pain in the backside. 😅

Again, it sounds like the Travis workflow needs to be re-evaluated. Replacing TravisCI and that myriad of other things with CircleCI using prebuilt docker images as a base and workflows to parallelise various different long running processes is probably going to be more useful than adding yet another service to the stack for what should be a fairly simple open source project.

@philsturgeon
Copy link
Contributor

Sorry for typos, this taxi is optimising for speed too.

I had a look for our circleci 2.0 config but pure not using it! If you want pipelines (they call them workflows) you’ll want to switch to using CircleCI 2.0. It’s dope we use it at work, magnificiantly quick compared to Travis even without parrellellising stuff (which you absolutely should do also)

@MartinDelille
Copy link
Contributor Author

I propose the following :

  1. Mirror to gitlab
  2. Setup gitlab ci
  3. Once pipelines are equivalent on both platforms, state which plateform should host the source repo

This question is overwhelming many github project issues list at the current time. Another solution would be to wait and see how people handle the issue and learn from them.

I wouldn’t be surprised that a solution allowing to manage a project independently from the platform will emerge in the next weeks/month if it doesn’t already exists!

@wing328
Copy link
Member

wing328 commented Jun 11, 2018

I had a look for our circleci 2.0 config but pure not using it! If you want pipelines (they call them workflows) you’ll want to switch to using CircleCI 2.0. It’s dope we use it at work, magnificiantly quick compared to Travis even without parrellellising stuff (which you absolutely should do also)

Yes, we'll do it shortly as CircleCI will sunset 1.0 config by Aug.

@MartinDelille
Copy link
Contributor Author

@MartinDelille
Copy link
Contributor Author

I propose moving Circle CI discussion here: #278

@jimschubert
Copy link
Member

jimschubert commented Jun 11, 2018

@philsturgeon mirroring does not mean copying issue trackers and all that. It just means that for every push to this repo, the other repo is automatically updated. This would allow users who ditch GitHub for GitLab to still clone the code and submit patches, which doesn't have to go through GitHub pull requests. Sure, they could also come over https but that would require pulling code from a host that many FOSS supporters would refuse (and some refuse GitHub anyway).

The discussion about evaluating GitLab pipelines is irrelevant. There's no difference between using something GitLab would provide for free and posting a badge... from doing the same with Travis, CircleCI, or any other build automation tool. They're all third party tools to GitHub, which is where we've all agreed thus far to remain.

Maven isn't even the main issue in our pipeline. It's the rigidity in preparing a build environment coupled with the amount of integration and regression testing we have to do across multiple operating systems. GitLab allows you private container hosting, and it's pipelines provide for multi-stage conditional workflows. It also allows self-hosting job runners.

As I read this thread over multiple times, you're the only one who is a hard pass. If I'm wrong here, please link to others who have commented a hard pass on mirroring.

My only point of my previous comment, really, was that you said there was absolutely no reason for the consideration and I asked for clarification as to which you were referring to. Your comments lead me to believe that you haven't used GitLab before, so I don't get the feeling that the absolute recommendation is well-informed.

We're not arguing for or against either migrating or mirroring, we're discussing the opportunities as they have arisen in the community. To keep the discussion healthy, this means that sometimes we have to at least consider options that we don't necessarily think are a good idea. Something that speeds our builds and productivity would be good for contributors and the community as a whole. Shirking opportunities with absolute statements and asserting that "everyone is a hard pass" when that's not the case just adds noise to the discussion.

edit: I re-read this comment, and I don't intend the last statement in a harsh way. It's more a "we should keep our minds open to opportunities". I apologize if it seemed biting, I shouldn't be commenting on issues when I've been awake for 20 hours.

@jimschubert
Copy link
Member

@MartinDelille

I think we could setup the CI on GitLab and evaluate the differences between what we have now and what that would offer us. I don't think using GitLab CI means we need to host the source there, unless there's no way to apply githooks to PRs or something. I would be very surprised if that were the case. I use GitLab CI for a handful of personal projects, but I've never had to investigate hooks.

I think migrating from GitHub fully would be a bad idea because I assume GitHub will still have significantly more developers than GitLab by the end of the year. It seems like most people in this thread are wary of excluding the larger of the two communities because that would imply that we reduce our contributor base

@MartinDelille
Copy link
Contributor Author

@jimschubert

I totally agree with you about not reducing our contributor base! The idea would be more to increase it!

Concerning Gitlab CI I use it for my professional project and it does amazing thing: one is multiplateform build since Gitlab CI runs on linux/mac/window.

I would like to know what do what in our current pipeline:

  • Travis
  • Appveyor
  • Circle CI
  • Shippable

Could you tell me more?

@jmini
Copy link
Member

jmini commented Jun 11, 2018

I think that having multiple source management platforms (GitHub, GitLab...) creates more work for the core-team (multiple locations for issues, for Pull-Request or Merge-Request, ...). I do not have the resource to check multiple source management platforms.

@MartinDelille
Copy link
Contributor Author

@jmini
Copy link
Member

jmini commented Jun 11, 2018

@MartinDelille :

I would like to know what do what in our current pipeline

I think that the best documentation source is https://github.com/OpenAPITools/openapi-generator/wiki/Integration-Tests (it might be not up-to-date).

@MartinDelille
Copy link
Contributor Author

I think that having multiple platforms creates more work for the core-team (multiple locations for issues, for Pull-Request or Merge-Request, ...). I do not have the resource to check multiple platform.

That is a good point! See the end of my comment.

@philsturgeon
Copy link
Contributor

@jimschubert

mirroring does not mean copying issue trackers and all that.

I didn't say that. I said that mirroring to use their issues trackers, project management, etc would lead to two different locations for issues and everything else. It was in response to this:

I mirror projects that I publish to maven from GitHub to GitLab and I use GitLab's pipelines for automated testing and deployment. Not only do they have a significantly better issue tracker, project management, permissions structure, ability to lock files, email-based "Service Desk", integrated pipelines, container registry, and work out in the open when troubleshooting issues… they also turn around fixes and new features much faster than GitHub.

It's exactly the same concern that @jmini has here.

That's also why I mentioned Maven. CircleCI 2.0 has exactly pipelines so #278 will solve your concerns there including "and it's pipelines provide for multi-stage conditional workflows."

As I read this thread over multiple times, you're the only one who is a hard pass. If I'm wrong here, please link to others who have commented a hard pass on mirroring.

Sadly you misread what I said, please look again: "Most folks here are saying hard pass on migration," not mirroring.

Seeing as this conversation is mostly about CI now, shutting this thread and discussing #278 seems the most productive output.

@jimschubert
Copy link
Member

@philsturgeon

Sadly you misread what I said, please look again: "Most folks here are saying hard pass on migration," not mirroring.

Thanks for pointing that out. I was confused after reading your comment twice as to which you were referring to, I don't know how I read it twice and still skipped that word (and possibly others). Again, probably because I was exhausted (we had a roof leak over the weekend).

@jimschubert
Copy link
Member

@jmini We have multi-platform because the project is multi-platform and because we have to run integration tests on each as there are often platform-specific things that get overlooked (like line separators). I don't think there's a way around multi-platform integration testing, unless we just don't test some cases for our users.

@jmini
Copy link
Member

jmini commented Jun 11, 2018

@jimschubert: we do not speak from the same. For the moment we have one source management platform (for PR, issues, discussions....): GitHub.

I would prefer not to have multiple: GitHub, GitLab...

I will edit my previous message

@jimschubert
Copy link
Member

@jmini I think I understand what you're saying. Let me clarify what I'm saying in case there's confusion:

Source:
Github = main (issues, wiki, pull requests, etc.)
GitLab = mirror (code only, backup)

CI (which I thought you meant by multi-platform):
GitHub = Travis + CircleCI + Shippable + Appveyor (to split build effort, and to support linux and windows environments)
GitLab = GitLab CI. Everything can be done in a single pipeline, meaning no duplicated environment variables, shared caching between builds, etc. Supports multi-platform (Linux, Mac, and Windows), as well as being allowed as a third-party CI pipeline for GitHub projects just as the 4 we currently use.

@philsturgeon
Copy link
Contributor

philsturgeon commented Jun 11, 2018 via email

@jimschubert
Copy link
Member

@philsturgeon I'll reiterate a sentiment I made previously. The focus on GitLab CI is because they're now offering Premium and Ultimate pricing for free for open source contributors. This is up to $99/month per user… but for free. This also means up to 50,000 build hours per month, as opposed to CircleCI 2.0 which would be 1,500 on the free plan.

Compare this to CircleCI 2.0's cost of a minimum of $50/month to suit our needs. The free version of CircleCI would be a single concurrent job and only up to 25 hours per month of build time. We use much more than that.

I don't think that's a false dichotomy; I've been perfectly transparent about this being a discussion about opportunism.

@MartinDelille
Copy link
Contributor Author

Ok @wing328 can I create the organization on Gitlab.com?

@philsturgeon
Copy link
Contributor

@jimschubert CircleCI 2.0 is free for open-source projects, not $50. https://circleci.com/docs/2.0/oss/

@philsturgeon
Copy link
Contributor

But yeah if the number of hours even under the OSS version is not enough, that's something. This last comment of yours is the first to provide any insight into why you'd specifically want GitLab CI over other solutions that also do the same thing. :)

@MartinDelille
Copy link
Contributor Author

MartinDelille commented Jun 18, 2018

I mirrored the repo on gitlab: https://gitlab.com/OpenAPITools/openapi-generator

MartinDelille added a commit to MartinDelille/openapi-generator that referenced this issue Jun 18, 2018
@jimschubert
Copy link
Member

Closing this as we've obviously stuck with GitHub, and Microsoft's acquisition of GitHub has made the platform more solid if anything. See the issue on mobile web which I reported and they fixed in under 4 hours. We've also spread our integration/regression/unit testing efforts across multiple free providers to spread the load and reduce wait time for feedback.

@MartinDelille
Copy link
Contributor Author

Ok I changed my mind regarding Microsoft acquisition and I agree with you.

What should we do with the current Gitlab repository?

  • Remove it
  • Manually mirror it

If there is some issue/merge request there we could eventually advertise that we use Github for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants