Skip to content

Explain how to work with subtree #70654

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Apr 14, 2020
Merged
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 63 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,69 @@ with one another are rolled up.
Speaking of tests, Rust has a comprehensive test suite. More information about
it can be found [here][rctd].

### External Dependencies
### External Dependencies (subtree)

As a developer to this repository, you don't have to treat the following external projects
differently from other crates that are directly in this repo:

* none so far, see https://github.com/rust-lang/rust/issues/70651 for more info

They are just regular files and directories. This is in contrast to `submodule` dependencies
(see below for those). Only tool authors will actually use any operations here.

#### Synchronizing a subtree

There are two synchronization directions: `subtree push` and `subtree pull`.

```
git subtree push -P src/tools/clippy [email protected]:your-github-name/rust-clippy rustup
```

takes all the changes that
happened to the copy in this repo and creates commits on the remote repo that match the local
changes. Every local commit that touched the subtree causes a commit on the remote repo, but is
modified to move the files from the specified directory to the tool repo root.

Make sure to not pick the `master` branch, so you can open a normal PR to the tool to merge that
subrepo push.

```
git subtree pull -P src/tools/clippy https://github.com/rust-lang/rust-clippy master
```

takes all changes since the last `subtree pull` from the tool repo
repo and adds these commits to the rustc repo + a merge commit that moves the tool changes into
the specified directory in the rust repository.

It is recommended that you always do a push first and get that merged to the tool master branch.
Then, when you do a pull, the merge works without conflicts.
While definitely possible to resolve conflicts during a pull, you may have to redo the conflict
resolution if your PR doesn't get merged fast enough and there are new conflicts. Do not try to
rebase the result of a `git subtree pull`, rebasing merge commits is a bad idea in general.

You always need to specify the `-P` prefix to the subtree directory and the corresponding remote
repository. If you specify the wrong directory or repository
you'll get very fun merges that try to push the wrong directory to the wrong remote repository.
Luckily you can just abort this without any consequences and try again. It is usually fairly obvious
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you abort?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When pulling, git merge --abort I'd guess? We should test it.

Pushing should always succeed, but if you try to open PR from a malformed branch, it will just look very strange and be unmergeable.

Presumably you can redo it with the right directory specified and the PR will become correct (idk if it overwrites an existing branch though, or maybe it has a -f/--force flag like git push does?).

that this is happening because you suddenly get thousands of commits that want to be synchronized.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I correct that subtrees mean that we should no longer need to give particular priority to syncs from clippy upstream? i.e. they're "just normal PRs"? (They would essentially be no different to other PRs being made to that code).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Priority meaning @bors p=1? That definitely should end with the gating.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jup, all the "time critical" stuff happens on the tool's repo side and not when syncing into rustc. Unless there are bug fixes, but our policy for tool bugfixes and rustc bugfixes willl not have to differ.


#### Creating a new subtree dependency

If you want to create a new subtree dependency from an existing repository, call (from this
repository's root directory!!)

```
git subtree add -P src/tools/clippy https://github.com/rust-lang/rust-clippy.git master
```

This will create a new commit, which you may not rebase under any circumstances! Delete the commit
and redo the operation if you need to rebase.

Now you're done, the `src/tools/clippy` directory behaves as if clippy were part of the rustc
monorepo, so no one but you (or others that synchronize subtrees) needs actually use `git subtree`.


### External Dependencies (submodules)

Currently building Rust will also build the following external projects:

Expand Down Expand Up @@ -221,7 +283,6 @@ before the PR is merged.

Rust's build system builds a number of tools that make use of the
internals of the compiler. This includes
[Clippy](https://github.com/rust-lang/rust-clippy),
[RLS](https://github.com/rust-lang/rls) and
[rustfmt](https://github.com/rust-lang/rustfmt). If these tools
break because of your changes, you may run into a sort of "chicken and egg"
Expand Down