Build Cop: Flaky Tests are hard to track

[nnegrey]

Flaky tests seem to get opened, closed, opened, closed, opened, closed by the bot, which makes it relatively difficult to properly track what I'm trying to fix.

One example: GoogleCloudPlatform/python-docs-samples#2910

[tbpg]

Idea:

If the bot sees that the issue has been opened and closed 2 or more times, add a buildcop: flaky label and comment on it saying it's up to you to close this issue.

Then, if the test fails after someone other than the bot closed it, reopen it and say it's up to you to close this.

[nnegrey]

Yea, something that reverts to a human having to come in and fix would be nifty.

I had 3 the other day that I created a PR to fix and before it was merged 2 of them automatically closed.

[tbpg]

Sorry about that! I'll move forward with the idea.

[tbpg]

This didn't work for GoogleCloudPlatform/python-docs-samples#2898.