[Ci-users] Service Degradation: Github triggered Jobs | 14-Feb-2019/Ongoing

Tue Feb 19 16:28:30 UTC 2019
Brian Stinson <brian at bstinson.com>

Date: 
14-Feb-2019 - Ongoing

Description:
 We are noticing that we are burning through our entire hourly Github API quota, sometimes quite rapidly. This affects jobs that trigger from Github, and jobs that report back using the githubNotify pipeline step.

Symptoms:
- Jobs that trigger using the GHPRB or Github-branch-source plugins (even with webhooks properly configured) may be delayed in triggering
- Jobs that report back to Github PRs may hang until it can grab the appropriate quota

Next steps:
- Clean up old repos [we've already done as many as we can, if you have dead jobs please contact us]
- Split up "heavy" Github API users to new Github tokens, with their own quotas

Timeline for resolution:
Ongoing, we'll post here as we have more details.