Ubicloud - Actions Job Delays – Incident details

Actions Job Delays

Resolved
Degraded performance
Started 1 day agoLasted 3 hours 9 minutes

Affected

GitHub Actions Service

Degraded performance from 2:47 PM to 3:10 PM, Operational from 3:10 PM to 3:20 PM, Degraded performance from 3:20 PM to 5:56 PM

GitHub → Actions

Degraded performance from 2:47 PM to 3:10 PM, Operational from 3:10 PM to 3:20 PM, Degraded performance from 3:20 PM to 5:56 PM

Updates
  • Resolved
    UTC
    Resolved

    This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available.

  • Update
    UTC
    Update

    GitHub Actions experienced degraded performance for some hosted runners due to throttling within an upstream Azure dependency. Service capacity has recovered, and we are continuing to monitor while working with Azure on the underlying condition.

  • Update
    UTC
    Update

    We are currently applying a mitigation and anticipate recovery within thirty minutes.

  • Update
    UTC
    Update

    We have identified an issue with our upstream provider which is causing Actions requests to 429 which is creating the delays. We have escalated to the owning team and are investigating how to mitigate the 429s.

  • Update
    UTC
    Update

    We are seeing a reoccurrence in run-start delays, and are continuing to investigate the issue. Customers will potentially experience delays of up to ten minutes.

  • Investigating
    UTC
    Investigating

    Actions is experiencing degraded performance. We are continuing to investigate.

  • Monitoring
    UTC
    Monitoring

    The degradation affecting Actions has been mitigated. We are monitoring to ensure stability.

  • Update
    UTC
    Update

    Run-start delays on Ubuntu runners have been resolved. We are investigating run-start delays on Windows runners and will share more information as it becomes available.

  • Update
    UTC
    Update

    We have identified the cause of increased Actions run start delays on Ubuntu runners and are actively deploying a fix. Customers may continue to experience intermittent delays while the mitigation rolls out and service metrics return to normal.

  • Investigating
    UTC
    Investigating

    We are investigating reports of degraded performance for Actions