All Systems Operational
Registry Operational
API Operational
Build System Operational
Frontend ? Operational
Security Scanning Operational
Amazon Web Services Operational
Database Operational
GitHub ? Operational
Payment Management Operational
Plan Management Operational
Atlassian Bitbucket Webhooks Operational
Atlassian Bitbucket SSH Operational
Atlassian Bitbucket API Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Scheduled Maintenance
Quay.io read-only on June 13th Jun 13, 12:00-20:00 EDT
On Saturday, June 13th, 2020 from 12 PM Eastern to 8 PM Eastern, Quay.io will be undergoing maintenance to migrate our primary storage.

During this time, Quay.io will be placed into a read-only state and the following operations will be suspended:
- All push operations will be suspended
- All build operations will be suspended
- All security scan operations will be suspended
- Any new ACI conversions will lack signing
- All other modification operations will be suspended

During this time, pull operations, squashed image pulls and our UI will remain operational.

We apologize for the inconvenience this may cause, but this migration will allow us to support better redundancy on our storage in the future.
Posted on May 18, 12:44 EDT
Past Incidents
May 25, 2020

No incidents reported today.

May 24, 2020

No incidents reported.

May 23, 2020

No incidents reported.

May 22, 2020
Resolved - The pods rollout has completed and the service is operational again.
May 22, 18:03 EDT
Identified - We are currently rolling our pods after a recent database failover
May 22, 16:53 EDT
May 21, 2020

No incidents reported.

May 20, 2020

No incidents reported.

May 19, 2020
Resolved - Currently service is restored and stable. At this time, the incident is resolved.
May 19, 22:40 EDT
Monitoring - We have successfully reverted service impacting changes. The service is currently operational and stable. We will continue to monitor. Next update in 30 minutes or as new information becomes available.
May 19, 22:11 EDT
Update - We are still investigating the root cause and continue to work with our infrastructure provider on restoring service. Next update in 30 minutes or as new information becomes available.
May 19, 21:22 EDT
Update - We are still investigating the root cause and continue to work with our infrastructure provider on restoring service. Next update in 30 minutes or as new information becomes available.
May 19, 21:19 EDT
Update - We are still investigating the root cause and continue to work with our infrastructure provider on restoring service. Next update in 30 minutes or as new information becomes available.
May 19, 20:48 EDT
Update - We are still investigating the root cause and will continue to update the status page. We are continuing to work with our infrastructure provider to restore service. Next update in 30 minutes or whenever we have new information.
May 19, 20:13 EDT
Update - We are still investigating the root cause and will continue to update the status page. We are continuing to work with our infrastructure provider to restore service. Next update in 30 minutes or whenever we have new information.
May 19, 19:32 EDT
Update - We are still investigating the root cause and will continue to update the status page. We are continuing to work with our infrastructure provider to restore service. Next update in 30 minutes or whenever we have new information.
May 19, 19:03 EDT
Update - We are still investigating the root cause and will continue to update the status page. We have engaged our infrastructure provider to assist with service restoration. Next update in 30 minutes or whenever we have new information.
May 19, 18:20 EDT
Update - We are still investigating the root cause and will continue to update the status page. Next update in 30 minutes or whenever we have new information.
May 19, 18:13 EDT
Update - We are still investigating the root cause and will continue to update the status page. Next update in 30 minutes or whenever we have new information.
May 19, 17:37 EDT
Update - We are still investigating the root cause and will continue to update the status page. Next update in 30 minutes or whenever we have new information.
May 19, 16:58 EDT
Investigating - We are still investigating the root cause and will continue to update the status page. Next update in 30 minutes or whenever we have new information.
May 19, 16:29 EDT
Update - Scaling up the database has caused the service to go back in full outage. We are still investigating root cause and further ways to workaround the issue.
May 19, 15:47 EDT
Update - We are scaling up the database instance which we hope will fix the occasional errors we are seeing. We will post a new update within the next 30 minutes as we confirm this has resolved the remaining issues.
May 19, 15:26 EDT
Identified - We are occasionally seeing 504 errors when pulling images even after turning off connection pooling. Our team is investigating further.
May 19, 14:52 EDT
Update - We are continuing to monitor for any further issues.
May 19, 14:17 EDT
Monitoring - Quay.io push operations have been re-enabled. The service is stable and we are monitoring the situation while we work on a definitive fix and a complete root cause analysis
May 19, 14:14 EDT
Update - We are working on further stabilizing the service and on re-enabling write operations. The workaround is still in place and root cause is still unknown.
May 19, 13:43 EDT
Identified - The temporary fix is proving to be effective and the service is stabilizing. Quay images pulling works albeit slower than normal. We are working on re-enabling write operations and a definitive fix.
May 19, 13:11 EDT
Update - Turning database connection pooling off seems to have mitigated the issue for now. We're monitoring closely and working on root cause analysis and a permanent fix
May 19, 12:43 EDT
Update - We are still investigating the root cause of the DB connections spike. Attempts at failing over to a standby cluster have been unsuccessful.
May 19, 12:10 EDT
Update - We are still investigating the root cause of the DB connections spike. Attempts at failing over to a standby cluster have been unsuccessful.
May 19, 11:38 EDT
Update - The root cause of the outage is still unknown. Our attempts at recovering the service have not been successful so far. We are working across the board to bring the service back up.
May 19, 11:02 EDT
Update - We are still working on fixing the issues that were identified previously.
May 19, 10:05 EDT
Update - We've identified multiple causes for this outage and continue to work on fixing these. We acknowledge that our consumers depend on Quay.io for their workloads and are attempting to resolve this with the highest urgency.
May 19, 09:16 EDT
Update - We are still investigating the root cause and will continue to update the status page. Next update in 30 minutes or whenever we have new information.
May 19, 08:45 EDT
Update - We are still investigating the root cause and will continue to update the status page. Next update in 30 minutes or whenever we have new information.
May 19, 08:12 EDT
Update - Quay.io is currently in read-only mode. We are still investigating the root cause and will continue to update the status page.
May 19, 07:39 EDT
Update - We are still working on bringing quay.io back online in read-only mode. We are currently able to bring it back online for a brief period of time, but the service continues to degrade due to the underlying database connections load.
May 19, 07:21 EDT
Update - We are working on bringing quay.io back online in read-only mode. Once we accomplish that we will continue working on bringing it up with full read/write support.
May 19, 06:38 EDT
Update - The number of open connections to the Quay.io database increased significantly causing an outage of the service, we are still investigating the root cause of this spike in DB connections. Right now we don't have an ETA for full service restoration.
May 19, 06:14 EDT
Investigating - We are currently investigating this issue.
May 19, 03:30 EDT
May 18, 2020
Resolved - Service has returned to operational status.
May 18, 14:09 EDT
Monitoring - A fix has been implemented and we are monitoring the results.
May 18, 14:03 EDT
Investigating - We are currently seeing elevated error rate and are working on resolution.
May 18, 13:55 EDT
May 17, 2020

No incidents reported.

May 16, 2020

No incidents reported.

May 15, 2020

No incidents reported.

May 14, 2020

No incidents reported.

May 13, 2020

No incidents reported.

May 12, 2020

No incidents reported.

May 11, 2020

No incidents reported.