Azure and GitHub outages stalled hybrid networks, AI services, and CI pipelines

Oct 9, 2026·8 min read·Row House Tech News
Read this issue on LinkedIn ↗

Azure’s September 30th, 2026, gateway outage disrupted private connections and a documented backup option. Microsoft documents a site-to-site virtual private network (VPN) through Azure VPN Gateway as an ExpressRoute backup. Both services were affected.

Microsoft’s own resiliency guidance says zone-redundant configurations don’t protect against full regional outages or human or automation errors. Separate AI and GitHub failures exposed dependencies in application requests and continuous integration (CI).

Infrastructure leaders should test whether each backup path survives the failure that takes out the primary, not only whether a second path exists.

A working backup depends on more than a second connection.

Microsoft documented five hours and 45 minutes of disruption beginning September 30th, 2026, with mitigation completed on October 1st. Customers did not all experience continuous downtime. Microsoft initially identified 18 affected regions, then flagged a regional correction in its status history.

The footprint covered North America, Europe, Asia, Africa, and the Middle East. Some gateways lost connectivity. Others degraded or retained connections with reduced redundancy. Azure VMware Solution connectivity also suffered. Applications could keep running while corporate infrastructure lost access to them.

ExpressRoute Gateway connects enterprise networks to Azure virtual networks through private connectivity. VPN Gateway supplies encrypted tunnels between those environments. Losing both complicated recovery for customers using one to back up the other.

Microsoft’s preliminary investigation identified two interacting changes. A gateway management change created unexpected load during operating system servicing. Pressure on dependencies blocked regional services from adding capacity as intended.

Pausing maintenance limited further exposure, but recovery also required reversing the management change. Microsoft reported that rollback reduced load and enabled recovery. Investigation into scaling behavior and safeguards continued, without confirmation that preventive changes were complete.

The disruption reached administration as well as traffic. Gateway resources could fail to load in Azure portal, and network management operations failed or slowed. Engineers restored supporting components through healthy alternative instances where available.

The final affected-service list included Azure Firewall, Application Gateway, and Web Application Firewall. A functioning connection did not guarantee functioning management tools, a distinction that changes how incident teams assess recovery.

The Operator’s Lens

  • Have the network team map shared Azure dependencies before approving primary and backup connectivity as independent protection.

  • Separate traffic and management failures in incident procedures, with escalation contacts accessible outside Azure portal.

Application availability also depends on the services behind each request, even when network connections remain healthy.

Azure OpenAI and AI Foundry Agent Services Went Down in Sweden Central

Repeated restarts drained capacity while the underlying dependency stayed overloaded.

Microsoft documented five hours and 55 minutes of disruption in Sweden Central on September 29th, 2026. Azure OpenAI, Foundry Agent Service, Foundry Models, and Cognitive Services were affected. Customers experienced intermittent failures, slower responses, and HTTP 5XX server errors.

The failure involved a shared backend, rather than a reported model fault. An internal process increased metadata retrieval workloads. Database and caching calls reached service thresholds, reads timed out, and backend instances became unhealthy.

Automated health checks repeatedly restarted instances, removing capacity from the available pool. The recovery mechanism increased pressure on surviving instances without clearing the dependency bottleneck.

Microsoft’s September 29th, 2026, timeline placed monitoring detection 11 minutes after customer impact began. Engineers identified database timeouts and investigated ongoing deployments. Initial scaling adjustments did not resolve the disruption.

Recovery required additional backend instances, higher resource limits, and disabling the health check. Engineers also restarted infrastructure nodes. Capacity and automation behavior both needed attention.

A process waiting on a slow dependency is different from one that has stopped functioning. Restarting it leaves the bottleneck intact.

Microsoft planned throttling, rate limits, and further service reconfiguration. These remained planned measures, with a final review pending.

For enterprise AI deployments, endpoint reachability and recovery behavior deserve attention alongside model performance. An application can stall for hours on a region the provider is still repairing.

The Operator’s Lens

  • Set timeouts and bounded retries on Azure OpenAI calls so a slow region fails fast instead of queuing requests behind it.

  • Exercise regional endpoint failure for critical AI applications, including unavailable models and interrupted agent work.

Software delivery faces similar exposure when available compute depends on remote coordination and the network paths connecting it.

A Network Failure Between GitHub’s Datacenter and Cloud Regions Stalled Actions Runners

Execution machines could not deliver builds without working job assignment.

GitHub traced its October 5th, 2026, failure to connectivity between an on-premises data center region and cloud regions. Connections serving database and storage services were affected. Hosted runner assignment failed, stopping managed execution machines from receiving jobs. Testing, packaging, and delivery stalled before code execution.

GitHub’s analysis measured both overall delays and the worst period:

  • 14.3% of hosted workflow runs exceeded five minutes before starting.

  • 26.5% of individual hosted jobs did not start within five minutes.

  • 47.0% of hosted workflow runs failed during the roughly 90-minute worst period.

  • 71.9% of hosted jobs did not start within five minutes in that period.

Workflows can contain multiple jobs, so these measures describe different units of delivery. They cover hosted workloads, not all Actions activity. Self-hosted runners were unaffected.

Existing GitHub Pages websites continued serving. Some repository and account administration pages returned intermittent errors, showing how service availability differed across the platform.

GitHub redirected database and storage traffic toward healthy regions and endpoints, then expanded affected compute capacity. Actions and hosted runners recovered at 21:54 UTC (5:54PM EDT/2:54PM PDT) on October 5th. All affected services recovered at 22:49 UTC (6:49PM EDT/3:49PM PDT). Clearing queued work required capacity as well as restored connectivity.

GitHub’s availability report described an Actions launch service in its data centers, with most execution running on Azure. GitHub said it was accelerating its Actions architecture roadmap. The October 5th explanation did not name the cloud provider, and the network investigation remained ongoing.

GitHub recorded further Actions disruptions on October 7th, alongside Git operations, issues, and pull requests. Its status record did not connect them to the October 5th failure.

The Operator’s Lens

  • Reserve customer-managed runners for release-critical workflows, with platform engineers accountable for patching, credentials, and workload isolation.

  • Alert separately on runner assignment delays and execution failures so delivery teams can distinguish infrastructure queues from broken builds.

Backup connections and alternative runners offer recovery options only within the limits of their remaining platform dependencies.

Provider Guidance Leaves Region Failover, Backup Paths and Runner Fallback to Customers

Fallback coverage still requires customer ownership, maintenance, and testing.

Microsoft’s ExpressRoute guidance assigns redundancy outside Microsoft’s network to customers and carriers. Duplicated components cannot eliminate failure across an entire connection location. Independent recovery paths address a broader exposure than duplicated equipment.

Uptime Institute’s May 13th, 2026, analysis described external infrastructure failures gaining prominence among publicly reported outages. Andy Lawrence, executive director of Uptime Intelligence, emphasized interactions between systems rather than isolated component failures.

The financial baseline is substantial. Uptime Institute’s 2025 Annual Survey found 57% of respondents’ most recent major outages cost more than $100,000. The same Uptime Institute survey found one in five respondents reported costs exceeding $1 million.

Microsoft warns that a single customer router or shared carrier port can undermine circuit redundancy. It recommends active-active operation, with both connections carrying traffic. A circuit remains tied to one connection location, so duplicated connections do not supply geographical failover.

Zone-redundant gateways distribute infrastructure within a region, not across every external dependency. Azure prefers ExpressRoute when both paths advertise the same route. Customers remain responsible for consistent outbound routing preferences.

Microsoft discourages VPN fallback for workloads with strict latency or heavy bandwidth requirements, favoring ExpressRoute connectivity across multiple sites. Passive configurations still need maintenance and testing.

Microsoft’s documented backup design terminates the site-to-site tunnel on an Azure VPN gateway, part of the same gateway layer disrupted on September 30th. In our view, a backup meant to survive a gateway-layer failure needs a path that avoids Azure-managed gateways. SD-WAN or a network virtual appliance over the internet can provide one. So can a second carrier into a different peering location.

Detection speed matters as much as the second path. Microsoft says Bidirectional Forwarding Detection on ExpressRoute private peering cuts failure detection from about three minutes to under a second.

Restored connectivity cannot make an unavailable AI endpoint operational. For AI workloads, Microsoft’s Foundry guidance describes hot/hot, hot/warm, and hot/cold multiregion designs. In a hot/warm design, the secondary region keeps critical resources such as deployed models ready to start. On September 29th, the failing backend served requests across the whole Sweden Central region.

Azure Service Health alerts can be scoped by subscription, service, and region, so the team that owns an ExpressRoute circuit or an AI endpoint hears about a provider incident directly.

GitHub allows self-hosted runners on premises or in another cloud. Customers pay machine costs and maintain operating systems and other software. Automatic updates cover only the runner application.

These machines still execute GitHub Actions jobs. GitHub’s documentation establishes neither automatic fallback nor independence from GitHub availability. Owning execution capacity does not remove every coordination dependency.

The Operator’s Lens

  • Move at least one backup path off Azure-managed gateways, through SD-WAN, a network virtual appliance, or a second carrier, and test failover to it every quarter.

  • Keep a warm model deployment in a second region for any AI application that must survive a regional outage.

  • Give one recovery owner responsibility for routing restoration and runner selection tests, with acceptance criteria based on business service recovery.

The outages involved different technical failures but exposed a common business constraint. Available components cannot deliver a service when required dependencies fail. Providers restored connectivity, request processing, and workflow assignment. Enterprise continuity depends on knowing which alternatives survive the same incident and which duplicate capacity behind the same dependency.

Sources
Azure Maintenance Cut ExpressRoute and VPN Gateway Links in 18 Regions
Azure OpenAI and AI Foundry Agent Services Went Down in Sweden Central
A Network Failure Between GitHub's Datacenter and Cloud Regions Stalled Actions Runners
Provider Guidance Leaves Region Failover, Backup Paths and Runner Fallback to Customers