On July 18, 2025, at approximately 12:28 PM ET, a server in our production environment became unreachable, causing a critical incident that impacted the Yola production services. This server had been provisioned during a recent deployment two days prior, which was marked as successful at the time.
Because the previous server continued serving traffic initially, the issue went unnoticed until it was terminated. Once the older server was removed, the new server failed to respond, leading to an immediate disruption of service.
We have since resolved the issue and restored service. To prevent this from happening again, we are improving our deployment process to include thorough validation of new servers’ readiness before fully switching traffic over.