Incidents reported on status page for Logto https://status.logto.io/ https://d1lppblt9t2x15.cloudfront.net/logos/cbf43f954be13b7cbfdc9bda14b0fd55.png https://status.logto.io/ en https://status.logto.io/incident/545773 Tue, 15 Apr 2025 17:35:00 -0000 https://status.logto.io/incident/545773#ece3c6eedc718f62f8f71a8601be19efc917b2061d9daa6cf03d01e318628569 ## Incident Summary Logto Cloud and US Auth services experienced approximately 7 minutes of downtime due to an unplanned disruption during a computing resource SKU upgrade. ## Root cause The downtime was caused by a SKU change in our cloud provider’s compute resources, which doesn't support 0 downtime. The upgrade process led to temporary unavailability as instances were replaced and rescheduled. ## Resolution Once the issue was identified, the team monitored the redeployment and confirmed service recovery within minutes. No manual intervention was required beyond observation. ## Action Items - Improve pre-upgrade impact assessment for infrastructure changes. - Explore zero-downtime deployment strategies for compute upgrades. We sincerely apologize for the inconvenience and are committed to preventing similar incidents in the future. https://status.logto.io/incident/545773 Tue, 15 Apr 2025 17:35:00 -0000 https://status.logto.io/incident/545773#ece3c6eedc718f62f8f71a8601be19efc917b2061d9daa6cf03d01e318628569 ## Incident Summary Logto Cloud and US Auth services experienced approximately 7 minutes of downtime due to an unplanned disruption during a computing resource SKU upgrade. ## Root cause The downtime was caused by a SKU change in our cloud provider’s compute resources, which doesn't support 0 downtime. The upgrade process led to temporary unavailability as instances were replaced and rescheduled. ## Resolution Once the issue was identified, the team monitored the redeployment and confirmed service recovery within minutes. No manual intervention was required beyond observation. ## Action Items - Improve pre-upgrade impact assessment for infrastructure changes. - Explore zero-downtime deployment strategies for compute upgrades. We sincerely apologize for the inconvenience and are committed to preventing similar incidents in the future. https://status.logto.io/incident/545773 Tue, 15 Apr 2025 17:28:00 -0000 https://status.logto.io/incident/545773#9bd8f2dd3b1286097c7dd11f84c078cd9b592ee352a162b948ebcaed60541755 Services are back online. https://status.logto.io/incident/545773 Tue, 15 Apr 2025 17:28:00 -0000 https://status.logto.io/incident/545773#9bd8f2dd3b1286097c7dd11f84c078cd9b592ee352a162b948ebcaed60541755 Services are back online. https://status.logto.io/ Tue, 15 Apr 2025 17:27:05 +0000 https://status.logto.io/#09d03a3091cd4d7aa869dbfe8c0a89d01b89ac37fe673450d40d4db560192b0c Auth service (US) recovered https://status.logto.io/ Tue, 15 Apr 2025 17:25:24 +0000 https://status.logto.io/#cf892eb17fb63115ed5212c99be8fa8cd0408ad367e8ee23fdef0223e6a729cc Cloud service recovered https://status.logto.io/incident/545773 Tue, 15 Apr 2025 17:21:00 -0000 https://status.logto.io/incident/545773#8f69290d555678117e43609e7850eb5ff4ae38b01953cfbaee7bfd8dc747500c We acknowledge that our Cloud and US Auth services are currently down, and we are investigating the issue. https://status.logto.io/incident/545773 Tue, 15 Apr 2025 17:21:00 -0000 https://status.logto.io/incident/545773#8f69290d555678117e43609e7850eb5ff4ae38b01953cfbaee7bfd8dc747500c We acknowledge that our Cloud and US Auth services are currently down, and we are investigating the issue. https://status.logto.io/ Tue, 15 Apr 2025 17:20:15 +0000 https://status.logto.io/#cf892eb17fb63115ed5212c99be8fa8cd0408ad367e8ee23fdef0223e6a729cc Cloud service went down https://status.logto.io/ Tue, 15 Apr 2025 17:18:16 +0000 https://status.logto.io/#09d03a3091cd4d7aa869dbfe8c0a89d01b89ac37fe673450d40d4db560192b0c Auth service (US) went down https://status.logto.io/incident/450597 Fri, 25 Oct 2024 22:07:00 -0000 https://status.logto.io/incident/450597#6a51f5e71afe94ddc1b3af5e3ce278dd95c2e6fa7d0df357ae0910f718dc500d Blog is up. https://status.logto.io/incident/450597 Fri, 25 Oct 2024 15:23:00 -0000 https://status.logto.io/incident/450597#65ac799994b2dbd9b5dbad91c130c14aa08d1497c7c0262a645ac8d5fc2ac442 The website has been recovered. The blog is still experiencing downtime. https://status.logto.io/incident/450597 Fri, 25 Oct 2024 14:59:00 -0000 https://status.logto.io/incident/450597#29ced484e2e548f9d3a54ebf49c2d59be1ef8f463febf911de88ee1f8b4c0f5c We noticed that website and blog are not available at this time and we are working on a fix. Your Logto instances and Logto Cloud services are safe. https://status.logto.io/incident/399966 Thu, 18 Jul 2024 12:00:00 -0000 https://status.logto.io/incident/399966#1d4847ba03e2587683426f4ce49945f979033acdd07d21b59add8ce6a3880d85 You can find the postmortem [here](https://blog.logto.io/postmortem-database-breaking-change-causes-500-error/?ref=status). https://status.logto.io/incident/399966 Thu, 18 Jul 2024 12:00:00 -0000 https://status.logto.io/incident/399966#1d4847ba03e2587683426f4ce49945f979033acdd07d21b59add8ce6a3880d85 You can find the postmortem [here](https://blog.logto.io/postmortem-database-breaking-change-causes-500-error/?ref=status). https://status.logto.io/incident/399966 Thu, 18 Jul 2024 09:32:00 -0000 https://status.logto.io/incident/399966#3e43b90cf5ef89f5487c6b8f109388bc90982010155a8fcbe3f326d3d0e322a8 The sign-in experience has recovered. **Cause**: During a recent deployment, a breaking database schema change caused an exception when swapping between the staging and production environments. We apologize for any inconvenience this may have caused. We are currently discussing and planning preventative measures to ensure this issue does not occur again in the future. These measures will be implemented as soon as possible. https://status.logto.io/incident/399966 Thu, 18 Jul 2024 09:32:00 -0000 https://status.logto.io/incident/399966#3e43b90cf5ef89f5487c6b8f109388bc90982010155a8fcbe3f326d3d0e322a8 The sign-in experience has recovered. **Cause**: During a recent deployment, a breaking database schema change caused an exception when swapping between the staging and production environments. We apologize for any inconvenience this may have caused. We are currently discussing and planning preventative measures to ensure this issue does not occur again in the future. These measures will be implemented as soon as possible. https://status.logto.io/incident/399966 Thu, 18 Jul 2024 08:57:00 -0000 https://status.logto.io/incident/399966#4f1d408181f4556f325ec52dbaed417a2f6a3b2a2648ebed78185f8d7817db36 The sign-in experience shows error code 500. https://status.logto.io/incident/399966 Thu, 18 Jul 2024 08:57:00 -0000 https://status.logto.io/incident/399966#4f1d408181f4556f325ec52dbaed417a2f6a3b2a2648ebed78185f8d7817db36 The sign-in experience shows error code 500. https://status.logto.io/incident/394292 Sat, 06 Jul 2024 02:35:00 -0000 https://status.logto.io/incident/394292#8bf70b557bc381d93fb1c6b31d3244504bff28bc08d491cf51088065f7016ebd We've removed some false positives on timeout. https://status.logto.io/incident/394292 Sat, 06 Jul 2024 02:35:00 -0000 https://status.logto.io/incident/394292#8bf70b557bc381d93fb1c6b31d3244504bff28bc08d491cf51088065f7016ebd We've removed some false positives on timeout. https://status.logto.io/incident/389705 Wed, 26 Jun 2024 09:35:00 -0000 https://status.logto.io/incident/389705#a6a44d2ecfc64b86623f80d6825645e4d1ee43e99e8d88ce747620301ca69417 The US region database (Azure) experienced an unexpected "Updating" status, causing a service outage for about 1 minute. The issue has now been resolved. https://status.logto.io/incident/378536 Mon, 03 Jun 2024 04:03:00 -0000 https://status.logto.io/incident/378536#7e23b910b7312e15d0d5525cbeafb90bf7fe15c26f5bc4fd26fe4c6fd0895690 We've removed a false positive incident for accuracy. https://status.logto.io/incident/378536 Mon, 03 Jun 2024 04:03:00 -0000 https://status.logto.io/incident/378536#7e23b910b7312e15d0d5525cbeafb90bf7fe15c26f5bc4fd26fe4c6fd0895690 We've removed a false positive incident for accuracy. https://status.logto.io/incident/378536 Mon, 03 Jun 2024 04:03:00 -0000 https://status.logto.io/incident/378536#7e23b910b7312e15d0d5525cbeafb90bf7fe15c26f5bc4fd26fe4c6fd0895690 We've removed a false positive incident for accuracy. https://status.logto.io/incident/371064 Sat, 18 May 2024 01:50:00 -0000 https://status.logto.io/incident/371064#9f56f09ecfb95ec91fb16bef658fdef3f5888ddbe901a50bd97f3a076c6772fd We've noticed some false negatives of downtime. We're taking a look on the logs to find out the cause. https://status.logto.io/incident/371064 Sat, 18 May 2024 01:50:00 -0000 https://status.logto.io/incident/371064#9f56f09ecfb95ec91fb16bef658fdef3f5888ddbe901a50bd97f3a076c6772fd We've noticed some false negatives of downtime. We're taking a look on the logs to find out the cause. https://status.logto.io/incident/366902 Thu, 09 May 2024 04:42:00 -0000 https://status.logto.io/incident/366902#e55645c6006085922b1ffb3e977f024b7e6ff6ae61063c074cdc7b39e5d84370 The Azure database met some unknown issues and it's recovered. We are discussing about the mitigate plans including relocating database to a region that supports zone redundant high availability. https://status.logto.io/incident/366902 Thu, 09 May 2024 04:42:00 -0000 https://status.logto.io/incident/366902#e55645c6006085922b1ffb3e977f024b7e6ff6ae61063c074cdc7b39e5d84370 The Azure database met some unknown issues and it's recovered. We are discussing about the mitigate plans including relocating database to a region that supports zone redundant high availability. https://status.logto.io/incident/366902 Thu, 09 May 2024 04:41:00 -0000 https://status.logto.io/incident/366902#18b04424e62504c3a35920ee7235fb33da9d3c7081677159862b53e663560dc7 We've noticed that our services are randomly responding 503. We are investigating it. https://status.logto.io/incident/366902 Thu, 09 May 2024 04:41:00 -0000 https://status.logto.io/incident/366902#18b04424e62504c3a35920ee7235fb33da9d3c7081677159862b53e663560dc7 We've noticed that our services are randomly responding 503. We are investigating it. https://status.logto.io/incident/361697 Sun, 28 Apr 2024 09:51:00 -0000 https://status.logto.io/incident/361697#1e5a39adf0baf515851c2c672f5be2c8726a6f239f3678dc97655d3ca7957ed6 **Root Cause:** Our Azure database experienced downtime due to an unknown issue. Here's the communication we received from Azure: > **Unknown Reason** > > Status: At 08:18 PM, Saturday, 27 April 2024 UTC, the Azure monitoring system detected an issue with your resource logto-eu: Your Azure Database for PostgreSQL - Flexible server is currently unavailable. We are actively working to resolve the problem. We're contacting the Azure team to see how we can mitigate similar issues in the future. https://status.logto.io/incident/361697 Sun, 28 Apr 2024 00:00:00 -0000 https://status.logto.io/incident/361697#a4f2f99d5f15278b5c47a768e1225c2f59297254996a959575d1d7ce56a3b7a9 We've noticed the increasing status code 500 error reports and we're investigating it. https://status.logto.io/incident/343488 Wed, 20 Mar 2024 09:04:00 -0000 https://status.logto.io/incident/343488#5f026ef138a131e7bad5de61033ef6826ce7ca0e50e29ad09aa78cac26b0a8ce ## Background We have recently resolved a bug related to the issuance of access tokens when using custom domain endpoints. Previously, even when a custom domain endpoint was used to request an access token, the issuer (the `iss` field) would still default to our standard domain (tenant-id.logto.app). We are pleased to announce that this issue has been fixed, and now the `iss` field will automatically reflect the domain used in the request. ## Action required If you are using a custom domain and have implemented `iss` field validation in your code, you will need to make an update. This change is critical for those who are validating the `iss` field against the expected issuer URL. ## How to proceed To align with this update, please ensure that the expected value for the `iss` field in your resource server or any relevant components is set to your custom domain format. For example, if your Logto custom domain is https://auth.your-domain.com/, it should now be https://auth.your-domain.com/oidc. --- This change was intended as a bug fix; however, it may lead to a breaking change in your existing code. If you are affected, please do not hesitate to contact us so we can make things right. We apologize for any inconvenience it may have caused. https://status.logto.io/incident/343488 Wed, 20 Mar 2024 09:04:00 -0000 https://status.logto.io/incident/343488#5f026ef138a131e7bad5de61033ef6826ce7ca0e50e29ad09aa78cac26b0a8ce ## Background We have recently resolved a bug related to the issuance of access tokens when using custom domain endpoints. Previously, even when a custom domain endpoint was used to request an access token, the issuer (the `iss` field) would still default to our standard domain (tenant-id.logto.app). We are pleased to announce that this issue has been fixed, and now the `iss` field will automatically reflect the domain used in the request. ## Action required If you are using a custom domain and have implemented `iss` field validation in your code, you will need to make an update. This change is critical for those who are validating the `iss` field against the expected issuer URL. ## How to proceed To align with this update, please ensure that the expected value for the `iss` field in your resource server or any relevant components is set to your custom domain format. For example, if your Logto custom domain is https://auth.your-domain.com/, it should now be https://auth.your-domain.com/oidc. --- This change was intended as a bug fix; however, it may lead to a breaking change in your existing code. If you are affected, please do not hesitate to contact us so we can make things right. We apologize for any inconvenience it may have caused. https://status.logto.io/incident/328835 Wed, 20 Mar 2024 09:00:00 -0000 https://status.logto.io/incident/328835#44f7753c23d7830630c176fbfa03c7601ea112ba3e9d2cbe2e4d9449f0349dfa The false positive timeout issue has been mitigated recently after adding BetterStack IPs to the whitelist. We've cleaned up those incidents for accuracy. We'll keep an eye on it. https://status.logto.io/incident/328835 Wed, 20 Mar 2024 09:00:00 -0000 https://status.logto.io/incident/328835#44f7753c23d7830630c176fbfa03c7601ea112ba3e9d2cbe2e4d9449f0349dfa The false positive timeout issue has been mitigated recently after adding BetterStack IPs to the whitelist. We've cleaned up those incidents for accuracy. We'll keep an eye on it. https://status.logto.io/incident/328835 Mon, 19 Feb 2024 06:43:00 -0000 https://status.logto.io/incident/328835#d8835c0afa8713aea5a6467f0abec0f0f8418cfc3d149a2c907fa473317caa9d We have recently detected several false positives related to timeouts. Although the services were operational, incidents were still triggered. We are investigating this to ensure a more accurate display of status. https://status.logto.io/incident/328835 Mon, 19 Feb 2024 06:43:00 -0000 https://status.logto.io/incident/328835#d8835c0afa8713aea5a6467f0abec0f0f8418cfc3d149a2c907fa473317caa9d We have recently detected several false positives related to timeouts. Although the services were operational, incidents were still triggered. We are investigating this to ensure a more accurate display of status. https://status.logto.io/incident/311523 Fri, 12 Jan 2024 10:34:00 -0000 https://status.logto.io/incident/311523#4c7d88596599ced56f3bd1b623ae7a67323eb0eeae36134677181d1d7675f209 Postmortem is available [here](https://blog.logto.io/postmortem-bad-gateway/). https://status.logto.io/incident/311523 Fri, 12 Jan 2024 10:34:00 -0000 https://status.logto.io/incident/311523#4c7d88596599ced56f3bd1b623ae7a67323eb0eeae36134677181d1d7675f209 Postmortem is available [here](https://blog.logto.io/postmortem-bad-gateway/). https://status.logto.io/incident/311523 Fri, 12 Jan 2024 00:49:00 -0000 https://status.logto.io/incident/311523#830726293191bb759035ea296c4ea9d50b2dae81bc81def8481fd653c17dca02 The issue has been resolved. Services are getting gradually recovered. A postmortem will be posted soon. https://status.logto.io/incident/311523 Fri, 12 Jan 2024 00:49:00 -0000 https://status.logto.io/incident/311523#830726293191bb759035ea296c4ea9d50b2dae81bc81def8481fd653c17dca02 The issue has been resolved. Services are getting gradually recovered. A postmortem will be posted soon. https://status.logto.io/incident/311523 Fri, 12 Jan 2024 00:24:00 -0000 https://status.logto.io/incident/311523#0eac2bd765689b05f558ef22f08fe3a6e295878cbdc8d0a9938ae938cfcd5231 We have identified the error. We are working on the fix. https://status.logto.io/incident/311523 Fri, 12 Jan 2024 00:24:00 -0000 https://status.logto.io/incident/311523#0eac2bd765689b05f558ef22f08fe3a6e295878cbdc8d0a9938ae938cfcd5231 We have identified the error. We are working on the fix. https://status.logto.io/incident/311523 Thu, 11 Jan 2024 15:42:00 -0000 https://status.logto.io/incident/311523#b46c8ae3ccdee83bb7d55a36f8f28a23d17a1770a2f53fcf50dcee5caf7c4cd0 Logto services may return 502 bad gateway error for some users. We are taking a look on it. https://status.logto.io/incident/311523 Thu, 11 Jan 2024 15:42:00 -0000 https://status.logto.io/incident/311523#b46c8ae3ccdee83bb7d55a36f8f28a23d17a1770a2f53fcf50dcee5caf7c4cd0 Logto services may return 502 bad gateway error for some users. We are taking a look on it. https://status.logto.io/incident/301545 Sun, 17 Dec 2023 10:50:00 -0000 https://status.logto.io/incident/301545#b625661b4ba46ec9b2065a6343bd3aa721407eb96c0be07ad3e601e62c657c38 Postmortem is available [here](https://blog.logto.io/postmortem-docker-image-not-found/). https://status.logto.io/incident/301545 Sun, 17 Dec 2023 04:14:00 -0000 https://status.logto.io/incident/301545#d8ba48626bd4c6191b602d966ad46222646b821e5f9d9b84635355d432ab45bc The incident has been resolved. We are delving into the details and working on protocols to ensure that a similar case will not happen again. A post-mortem will be published later. https://status.logto.io/incident/301545 Sun, 17 Dec 2023 04:09:00 -0000 https://status.logto.io/incident/301545#7bf3af41d5d831d620119ebe55414d2da29ed84634526850cc10a0819f359395 We acknowledged this incident and we are working on the fix.