AppSheet is fully back up again. While we will do a deeper analysis, it appears this happened because of automatic security enforcement rules that (erroneously) shutdown firewall access for our own services. It took a while to discover this, and once we did, we had to manually override this change and disable the automatic enforcer.
This is about all we know at the moment. Exactly why this happened at 2:25PM PST on a Tuesday is unclear and we are determined to find out.
We are aware of and actively investigating a major incident. Will provide status updates here.
The issue started approx 2:25PM PST. No AppSheet deployment was ongoing at that time. It appears to be a network configuration issue. We’re trying to determine what exactly. At the moment, our services are receiving no network traffic and that is why nobody is able to access the services.
It’s going to be slow for a few minutes as the server farm ramps up again (scales out with many instances — all the instances had scaled down as the traffic completely dropped).
Hi @Cesferb , not it should not have. Any data uploads during that period would have failed and the changes would have been queued locally until they could have been saved.
The only exception is if you were running in a browser and you closed the browser before the data was synced to the server,
I am using some bots to call webhook(both data change trigger and schedule trigger) and these worked well till 8/9/2021 9:30:01 PM(UTC). After that time, bots are working but all webhook gots timeout. Exception message is as below:
"Exception": "Webhook HTTP post request failed with exception Unable to connect to the remote server A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 10.41.0.250:8080",
for checking purpose, I call webhook from other channel and went well without any problem.
