POSSIBLE DISRUPTION IN BACKEND SERVER FOR APPSHEET

We recently see the error message like this quite often. This is not only my private account, and it is happening all of the account we manage, including our client appsheet account, so possibly it is a potential glitch is down there in AppSheet Server.

ERROR

Unable to update row in table ‘xxxxxxxxxxx’. → The service sheets has thrown an exception. HttpStatusCode is TooManyRequests. Quota exceeded for quota metric ‘Read requests’ and limit ‘Read requests per minute’ of service ‘sheets.googleapis.com’ for consumer ‘project_number:953348912797’.: Message[Quota exceeded for quota metric ‘Read requests’ and limit ‘Read requests per minute’ of service ‘sheets.googleapis.com’ for consumer ‘project_number:953348912797’.] Location[ - ] Reason[rateLimitExceeded] Domain[global]


There is not routine before this error is popping us, this error is thrown randamly but pretty much frequently.

Could you check the DEV team who look after the backend stuffs?

Our test (where we see error above) is our sandbox account where nobody else are using the same account at the same time. So it does not make sense why we see such error (on edit to AppSheet Table) where no other persons interact with our Google Drive/AppSheet concurrently.

@lizlynch

@AleksiAlkio

CC @takuya_miyai

1 Like

Error message always indicate the unknown project number like 'project_number:953348912797

It seems to be a projuct number of Google Cloud ,but we dont have such a project with our Google Cloud Console. It could be project managed and hosted by AppSheet Team, which is my suspicion.

1 Like

What are you using as a data source with this? Google Sheets I assume? When did this approx. start?

backend used for those apps are all with Google Sheet as data source.

1 Like

It looks that this started to happen for the last few days range, including now (today)…

We checked the Google Cloud Console with us, but there is no project with the number the error indicated. Definitely not with us, but Google/AppSheet should host.

1 Like

@Adam-google

Any idea?

1 Like

Anyone in Google team can give me any feedback?

Still we see the same error even today continuously. Obviously there is a glitch behind the scene.

1 Like

Yes, I have encountered this issue many times recently.

The same phenomenon occurred with the AppSheet app hosted by a completely different organization.
It appears to occur when data changes are executed all at once.

(However, it is a manual update operation of less than 100 records. That’s a minor amount of data for the system to process.)

953348912797 is a strange magic project number.

We seeing bunch of failure logs with our app, the error message is not exactly the same but something like ; -

Our logs are full with those error message, we desparately need helps from AppSheet/Google team… Definitely something strange is going on the server side.

@Adam-google

@AleksiAlkio

@lizlynch

@takuya_miyai

Looks like many other service like chatgpt, anthropic, google cloud etc.. are also down along with appsheet.

1 Like

I would say AppSheet is currently down for few days , technically speaking. Bu no proper response back from AppSheet team at all. ? :sob:

I personnelly am not sure why other community members so quiet on this issue… I don’t think this is not happening to my account alone, as we observe this is happenig to our direct customers accounts.

I would suggest all the creators here to check the AUDIT LOG to see what is happenig to your apps.

\

After reading this post, I have tried the following and “for now”, it is working fine in my environment.
If I encounter similar problems, I will post here.

■Actions I have tried.
・Bulk actions.
・Bulk registration of multiple records using the Appsheet API

I have also seen that error come up randomly. It disappears when I re-try whatever action It was doing.

1 Like

Thank you for reporting this issue, we are working with the Google Sheets team to get it resolved ASAP. After investigating we see that overload protection is kicking in impacting Apps that are using Sheets as the data source. Our metrics show that changes in traffic patterns have made this more common over the last week.

We have escalated the issue and I will report back once a fix is deployed to production.

For production impacting issues like this we rely on both internal metrics and support cases to gauge the severity, please file tickets with Support (available 24/7 via both Chat and Email ) to draw attention to serious issues impacting your applications.

5 Likes

We just deployed a fix in production, this issue should be resolved for all apps. Please file tickets with Support if you see any new problems.

We will do an internal RCA to understand the root cause and to avoid any reoccurrence of this. We will also improve our automated monitoring so it detects issues like this before they are impacting Apps.

1 Like

Hi @isdal

Thanks for releasing a fix to solve the issues. We will keep carefully monitoring the apps where it throw serious of errors so far.

In the meantime, we have been seeing the error message which is not exactly the same as originally I posted. However, they are common. The AppSheet server failed to access to resources such as Sheets, Google Docs. For example,

Do you think those error would be resolved with your fix you released today? Do they have common route causes, or due to another potentially hidden reasons?

@takuya_miyai

1 Like

Unfortunately the “Google drive Internal Server error” issue will not be covered by the fix we rolled out yesterday, that error is something we are investigating separately.

1 Like