Upload takes so long for a small file

Google Cloud Compute Infrastructure
1

I’m trying to perform a bulk upload of thousands of files — very small JSON blobs. On average, each upload takes around 300ms.
I’m using the save function from the File module of the google-cloud/storage SDK, like this:

await storage.bucket(bucketName).file(destFileName).save(contents);

However, when I try to process many uploads simultaneously, I start encountering Socket Hang Up and ECONNRESET errors.

The number of concurrent requests is quite lower than the quota limit.

What would be the best way to handle high-concurrency uploads in this scenario? In order the achieve the best performance?

2

Hello @processNextTIck ,
For high-concurrency uploads, it’s best to implement batching, use connection pooling, and apply retry logic with backoff strategies.
Also, limiting the number of simultaneous uploads (for example, 50–100 at a time) can help avoid ECONNRESET and Socket Hang Up errors.