I’m working on a feature where users need to upload a large number of files (sometimes 1000+) to a Google Cloud Storage bucket through my API. What I’m hoping to do is let the user make a request to my backend, and in response get one URL or token they can use to upload all their files to GCS, without needing a signed URL per file.
I came across an approach called Signed Policy Documents, it seems like you can define a policy (with a prefix, expiration, etc.), sign it, and let users POST files directly to GCS as long as they match the conditions. Each file is uploaded via a separate POST, but the policy itself only has to be generated once.
I’m having trouble finding clear, up-to-date documentation about this on the official GCP site. Most of what I’ve found is from blog posts or older discussions, so I’m unsure if it’s still a good or supported method. Is the Signed Policy Document method still supported and recommended? Is there a newer or better way to let users upload multiple files with a single signed token or credential?
The Signed Policy Document method is still supported and recommended by Google Cloud Storage for secure, direct file uploads from a browser using HTTP POST. Google Cloud client libraries, like the Node.js SDK, offer methods such as generateSignedPostPolicyV2 and generateSignedPostPolicyV4, showing that this approach is still valid and useful.
However, it’s important to note the limitation: even though your backend only needs to generate the policy once, each file must still be uploaded in a separate POST request. So, if you’re uploading 1000+ files, the client still needs to send 1000+ POST requests—one per file. The policy simplifies authentication, but it doesn’t allow you to upload multiple files in a single request.
For the specific use case of uploading 1000 or more files, the Google Cloud Storage Transfer Manager, when utilized from your backend, represents a recommended approach. The Transfer Manager allows your backend to upload many files efficiently using a single main credential (like Application Default Credentials). This simplifies server-side authentication and management. However, it’s important to note that each file is still uploaded individually to Google Cloud Storage (GCS), not as part of a single, multi-file HTTP request from the client.
Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.