The issue “Illegal base64 character 2d” likely shows that there is a mistake with how the SSH private key is being handled, most likely because of how the key is taken from the Google Cloud Secret Manager and the formatting expected by Dataform.
To solve the issue at hand, I suggest these specific steps:
Check If the Right Private Key Format is Used
Usually, Dataform along with the related git client expects the private key to be in OpenSSH PEM format and expects certain metadata, line breaks, and extra information to be missing. It is possible some newer OpenSSH private keys (starting with -----BEGIN OPENSSH PRIVATE KEY-----) do not work well. When dealing with this type of key, my advice is to convert it to the older PEM format:
ssh-keygen -p -m PEM -f ~/.ssh/id_rsa
This command is useful as it changes the private key to the “PEM format (-----BEGIN RSA PRIVATE KEY-----)” which is reliably supported.
Do Not Use File Upload in the Secret Manager UI
Using the “Upload file” feature through the Secret Manager UI automatically changes line endings and adjusts hidden characters. This behavior affects the private key when this method is used. Instead, I suggest splitting the file and adding the private key as plain text secret:
cat ~/.ssh/id_rsa | gcloud secrets versions add YOUR_SECRET_NAME --data-file=-
This keeps the line breaks exact as it captures the secret without adding any extra or hidden marks.
Check for Hidden or Invalid Characters
Check the private key file to make sure there are no extra whitespace characters, line endings (\r), or invalid base64 characters (the dash ‘-’ is ASCII 0x2d which may show up if line breaks are read wrong). Use:
cat -A ~/.ssh/id_rsa
to check unwanted characters.
Confirm Secret Access Control
Make sure the service account of Dataform has permission for the secret and check that no changes (e.g., automatic base64 encoding /decoding) happen on retrieval.
Consider Using SSH with App Passwords as a Temporary Step
Because HTTPS authentication works, it would make sense to switch to using Bitbucket App Passwords via HTTPS while fixing SSH as a temporary workaround.
Check Dataform and GCP Documentation/Support
This usually happens because of a small logic error inside the combination of Dataform and Google Cloud SDK, especially how secrets are retrieved and SSH keys are read. Look into recent updates, known problems, or maybe send a support case with Dataform or GCP.