We want to migrate data from on-perm SAP IBP data store to Google BigQuery using Google Cloud Datafusion.
Please see attached image (bottom) for more info.
Description:-
- We have data in on-perm SAP IBP data store
- We need to develop and run data pipeline in Google Cloud Datafusion and store data into Google BigQuery
- SAP IBP certificate is stored in a GCS bucket
- In Cloud Datafusion, we enabled SSL/tls (keystore/truststore) certificate to read it from a GCS bucket.
- Cloud Dataproc uses that certificate stored in GCS, connect to SAP IBP datastore, read data, run ETL, and Store data into Google BigQuery
- We are using the existing Dataproc cluster (not the default one created by Datafusion internally)
Problem:-
When we run this data pipeline, we are getting the following error:
Failed to create an SSL connection factory.
java.nio.file.NoSuchFileException
Questions:-
Where (source or target) and how to configure SAP IBP certificate in Cloud Datafusion to read by Dataproc cluster to connect to on-perm SAP IBP datastore? Is it possible to create a Dataproc cluster by specifying an SSL certificate?
For more info please ream my Linkedin post @ https://www.linkedin.com/posts/rambabu-posa_ramgcpdiscussions-gcpcloud-gcpengineer-activity-6839086257553907713-XsHb/