I am using beam just to read data from pubsub and doing some transformation and loading the data into bigquery. When I am running my code using colob my code is runnign fine but when i am using dataflow as runner i am getting error like ‘PicklingError’ or ‘process NameError: name ‘BQ_CLIENT’ is not defined During handling of the above exception, another exception occurred’. I tried option like using
save_main_session=True but still getting error. Can someone please help me on this. Appreciate your quick reply.