hello, I am currently implementing the DatahubSpar...
# ingestion
d
hello, I am currently implementing the DatahubSparkListener. We have a token setup, but it gets logged in the spark submit command. Is there anyway to hide the token or configure the token through environment variables?
h
Hi @dazzling-judge-80093, could you help answer this question?
Hi @delightful-barista-90363, could you try if the following syntax works
"spark.datahub.rest.token" = ${?DATAHUB_TOKEN},
for passing the token via the environment variable
DATAHUB_TOKEN
?
d
I can try that out
Hey, could not get this to work since we are using subprocess in python (without shell)