Hi, I have an issue with Parquet ingestion. When I...
# ingestion
p
Hi, I have an issue with Parquet ingestion. When I execute an ingestion then every Parquet file will be a standalone ingestion, and the {table}, {partition_key} and {partition} marks has no effects. "c:/users/szger/PARQUET/{table}/{partition_key[0]}={partition[0]}/*.parquet" It is creating more hundred entries in the output file, one for every parquet file. I attach the sample parquet folders too. What I expecting is that every table has only one schema definition in the file.
βœ… 1
l
Hey there πŸ‘‹ I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: βœ… There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? βœ… button 2️⃣ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues? Yes button Did you find a solution to your issue?
p
no
I still cannot manage this
a
Hi There, I responded in your more recent thread, let’s troubleshoot there!