Is there a way to just clean the sample data (boot...
# ingestion
w
Is there a way to just clean the sample data (bootstrap_mce.json) so that I could modify and ingest it cleanly? I've been using
datahub docker nuke
. But this removes all the containers and subsequent
datahub docker quickstart
results into pulling all the containers again over the network.
b
Not sure about the docker setup but do you see the ingested data with
datahub ingest list-runs
? then you could do a rollback of the ingest (
datahub ingest rollback --run-id
)
👀 1
p
This is taking a long time, how do I see the progress of this rollback? (assume there will be some versbose logging, but I do not see even a min logging thats enough to provide some status?)
b
for us it was really fast, but I guess it depends on the number of items to rollback. Have you ingested a lot of data in one run?
p
yeah it shows 37000+ rows against that run id
any way to run it in verbose mode @bland-orange-95847?
for smaller rows, (upto a 1500 or so), it was pretty quick
but this is not even translating to 30 times that speed, so I am assuming it is not batching or anything like that..
b
mhm I am sorry I cannot help. As we do not have so many rows I cannot reproduce
p
(insert into the table manually as a hack ??)
anycase i nuked the data and started off….