Hi, Can i ask where is my metadata is stored on my local machine ? How can i backup my metadata just in case something bad happened ? If it's okay, can you give me the directory ? Once I start docker, everything runs perfectly and I have no clue where it is ?

docker volumes are stored in /var/lib/docker/volumes

The main data is stored in a mysql database, with secondary state stores being elasticsearch and neo4j. cc @early-lamp-41924 , who is working on the backup/restore story

As Harshal mentioned, the source of truth for datahub is mysql. As of now, you can recreate all the other components based on the mysql database. I will be sending out a change that let’s you recreate search and graph indices from the local db, and a framework for restoring mysql and other storage components from a snapshot. We will continue improving this to take in snapshots of any format

If you ran datahub with

datahub docker quickstart

, then everything is stored locally. It is possible to export your data and move it to other places, or to query our metadata using our APIs

If you’re having trouble with quickstart, please make sure you’re using the most up-to-date version of DataHub by following the steps in the quickstart deployment guide: https://datahubproject.io/docs/quickstart/#deploying-datahub. Specifically, ensure you’re up to date with the DataHub CLI:

python3 -m pip install --upgrade pip wheel setuptools
python3 -m pip install --upgrade acryl-datahub
datahub version

Correct, used

datahub docker quickstart

. Much appreciated, what I was hoping to confirm!

If you’re having trouble with quickstart, please make sure you’re using the most up-to-date version of DataHub by following the steps in the quickstart deployment guide: https://datahubproject.io/docs/quickstart/#deploying-datahub. Specifically, ensure you’re up to date with the DataHub CLI:

python3 -m pip install --upgrade pip wheel setuptools
python3 -m pip install --upgrade acryl-datahub
datahub version