https://datahubproject.io logo
Join Slack
Powered by
Hi, everyone, how to use GE? Even if it's not.Yaml...
# troubleshoot
l
Hi, everyone, how to use GE? Even if it's not.Yaml, it's the same alarm
Copy code
Error Msg: Could not find checkpoint 'xxxxxx' (or its configuration is invalid)
m
How have you created the checkpoint file?? I remember getting that error if I did it "locally". The solution was to create it via the command 
great_expectations checkpoint new
and create the files via a jupyter notebook. The URL of jupyter is indicated after executing the commands.
This are the steps that I followed to create all the files needed: 1. great_expectations datasource new --no-jupyter 2. Enter option 2 3. Enter option 2 4. Install the dependency 
psycopg2-binary
if it is asked to 5. jupyter notebook /great_expectations/uncommitted/datasource_new.ipynb --allow-root --ip 0.0.0.0 6. great_expectations suite new 7. Option 2 8. Index of the table of which you want to create the suite 9. Enter the file name 10. great_expectations suite edit <nombre fichero> --no-jupyter 11. jupyter notebook /great_expectations/uncommitted/edit_<nombre fichero>.ipynb ---allow-root –ip 0.0.0.0 12. great_expectations checkpoint new <name> --no-jupyter 13. jupyter notebook /great_expectations/uncommitted/edit_checkpoint_<name>.ipynb --allow-root --ip 0.0.0.0 This pages will help in the 11th step: https://docs.greatexpectations.io/docs/guides/expectations/how_to_create_and_edit_expectations_based_on_domain_knowledge_without_inspecting_data_directly/ https://docs.greatexpectations.io/docs/tutorials/getting_started/tutorial_create_expectations/
I think this should be enough for you to be able to create all the files needed!
g
@microscopic-mechanic-13766, what I need to do in the 5th step?
I already open the notebook, what the next step? btw, this steps to generate 
Checkpoint
right? https://datahubproject.io/docs/metadata-ingestion/integration_docs/great-expectations
m
So after the 5th step you would have to create the file that would indicate the checkpoint (if I am not mistaken) the source from which it should make the test. Same thing would happen with the suite
Both files are needed for the creation and correct execution of the checkpoint
In the notebook should appear some guides of what you should write there. Anyways I will try to send you a sample file of each asap for better understanding
g
thanks @microscopic-mechanic-13766. tried to understand, but cant find any sample of checkpoints file. hence, a bit confused.
m
Note that as it was made in jupyter notebook, the language that is made on is Python. You can also notice that although the language is Python, internally it is used the format YAML
Hope this helps. You must notice that all 3 files are needed and that if one is incorrectly build or has some mistakes it will make the checkpoint fail.
g
@microscopic-mechanic-13766, meaning every time I want to add new checkpoint, expectation. I need to use 
notebook
?
from my understanding, after run 
notebook
it will automatically create a yaml file
actually, I tried using the step you shared before 🙂 its wonderful. I got the result show in DataHub.. just not very clear on add new checkpoint bla bla bla
btw, its awesome! thanks much for your help @microscopic-mechanic-13766
m
Theorically there are other ways, but I haven't made then work. The notebook is the easiest way. I know it is not the best option and can be tedious but haven't found any other way to work it around. If you find it I will be glad to hear about it 🙂
g
yeah, not sure if I can copy current file and replace some important field for a new one.
m
It will not create a yml file but a jupyter notebook in python that will internally use YAML
Open in Slack
PreviousNext
Powered by