https://datahubproject.io logo
Join Slack
Powered by
Hey Team, I am ingesting mysql database and this i...
# ingestion
g
Hey Team, I am ingesting mysql database and this is my recipe: 
source:
type: mysql
config:
# Coordinates
host_port:
database: tableau_advocate_lsnr
# Credentials
username:
password:
schema_pattern:
allow:
- "tableau_advocate_lsnr"
profiling:
enabled: true
profile_pattern:
allow:
- ".*standard_module_daily"
sink:
type: "datahub-rest"
config:
server: "<http://localhost:8080>"
d
the example is for bigquery but it shows where it should be
g
profile pattern and profiling are in the same level, right? and under config?
d
Right
g
I have the same exact setting but getting error.
and from the recipe code that I posted, it seems right to me.
d
Hmm, I tried the exact same config and for me it does not complain about the config.
g
do you think it can be port or connection issue?
d
do you get the same extra fields error?
g
yes
d
it should not be that after the proper indtation like the one above
Can you double check if in the config file the indent is the same as above and the file is saved.
?
g
yes sure
I checked the config and the indentation looks fine.
did you test my config?
m
Something not very well known is that 
json
is also valid 
yaml
. So you could just write the yaml in json format to avoid indentation related issues.
e.g. something like
Copy code
{ 
"source": {
    "type": "mysql",
    "config": {
        "host_port": "localhost:3306",
         ...
     }
  },
"sink": {
    "type": "file",
    "config": { 
        "filename": "metadata_events.json"
     }
   }
}
g
Thanks @mammoth-bear-12532, I am going to try that.
I did an update on datahub 8.21.0 and my config worked again. definitely it wasnt a config issue.
👍 3
b
Glad we got this resolved!
Open in Slack
PreviousNext
Powered by