A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

Hello People

I need help while ingesting data from mongo as source and datahub reset sink. I get below error while ingesting data from mongodb
Error:
3 validation errors for mongoDBConfig
env extra field not permitted
enableSchemaInference: True not permitted
schemaSamplingSize: 1000 not permitted

These parameters were introduced in a recent version. You will need to upgrade your version of acryl-datahub to 0.3.3

<https://pypi.org/project/acryl-datahub/0.3.3/>

<@U01C3DMG2GL> Thanks version 0.3.3 worked well and we were able to inference schema from mongodb.
Also looks like the env parameter is not supported by mongodb source.

<@U02255U7NR3> Could you please elaborate on what triggered the error with `env` ? Were you including it in your yml?

<@U01U69UJNUF>
```## command Executed
datahub ingest -c exmaples/mongodb_to_datahub.yml

# cat mongodb_to_datahub.yml

source:
  type: "mongodb"
  config:
    connect_uri: "<mongodb://localhost>"
    username: admin
    password: password
    authMechanism: "DEFAULT"
    env: "DEV"
    options: {}
    database_pattern: {}
    collection_pattern: {}
    enableSchemaInference: True
    schemaSamplingSize: 1000
    # database_pattern/collection_pattern are similar to schema_pattern/table_pattern from above

sink:
  type: "datahub-rest"
  config:
    server: "<http://localhost:8080>"


# Error encountered:

1 validation erro for MongoDBConfig
env
  extra fields not permitted (type=value_error.extra)```

<@U01U69UJNUF> please suggest how we can change name prod to some other value

Hey <@U02255U7NR3>! Sorry for not getting back to you. Which version of DataHub are you running? I believe we may have fixed a similar bug earlier.

Hmm could you give v0.8 or later a shot? In the meantime I'll try testing on this earlier version myself