A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

<!here> :mega: <https://github.com/linkedin/datahub/releases/tag/v0.8.9|Release 0.8.9> just dropped! along with the companion pip package `acryl_datahub` <https://pypi.org/project/acryl-datahub/0.8.9.0/|v0.8.9.0>.
Release Highlights:
• Support for nested structs, union types and key-value schemas in Kafka
• Support for JDBC Connector based sources in Kafka Connect
• Support for Okta as a source for User and Group metadata
• Support for using AWS Glue schema registry
Detailed notes are <https://github.com/linkedin/datahub/releases/tag/v0.8.9|here>. Enjoy! :fire:

image.png

Errors in `datahub-datahub-upgrade-job`
```2021-08-13 06:50:27.322 INFO 1 --- [ main] io.ebean.internal.DefaultContainer : DatabasePlatform name:gmsEbeanServiceConfig platform:mysql
ERROR LoggingFailureAnalysisReporter
{}
APPLICATION FAILED TO START
Field noCodeUpgrade in com.linkedin.datahub.upgrade.UpgradeCli required a bean named 'kafkaEventProducer' that could not be found.
Consider defining a bean named 'kafkaEventProducer' in your configuration.```

i have a similar problem in my mae-consumer….

```
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
14:17:03.329 [main] ERROR o.s.b.d.LoggingFailureAnalysisReporter - 

***************************
APPLICATION FAILED TO START
***************************

Description:

A component required a bean named 'simpleKafkaConsumer' that could not be found.```

and of course the same error when i try to upgrade datahub with the upgrade job… :cry:

Hey folks- just as a heads up, there is no need to upgrade on this release! The Datahub upgrade script was a one time upgrade for 8.0. We try to avoid making you run upgrade scripts as much as possible.

If we ever need you to run another upgrade script- we will let you know very explicitly!

I kept it enabled for all minor 0.8.x upgrades so far and it did not give any errors. <@U01JUBAPZPS> I don't mind disabling it. But something has broken in this upgrade job from 0.8.8 to 0.8.9

<@U027B7R23J4>: +1 on your request. We are working to isolate the problem and fix it.

support for nested structs, union types :heart_eyes: I can’t wait for seeing this part.

&gt; We try to avoid making you run upgrade scripts as much as possible.
Some feedback: from the perspective of people like me using the `datahub` helm chart, needing to manually check Slack (?) or release notes (?) for whether or not to run the DataHub upgrade job when upgrading DataHub adds friction to the upgrade process.

If it's possible, I have a strong preference for that job just always being enabled and expected to work.

<@U027GCT2L1M>: +1 on this. We'll add smoke tests that ensure `datahub-upgrade` doesn't break on releases.