square-activity-64562
07/22/2021, 9:26 AMsquare-activity-64562
07/23/2021, 4:51 AMfaint-hair-91313
07/24/2021, 12:04 PMsquare-activity-64562
07/25/2021, 6:41 AMsample values
is something that I am not sure I would like to have in the UI. So wanted to try to ingest it once. But in case wanted to revert it should be able to do thatstrong-restaurant-35629
07/26/2021, 1:48 PMmillions-engineer-56536
07/26/2021, 9:11 PMsquare-activity-64562
07/28/2021, 7:44 AMsquare-activity-64562
07/28/2021, 8:31 AMusers -> owner of number of datasets
groups -> owner of number of datasets
quick-restaurant-75578
07/29/2021, 3:09 AMsquare-activity-64562
07/30/2021, 7:42 AMbig-carpet-38439
07/30/2021, 3:16 PMbetter-orange-49102
08/02/2021, 9:34 AMmammoth-bear-12532
mammoth-bear-12532
quick-restaurant-75578
08/03/2021, 3:38 AMsquare-activity-64562
08/03/2021, 8:45 AMDatasetLineageTypeClass
https://github.com/linkedin/datahub/blob/5eee818a619e27300f8ae3cf749d3f82aa23f43e/metadata-ingestion/src/datahub/metadata/schema_classes.py#L2279
Does it show up differently in the UI?ambitious-airline-8020
08/03/2021, 12:49 PMdocker/nuke.sh
, just drop meta
P.S. It is about dev env, running from docker/dev.sh
, so not related to any production casesmammoth-bear-12532
acryl_datahub
PyPi package (version 0.8.7.0).
Looking forward to hearing all the awesome things you all will do with this!prehistoric-yak-75049
08/05/2021, 6:41 PMdocker pull acryldata/datahub-upgrade:head && docker run --env-file ~/setup/docker/datahub-upgrade/env/docker.env acryldata/datahub-upgrade:head -u NoCodeDataMigration
but getting below exception to build docker image
ERROR SpringApplication Application run failed
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'upgradeCli': Unsatisfied dependency expressed through field 'noCodeUpgrade'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'com.linkedin.gms.factory.entityregistry.EntityRegistryFactory': Unsatisfied dependency expressed through field 'configEntityRegistry'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'configEntityRegistry' defined in class path resource [com/linkedin/gms/factory/entityregistry/ConfigEntityRegistryFactory.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.linkedin.metadata.models.registry.ConfigEntityRegistry]: Factory method 'getInstance' threw exception; nested exception is java.io.FileNotFoundException: ../../metadata-models/src/main/resources/entity-registry.yml (No such file or directory)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:643)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:116)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1422)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:879)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:878)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:316)
at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:139)
at com.linkedin.datahub.upgrade.UpgradeCliApplication.main(UpgradeCliApplication.java:13)
...
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'com.linkedin.gms.factory.entityregistry.EntityRegistryFactory': Unsatisfied dependency expressed through field 'configEntityRegistry'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'configEntityRegistry' defined in class path resource [com/linkedin/gms/factory/entityregistry/ConfigEntityRegistryFactory.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.linkedin.metadata.models.registry.ConfigEntityRegistry]: Factory method 'getInstance' threw exception; nested exception is java.io.FileNotFoundException: ../../metadata-models/src/main/resources/entity-registry.yml (No such file or directory)
crooked-toddler-8683
08/06/2021, 4:48 PMwitty-butcher-82399
08/09/2021, 12:29 PMmagnificent-camera-71872
08/10/2021, 1:29 AMmagnificent-camera-71872
08/10/2021, 3:32 AMblue-megabyte-68048
08/10/2021, 12:53 PMmammoth-bear-12532
blue-megabyte-68048
08/11/2021, 4:38 PMmagnificent-camera-71872
08/12/2021, 5:14 AMcurl -H 'X-RestLi-Protocol-Version:2.0.0' -H 'X-RestLi-Method: get' '<https://datahub.xxxxxxxx-nonprod-1.yyyyyyyy-datalake-nonprod.com.au/dataset/urn%3Ali%3Adataset%3A(urn%3Ali%3AdataPlatform:redshift,datalake.kayo_datalake_transform_current.ares_masterindex_daily_snapshot_view,DEV)/schema>'
I thought maybe I'm using the wrong endpoint, but curl for the config works fine:
curl '<https://datahub.xxxxxxxx-nonprod-1.yyyyyyyy-datalake-nonprod.com.au/config>'
Does anyone have any ideas ?bland-orange-13353
08/12/2021, 6:59 PMmammoth-bear-12532
acryl_datahub
v0.8.9.0.
Release Highlights:
β’ Support for nested structs, union types and key-value schemas in Kafka
β’ Support for JDBC Connector based sources in Kafka Connect
β’ Support for Okta as a source for User and Group metadata
β’ Support for using AWS Glue schema registry
Detailed notes are here. Enjoy! π₯microscopic-musician-99632
08/13/2021, 5:52 AM