white-beach-27328
06/15/2021, 6:42 PMextraEnvs
entries in the charts. Also what does ES_BULK_REQUESTS_LIMIT
and ES_BULK_FLUSH_PERIOD
set?
* ES_BULK_REQUESTS_LIMIT
* ES_BULK_NUM_RETRIES
* ES_BULK_RETRY_INTERVAL
* ES_BULK_FLUSH_PERIODminiature-airport-96424
06/16/2021, 11:37 AMWarning FailedReleaseSync 5s (x2 over 27s) helm-operator synchronization of release 'datahub-frontend' in namespace 'datahub' failed: installation failed: template: datahub-frontend/templates/deployment.yaml:118:12: executing "datahub-frontend/templates/deployment.yaml" at <include "datahub-jmxexporter.container" .>: error calling include: template: no template "datahub-jmxexporter.container" associated with template "gotpl"
miniature-airport-96424
06/17/2021, 2:41 PM2021-06-17T14:31:48.873793757Z ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
miniature-airport-96424
06/17/2021, 2:42 PMNAME READY STATUS RESTARTS AGE
datahub-datahub-frontend-866cb6c97b-tss6f 1/1 Running 0 110s
datahub-datahub-gms-84d8f8599b-9l7pn 1/1 Running 0 110s
datahub-datahub-upgrade-job-6ktd7 1/1 Running 0 18s
datahub-elasticsearch-setup-job-gxv6d 0/1 Completed 0 108s
datahub-kafka-setup-job-97jfc 0/1 Completed 0 106s
datahub-mysql-setup-job-w6jdt 0/1 Completed 0 110s
elasticsearch-master-0 1/1 Running 0 21m
elasticsearch-master-1 1/1 Running 0 21m
elasticsearch-master-2 1/1 Running 0 21m
flux-68bdf85d98-gmblt 1/1 Running 0 15h
prerequisites-cp-schema-registry-598fd5876b-lrjcq 2/2 Running 1 21m
prerequisites-kafka-0 1/1 Running 1 21m
prerequisites-neo4j-community-0 1/1 Running 0 21m
prerequisites-zookeeper-0 1/1 Running 0 21m
elegant-summer-74561
06/18/2021, 5:43 AMsticky-television-18623
06/18/2021, 4:37 PMproud-jelly-46237
06/18/2021, 5:35 PMbland-orange-13353
06/28/2021, 3:51 PMcool-iron-6335
06/29/2021, 8:18 AM2021/06/29 08:13:35 Waiting for: <tcp://prerequisites-mysql:3306>
2021/06/29 08:13:35 Waiting for: <tcp://prerequisites-kafka:9092>
2021/06/29 08:13:35 Waiting for: <http://elasticsearch-master:9200>
2021/06/29 08:13:35 Waiting for: http:
2021/06/29 08:13:35 Problem with request: Get http:: http: no Host in request URL. Sleeping 1s
2021/06/29 08:13:35 Connected to <tcp://prerequisites-mysql:3306>
2021/06/29 08:13:35 Connected to <tcp://prerequisites-kafka:9092>
2021/06/29 08:13:35 Received 200 from <http://elasticsearch-master:9200>
2021/06/29 08:13:36 Problem with request: Get http:: http: no Host in request URL. Sleeping 1s
2021/06/29 08:13:37 Problem with request: Get http:: http: no Host in request URL. Sleeping 1s
2021/06/29 08:13:38 Problem with request: Get http:: http: no Host in request URL. Sleeping 1s
2021/06/29 08:13:39 Problem with request: Get http:: http: no Host in request URL. Sleeping 1s
2021/06/29 08:13:40 Problem with request: Get http:: http: no Host in request URL. Sleeping 1s
2021/06/29 08:13:41 Problem with request: Get http:: http: no Host in request URL. Sleeping 1s
2021/06/29 08:13:42 Problem with request: Get http:: http: no Host in request URL. Sleeping 1s
2021/06/29 08:13:43 Problem with request: Get http:: http: no Host in request URL. Sleeping 1s
2021/06/29 08:13:44 Problem with request: Get http:: http: no Host in request URL. Sleeping 1s
2021/06/29 08:13:45 Problem with request: Get http:: http: no Host in request URL. Sleeping 1s
square-activity-64562
06/30/2021, 10:39 AMsquare-activity-64562
06/30/2021, 11:55 AMnodeSelector
? I could not find anyrich-policeman-92383
07/01/2021, 6:03 AMdatahub-upgrade]$ cat ./nocode/run_upgrade.sh
#!/bin/bash
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
cd $DIR && ../datahub-upgrade.sh -u NoCodeDataMigration -a batchSize=1000 -a batchDelayMs=100 -a dbType=POSTGRES
datahub-upgrade]$ ./nocode/run_upgrade.sh
privateregistry/datahub/datahub-upgrade:head
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.1.4.RELEASE)
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See <http://www.slf4j.org/codes.html#StaticLoggerBinder> for further details.
ERROR SpringApplication Application run failed
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'upgradeCli': Unsatisfied dependency expressed through field 'noCodeUpgrade'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ebeanAspectDao' defined in class path resource [com/linkedin/gms/factory/entity/EbeanAspectDaoFactory.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.linkedin.metadata.entity.ebean.EbeanAspectDao]: Factory method 'createInstance' threw exception; nested exception is java.lang.NullPointerException
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:643)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:116)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1422)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:879)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:878)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:316)
at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:139)
at com.linkedin.datahub.upgrade.UpgradeCliApplication.main(UpgradeCliApplication.java:13)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ebeanAspectDao' defined in class path resource [com/linkedin/gms/factory/entity/EbeanAspectDaoFactory.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.linkedin.metadata.entity.ebean.EbeanAspectDao]: Factory method 'createInstance' threw exception; nested exception is java.lang.NullPointerException
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:656)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:484)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1338)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1177)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:557)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:310)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:310)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1287)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1207)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:640)
... 25 more
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.linkedin.metadata.entity.ebean.EbeanAspectDao]: Factory method 'createInstance' threw exception; nested exception is java.lang.NullPointerException
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185)
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:651)
... 42 more
Caused by: java.lang.NullPointerException
at io.ebean.datasource.pool.ConnectionPool.notifyDataSourceIsDown(ConnectionPool.java:406)
at io.ebean.datasource.pool.ConnectionPool.createUnpooledConnection(ConnectionPool.java:535)
at io.ebean.datasource.pool.ConnectionPool.createUnpooledConnection(ConnectionPool.java:524)
at io.ebean.datasource.pool.ConnectionPool.createConnectionForQueue(ConnectionPool.java:766)
at io.ebean.datasource.pool.PooledConnectionQueue.ensureMinimumConnections(PooledConnectionQueue.java:227)
at io.ebean.datasource.pool.ConnectionPool.initialise(ConnectionPool.java:301)
at io.ebean.datasource.pool.ConnectionPool.<init>(ConnectionPool.java:246)
at io.ebean.datasource.core.Factory.createPool(Factory.java:15)
at io.ebeaninternal.server.core.DefaultContainer.getDataSourceFromConfig(DefaultContainer.java:273)
at io.ebeaninternal.server.core.DefaultContainer.setDataSource(DefaultContainer.java:217)
at io.ebeaninternal.server.core.DefaultContainer.createServer(DefaultContainer.java:103)
at io.ebeaninternal.server.core.DefaultContainer.createServer(DefaultContainer.java:35)
at io.ebean.EbeanServerFactory.createInternal(EbeanServerFactory.java:109)
at io.ebean.EbeanServerFactory.create(EbeanServerFactory.java:70)
at com.linkedin.metadata.entity.ebean.EbeanAspectDao.createServer(EbeanAspectDao.java:594)
at com.linkedin.metadata.entity.ebean.EbeanAspectDao.<init>(EbeanAspectDao.java:65)
at com.linkedin.gms.factory.entity.EbeanAspectDaoFactory.createInstance(EbeanAspectDaoFactory.java:24)
at com.linkedin.gms.factory.entity.EbeanAspectDaoFactory$$EnhancerBySpringCGLIB$$10d70657.CGLIB$createInstance$0(<generated>)
at com.linkedin.gms.factory.entity.EbeanAspectDaoFactory$$EnhancerBySpringCGLIB$$10d70657$$FastClassBySpringCGLIB$$68ed9ed1.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:363)
at com.linkedin.gms.factory.entity.EbeanAspectDaoFactory$$EnhancerBySpringCGLIB$$10d70657.createInstance(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
... 43 more
rich-policeman-92383
07/01/2021, 6:03 AMglamorous-kite-95510
07/02/2021, 7:27 AMsticky-television-18623
07/02/2021, 6:22 PMcool-iron-6335
07/06/2021, 7:58 AMprerequisites/Chart.yml
). Everything worked well but i got this error Exception while fetching data (/browse) : java.lang.RuntimeException: Failed to execute browse: entity type: DATASET, path: [], filters: null, start: 0, count: 10
when i browsed on UIadamant-pharmacist-61996
07/06/2021, 10:34 AMblue-holiday-20644
07/06/2021, 12:56 PMsquare-activity-64562
07/07/2021, 5:04 AMdatahub docker nuke
is not working in 0.8.5
docker.errors.APIError: 409 Client Error for <http+docker://localhost/v1.41/volumes/datahub_mysqldata>: Conflict ("remove datahub_mysqldata: volume is in use - [95df02e070a6a5685c6c56fd59de4b4c3d482963dadd72e9138742ac0f2411fe]")
It seems the order of removal is wrong and it is trying to remove volumes before containersfresh-fish-73471
07/07/2021, 9:29 AMsquare-activity-64562
07/08/2021, 10:20 AMglobal.kafka.*
settings. There is no mention of any API key to authenticate with confluent cloudcool-iron-6335
07/08/2021, 11:09 AMdatahub/charts/datahub-frontend/values.yaml
file. But when i check kubectl get ingress
, there is no resource. How can i expose the 9002 port to the public ?square-activity-64562
07/08/2021, 12:14 PMdatahub-datahub-gms
secret "neo4j-secrets" not found: CreateContainerConfigError
but my neo4j
is disabled. This is datahub chart 0.2.1
with app version 0.8.3
. I guess the helm chart was not updated on the helm repository
helm search repo datahub --versions
NAME CHART VERSION APP VERSION DESCRIPTION
datahub/datahub 0.2.1 0.8.3 A Helm chart for LinkedIn DataHub
datahub/datahub 0.2.0 0.8.1 A Helm chart for LinkedIn DataHub
datahub/datahub 0.1.2 latest A Helm chart for LinkedIn DataHub
datahub/datahub-prerequisites 0.0.1 A Helm chart for packages that Datahub depends on
square-activity-64562
07/08/2021, 12:34 PMindexPrefix
for elasticsearch work? I see there is an option for that https://github.com/linkedin/datahub/blob/eb60da975dce1f855883e628541cdbcef89717af/datahub-kubernetes/datahub/quickstart-values.yaml#L51 as well as in https://datahubproject.io/docs/deploy/aws/
I have this value in my kubernetes cluster
global:
graph_service_impl: elasticsearch
datahub_analytics_enabled: true
datahub_standalone_consumers_enabled: false
elasticsearch:
host: "elasticsearch-master.apps.svc.cluster.local"
port: "9200"
indexPrefix: datahub
The elasticsearch setup job is complete. But I am seeing this in my indices. There is no "datahub" prefix for the index names. Did I understand this option incorrectly?
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open datasetdocument lHs5ppPkTgWYBud82GEKvw 1 1 0 0 416b 208b
green open dashboarddocument 84GsR1ZORtyRaX7yMdgOxg 1 1 0 0 416b 208b
green open glossaryterminfodocument GqaUw-bESp69Hp1tiNNXRA 1 1 0 0 416b 208b
green open chartdocument wcR3nE1mT3ut8f9yVb_VKA 1 1 0 0 416b 208b
green open datajobdocument Y4HSYjrHQbmcYx5ioFK-yg 1 1 0 0 416b 208b
green open dataflowdocument tYOFqQYZRYuFd3z6vhr1Kg 1 1 0 0 416b 208b
green open dataprocessdocument QMiZcLs6SR6aEp0d6s3wyA 1 1 0 0 416b 208b
green open mlmodeldocument Yr9TxaRHTvWiDDGpRNiwCA 1 1 0 0 416b 208b
green open glossarynodeinfodocument _vo5wMgKRAqOQWkDSS-nVA 1 1 0 0 416b 208b
green open corpuserinfodocument zSJp34coQSyB7vjJBj4BFQ 1 1 0 0 416b 208b
green open tagdocument F2EqZ8cTTl-xd5g3eode7w 1 1 0 0 416b 208b
adventurous-scooter-52064
07/09/2021, 4:58 AMblue-holiday-20644
07/09/2021, 3:46 PMnice-yak-49999
07/12/2021, 2:11 AMsquare-activity-64562
07/12/2021, 8:06 AMdatahub-frontend:
enabled: true
image:
repository: linkedin/datahub-frontend-react
tag: "v0.8.5"
# Set up ingress to expose react front-end
ingress:
enabled: false
but the ingress is still being created. Same for datahub-gms
This is with
chart_ref: datahub/datahub
chart_version: 0.2.6
square-activity-64562
07/12/2021, 12:38 PMcool-iron-6335
07/13/2021, 3:44 AMType Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 9m48s default-scheduler Successfully assigned default/datahub-datahub-mce-consumer-5775f5c487-8z9dr to minikube
Warning FailedMount 9m48s kubelet MountVolume.SetUp failed for volume "datahub-datahub-mce-consumer-token-8s9sh" : failed to sync secret cache: timed out waiting for the condition
Normal Pulling 9m47s kubelet Pulling image "linkedin/datahub-mce-consumer:v0.8.6"
Normal Pulled 8m29s kubelet Successfully pulled image "linkedin/datahub-mce-consumer:v0.8.6" in 1m18.086509993s
Normal Created 8m29s kubelet Created container datahub-mce-consumer
Normal Started 8m28s kubelet Started container datahub-mce-consumer