Hi. I Want to extract groups of an user logged in via azure ad, can anyone help on how to setup group claim in azure portal?

What is the purpose of system_metadata_service_v1 index in Elasticsearch? I see a lot of ingestion events present there (e.g. datasets)?

@mammoth-bear-12532 We were testing the ability to add a new entity type by using a custom repository and we ran into an exception that tell us that adding a new entity type that way is not supported. You told us earlier that this should be possible, but it doesn't seem to be. FWIW I can see from the commit history that you were the one that added the exception logic. Does this check just need to be removed, or is there some other piece of the puzzle missing? If it can be removed, I can try making the change and sending a pull request.

Hello - did something change recently with the Snowflake Usage plugin? We started seeing this error since yesterday

[2022-09-04, 01:37:48 UTC] {{cursor.py:710}} INFO - query: [SELECT -- access_history.query_id, -- only for debugging purposes access_history...]
[2022-09-04, 01:44:37 UTC] {{cursor.py:734}} INFO - query execution done
[2022-09-04, 01:44:40 UTC] {{stateful_ingestion_base.py:93}} WARNING - usage => Failed to parse usage line {'query_start_time': datetime.datetime(2022, 9, 4, 1, 30, 6, tzinfo=datetime.timezone.utc), 'query_text': "select TABLE_NAME, TIMEDIFF('hour', LAST_ALTERED, CURRENT_TIMESTAMP()) AS T\nfrom SUMUP_DWH_PROD.INFORMATION_SCHEMA.TABLES\nwhere 1=1\nAND TABLE_SCHEMA = 'SRC_PLUGGER'\nHAVING T > 6", 'query_type': 'SELECT', 'rows_inserted': 0, 'rows_updated': 0, 'rows_deleted': 0, 'base_objects_accessed': [], 'direct_objects_accessed': [{'columns': [{'columnName': 'TABLE_NAME'}, {'columnName': 'TABLE_SCHEMA'}, {'columnName': 'LAST_ALTERED'}], 'objectDomain': 'View', 'objectName': 'INFORMATION_SCHEMA.TABLES'}], 'user_name': 'ADWH_DATA_CATALOG', 'first_name': None, 'last_name': None, 'display_name': 'DATA_CATALOG', 'email': '<mailto:DATA.CATALOG@sumup.com|DATA.CATALOG@sumup.com>', 'role_name': 'ACCOUNT_MONITOR_USAGE'}, 4 validation errors for SnowflakeJoinedAccessEvent
direct_objects_accessed -> 0 -> columns -> 0
  __init__() missing 1 required positional argument: 'columnId' (type=type_error)
direct_objects_accessed -> 0 -> columns -> 1
  __init__() missing 1 required positional argument: 'columnId' (type=type_error)
direct_objects_accessed -> 0 -> columns -> 2
  __init__() missing 1 required positional argument: 'columnId' (type=type_error)
direct_objects_accessed -> 0 -> objectId
  field required (type=value_error.missing)

I am trying tableau ingestion. I'm getting the following error and I'm not sure why I'm getting this error. The following is the log of the result of ingestion with the debug option turned on. Hope you can help me what this log means.

datahub --debug ingestion -c tableau-recipe.yml

Hi I am trying to do a delete request on

entities/v1/

endpoint as documented on

my-datahub-instance/openapi/swagger-ui/index.html#/Entities/deleteEntities

. But I am getting:

DELETE operation not supported for URI: '/entities/v1/'

Is this intended or am I doing something wrong?

Hi! I am new here, testing possibilities with datahub. With ingestion and profiling, I have a problem. My recipe looks like this source: type: mssql The ingestion works, but I dont see any stats. But there is a bunch of error messages right before:

"AttributeError: 'Insert' object has no attribute 'columns'\n"
'[2022-08-31 09:46:52,291] ERROR    {datahub.utilities.sqlalchemy_query_combiner:249} - Failed to execute query normally, using '
           'fallback: \n'
           'CREATE TABLE "#ge_temp_dbf5dfdd" (\n'
           '\tcondition INTEGER NOT NULL\n'
           ')\n'
           '\n'

I found this question in the history of Slack, but there is no answer there. Could you help me with this problem?🙏

Hey guys, after upgrading from v0.8.43 to v0.8.44 i'm not able to run ingest via UI anymore, getting the same message for any recipe:

~~~~ Execution Summary ~~~~

RUN_INGEST - {'errors': [],
 'exec_id': '6162dff1-1fc1-49a9-8e04-1b5519458091',
 'infos': ['2022-09-05 22:16:28.588080 [exec_id=6162dff1-1fc1-49a9-8e04-1b5519458091] INFO: Starting execution for task with name=RUN_INGEST',
           '2022-09-05 22:16:28.588763 [exec_id=6162dff1-1fc1-49a9-8e04-1b5519458091] INFO: Caught exception EXECUTING '
           'task_id=6162dff1-1fc1-49a9-8e04-1b5519458091, name=RUN_INGEST, stacktrace=Traceback (most recent call last):\n'
           '  File "/usr/local/lib/python3.9/site-packages/acryl/executor/execution/default_executor.py", line 121, in execute_task\n'
           '    self.event_loop.run_until_complete(task_future)\n'
           '  File "/usr/local/lib/python3.9/site-packages/nest_asyncio.py", line 89, in run_until_complete\n'
           '    return f.result()\n'
           '  File "/usr/local/lib/python3.9/asyncio/futures.py", line 201, in result\n'
           '    raise self._exception\n'
           '  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 256, in __step\n'
           '    result = coro.send(None)\n'
           '  File "/usr/local/lib/python3.9/site-packages/acryl/executor/execution/sub_process_ingestion_task.py", line 71, in execute\n'
           '    validated_args = SubProcessIngestionTaskArgs.parse_obj(args)\n'
           '  File "pydantic/main.py", line 521, in pydantic.main.BaseModel.parse_obj\n'
           '  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__\n'
           'pydantic.error_wrappers.ValidationError: 1 validation error for SubProcessIngestionTaskArgs\n'
           'debug_mode\n'
           '  extra fields not permitted (type=value_error.extra)\n']}
Execution finished with errors.

I tried to recreate the recipe using the UI, but i'm getting the same error. Any thoughts? PS: I'm using datahub-helm (v0.2.93) and ingestion is handled by datahub-actions (v0.0.4)

Hi all, does anyone know why the "details" icon is not working after ingesting metadata of a table ?

Hey guys, Is there a way to combine set of files into 1 metadata? e.g. currently I have a folder in S3 bucket that contains multiple csv files (same data generated on different days), can I create a metadata in DataHub for all of them as if they were one ?

Hello Team, i’m trying to test upgrade job restore indices so i’ve deleted all indices of elasticsearch and tried to restore it according to Mysql i tested it by editing datahub-upgrade-job.yml from:

args:

- "-u"

- "NoCodeDataMigration"

- "-a"

- "batchSize=1000"

- "-a"

- "batchDelayMs=100"

- "-a"

- "dbType={{ .Values.datahubUpgrade.noCodeDataMigration.sqlDbType }}"

to:

args:

- "-u"

- "RestoreIndices"

but i get log ‘no active profile set’ so my question is Is RestoreIndices’s role is what i’m expecting ? how can i debug that situation??

Hi Team, I am trying to ingest the kafka data into datahub Can someone please help me with the cluster_id? How should I find my cluster_id and what should I put in the platform_instance? Below is the config that I have source: type: kafka config: # Replace with your cluster ID platform_instance: connection: bootstrap: "http://prerequisites-kafka.telco-dataprocessing-mvp.svc.cluster.local:9092" # Uncomment and add secrets in Secrets Tab # consumer_config: # security.protocol: "SASL_SSL" # sasl.mechanism: "PLAIN" # sasl.username: "${CLUSTER_API_KEY_ID}" # sasl.password: "${CLUSTER_API_KEY_SECRET}" schema_registry_url: "http://prerequisites-kafka.telco-dataprocessing-mvp.svc.cluster.local:9092" # Uncomment and add secrets in Secrets Tab # schema_registry_config: # basic.auth.user.info: "${REGISTRY_API_KEY_ID}:${REGISTRY_API_KEY_SECRET}" sink: type: datahub-rest config: server: 'http://datahub-mvp-datahub-gms.telco-dataprocessing-mvp.svc.cluster.local:8080' # Add a secret in secrets Tab token: null

hello guys, I am having the following issue: I got a request from a user that she wasn't able to delete a glossary term under a glossary node. I did delete it from the datahub cli, I did a hard delete. It didn't have a name, it was in a form of an id. The problem is that it is still visible under the UI and now I can't move it or delete it, as it is already deleted. Any ideas on how I can get rid of it from the UI? Thanks

Another problem that I am facing is that I can't add description, tag or term in an element which is under an array or a struct in a dataset... The error that I am getting is this:

Update Failed! Failed to update urn:li:dataset:(urn:li:dataPlatform:dbt,gluegen.raw_events_testapp06.eis_motorquote_2,DEV) & field [version=2.0].[type=struct].[type=array].[type=struct].vehicles.[type=string].uuid. Field [version=2.0].[type=struct].[type=array].[type=struct].vehicles.[type=string].uuid does not exist in the datasets schema.

This message was deleted.

Hey team, I'm having an issue installing datahub: Here is what I did:

python3 -m pip install --upgrade pip wheel setuptools
python3 -m pip uninstall datahub acryl-datahub || true  # sanity check - ok if it fails
python3 -m pip install --upgrade acryl-datahub
datahub version
datahub docker quickstart

And I'm getting the error:

PermissionError: [Errno 13] Permission denied: '/home/stephane/.datahub/plugins/auth'
[2022-09-06 11:22:25,543] ERROR    {datahub.entrypoints:195} - Command failed: 
	[Errno 13] Permission denied: '/home/stephane/.datahub/plugins/auth'.

Any ideas?

Hey, all! I was testing ingesting DatasetProperties with OpenAPI and whenever I make the POST call with random stuff on the URI field the dataset UI page brakes. I can still navigate to it but whenever I click on the dataset itself the page shows up as the screenshot bellow. Is this an expected behavior?

hey team How can i check if a validation rule exists or not by python code?

Hello, I hope everyone is having a great day today I wanted to ask - do you guys have some code examples in github on how to use your SDK for ingesting a data? Currently I am ingesting via the curl command and it looks like it can be achieved in much simpler way using the SDK. I have found some code in your repository for integration tests i.e: https://github.com/datahub-project/datahub/blob/85a55ffac7b4cfa4594bb93cc960656886bbc440/metadata-ingestion/tests/integration/kafka/test_kafka.py This example above uses the mce_helpers and I am looking for some example that is not a test framework, do you have that somewhere? Thanks a lot for a reply in an advance! : )

Hello! I get this error when trying to get the profiling for my snowflake's tables:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/source/ge_data_profiler.py", line 924, in _generate_single_profile
    profile = _SingleDatasetProfiler(
  File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/source/ge_data_profiler.py", line 511, in generate_dataset_profile
    self.query_combiner.flush()
  File "/usr/local/lib/python3.9/site-packages/datahub/utilities/sqlalchemy_query_combiner.py", line 395, in flush
    let.switch()
  File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/source/ge_data_profiler.py", line 236, in <lambda>
    return self.query_combiner.run(lambda: method(self, *args, **kwargs))
  File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/source/ge_data_profiler.py", line 490, in _get_dataset_column_sample_values
    str(v) for v in res["partial_unexpected_list"]
KeyError: 'partial_unexpected_list'

Can you help me? Thanks!

Hello. Why i get this error in graphql?

query{
  dataset(urn:"urn:li:dataset:(urn:li:dataPlatform:vertica,da.analytics.prod_subs_state,PROD)"){
    urn
    aspects {
      aspectName
    }
  }
}

Guys, need some help with strange lineage behavior. More info in thread

We have lineage data at the fine-grained level and we're trying to get it out of graphQL, but we're not having any luck. Where would we go to see/search this information?

👋 Hello, team!

We are getting below two messages in UI while selecting any snowflake table/view. Validation error of type FieldUndefined: Field 'latestVersion' in type 'GetSchemaBlameResult' is undefined @ 'getSchemaBlame/latestVersion' (code undefined) The variables input contains a field name 'categories' that is not defined for input object type 'GetSchemaBlameInput' (code undefined) What needs to be done to resolve it?

Hi all. I want to make airflow integration using ‘kafka’(

DataHub Kafka Sink

). So I set up airflow connections like this.

$ airflow connections add  --conn-type 'datahub_kafka' 'datahub_rest_default' --conn-host 'broker:9092' --conn-extra '{}'

because i didn’t modify

datahub.datahub_conn_id

in

airflow.cfg

, i am using

datahub_rest_default

as connection name. When I trigger DAG, I get this error log.

[2022-09-08, 11:49:28 ] {log.py:232} WARNING - 2022-09-08, 11:49:28  WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))': /aspects?action=ingestProposal
[2022-09-08, 11:49:28 ] {connectionpool.py:810} WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))': /aspects?action=ingestProposal
[2022-09-08, 11:49:33 ] {log.py:232} WARNING - 2022-09-08, 11:49:33  WARNING Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))': /aspects?action=ingestProposal
[2022-09-08, 11:49:33 ] {connectionpool.py:810} WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))': /aspects?action=ingestProposal

I don’t know why

Connection reset by peer

happens.. Anyone who has exp on kafka sink..?

Hi, After upgrading to 0.8.43 we are facing strange issues with Users & Groups. We get following error when we try to add users to a group : Failed to group members: An unknown error occurred. The default policy 'All Users - All Platform Privileges' is already activated. We wonder if someone else has faced similar problems after upgrading to latest version ? (edited)

Hi, I am trying to deploy Datahub on k8s with dependencies as managed AWS services(RDS, Opensearch and MSK). I have a VPC Elasticsearch domain with fine-grained access enabled and an open domain policy. I use basic authentication with username and password (no funny characters that unauthorize the request).

elasticsearchSetupJob

job works fine and creates indices on ES. But

datahub-gms

cannot connect to the domain and gives the following error

ElasticsearchStatusException[method [HEAD], host [<vpc-endpoint>:443], URI [/graph_service_v1?ignore_throttled=false&ignore_unavailable=false&expand_wildcards=open%2Cclosed&allow_no_indices=false], status line [HTTP/1.1 401 Unauthorized]]; nested: ResponseException[method [HEAD], host [<vpc-endpoint>:443], URI [/graph_service_v1?ignore_throttled=false&ignore_unavailable=false&expand_wildcards=open%2Cclosed&allow_no_indices=false], status line [HTTP/1.1 401 Unauthorized]];
    at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1866)
    at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1626)
    at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1598)
    at org.elasticsearch.client.IndicesClient.exists(IndicesClient.java:974)
    at com.linkedin.metadata.search.elasticsearch.indexbuilder.ESIndexBuilder.buildIndex(ESIndexBuilder.java:51)
    at com.linkedin.metadata.graph.elastic.ElasticSearchGraphService.configure(ElasticSearchGraphService.java:238)
    at com.linkedin.metadata.kafka.hook.UpdateIndicesHook.<init>(UpdateIndicesHook.java:81)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:211)
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:117)
    at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:311)
    at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:296)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
    at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1389)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1309)
    at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887)
    at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791)
    at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:953)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
    at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:401)
    at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:292)
    at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103)
    at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073)
    at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
    at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002)
    at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746)
    at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
    at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
    at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
    at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
    at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
    at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
    at org.eclipse.jetty.server.Server.start(Server.java:423)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
    at org.eclipse.jetty.server.Server.doStart(Server.java:387)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at org.eclipse.jetty.runner.Runner.run(Runner.java:519)
    at org.eclipse.jetty.runner.Runner.main(Runner.java:564)

Some diagnostics: • Running datahub v0.8.41 • Elasticsearch v7.10 on AWS Opensearch • curling the domain endpoint from the K8s pod works fine Elasticsearch values from

values.yaml

are

elasticsearch:
      host: "<vpc-endpoint>"
      port: "443"
      useSSL: "true"
      auth:
        username: admin
        password:
          secretRef: elasticsearch-secrets
          secretKey: elasticsearch-password

issue 5406 seems to describe my issue correctly but the fix is already in the code and still the calls don’t work. Can someone help please? Thank you.

Hello Team, I get this error when ingest metadata from clickhouse 'default.exchange_rate2eur': ["Ingestion error: Orig exception: Code: 47, e.displayText() = DB:Exception Missing columns: 'comment' while processing query: 'SELECT database, name AS table_name, comment, formatRow('JSONEachRow', engine, partition_key, sorting_key, primary_key, sampling_key, storage_policy, metadata_modification_time, total_rows, total_bytes, data_paths, metadata_path) AS properties FROM system.tables WHERE name NOT LIKE '.inner%'', required columns: 'comment' 'primary_key' 'engine' 'data_paths' 'name' 'metadata_modification_time' 'metadata_path' 'partition_key' 'sampling_key' 'storage_policy' 'total_bytes' 'sorting_key' 'database' 'total_rows', maybe you meant: ['primary_key','engine','data_paths','name','metadata_modification_time','metadata_path','partition_key','sampling_key','storage_policy','total_bytes','sorting_key','database','total_rows'] (version 21.3.2.5 (official build))\n"], yml file source: config: host_port: "xxxxxxxxxxxxxxxxx" password: xxxxxxxxxxxxxxxx username: xxxxxxxxxxxxxxxxxxx type: clickhouse Hope you guys can take a look, thank you so much

@here we are facing issue. It seems like GMS service stops connecting to database after some time. This is the exception trace. Please help me if you know any pointers to troubleshoot it

10:07:34.031 [Thread-658] ERROR c.l.d.g.e.DataHubDataFetcherExceptionHandler:21 - Failed to execute DataFetcher
java.util.concurrent.CompletionException: javax.persistence.PersistenceException: java.sql.SQLNonTransientConnectionException: Could not connect to address=(host=<db>)(port=3306)(type=master) : Could not connect to <db>:3306 : PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1606)
	at java.lang.Thread.run(Thread.java:748)
Caused by: javax.persistence.PersistenceException: java.sql.SQLNonTransientConnectionException: Could not connect to address=(host=<db>)(port=3306)(type=master) : Could not connect to <db>:3306 : PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at io.ebeaninternal.server.transaction.TransactionFactoryBasic.createQueryTransaction(TransactionFactoryBasic.java:35)
	at io.ebeaninternal.server.transaction.TransactionManager.createQueryTransaction(TransactionManager.java:360)
	at io.ebeaninternal.server.core.DefaultServer.createQueryTransaction(DefaultServer.java:2306)
	at io.ebeaninternal.server.core.OrmQueryRequest.initTransIfRequired(OrmQueryRequest.java:282)
	at io.ebeaninternal.server.core.DefaultServer.findList(DefaultServer.java:1595)
	at io.ebeaninternal.server.core.DefaultServer.findList(DefaultServer.java:1574)
	at io.ebeaninternal.server.querydefn.DefaultOrmQuery.findList(DefaultOrmQuery.java:1481)
	at com.linkedin.metadata.entity.ebean.EbeanAspectDao.batchGetUnion(EbeanAspectDao.java:359)
	at com.linkedin.metadata.entity.ebean.EbeanAspectDao.batchGet(EbeanAspectDao.java:279)
	at com.linkedin.metadata.entity.ebean.EbeanAspectDao.batchGet(EbeanAspectDao.java:260)
	at com.linkedin.metadata.entity.EntityService.getEnvelopedAspects(EntityService.java:1504)
	at com.linkedin.metadata.entity.EntityService.getCorrespondingAspects(EntityService.java:353)
	at com.linkedin.metadata.entity.EntityService.getLatestEnvelopedAspects(EntityService.java:307)
	at com.linkedin.metadata.entity.EntityService.getEntitiesV2(EntityService.java:263)
	at com.linkedin.entity.client.JavaEntityClient.batchGetV2(JavaEntityClient.java:106)
	at com.linkedin.datahub.graphql.resolvers.MeResolver.lambda$get$0(MeResolver.java:55)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
	... 1 common frames omitted
Caused by: java.sql.SQLNonTransientConnectionException: Could not connect to address=(host=<db>)(port=3306)(type=master) : Could not connect to <db>:3306 : PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.createException(ExceptionFactory.java:73)
	at org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.create(ExceptionFactory.java:192)
	at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(AbstractConnectProtocol.java:1346)
	at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:634)
	at org.mariadb.jdbc.MariaDbConnection.newConnection(MariaDbConnection.java:150)
	at org.mariadb.jdbc.Driver.connect(Driver.java:89)
	at java.sql.DriverManager.getConnection(DriverManager.java:664)
	at java.sql.DriverManager.getConnection(DriverManager.java:208)
	at io.ebean.datasource.pool.ConnectionPool.createUnpooledConnection(ConnectionPool.java:529)
	at io.ebean.datasource.pool.ConnectionPool.createUnpooledConnection(ConnectionPool.java:524)
	at io.ebean.datasource.pool.ConnectionPool.createConnectionForQueue(ConnectionPool.java:766)
	at io.ebean.datasource.pool.PooledConnectionQueue._getPooledConnection(PooledConnectionQueue.java:314)
	at io.ebean.datasource.pool.PooledConnectionQueue.getPooledConnection(PooledConnectionQueue.java:270)
	at io.ebean.datasource.pool.ConnectionPool.getPooledConnection(ConnectionPool.java:817)
	at io.ebean.datasource.pool.ConnectionPool.getConnection(ConnectionPool.java:805)
	at io.ebeaninternal.server.transaction.TransactionFactoryBasic.createQueryTransaction(TransactionFactoryBasic.java:28)
	... 17 common frames omitted
Caused by: java.sql.SQLNonTransientConnectionException: Could not connect to <db>:3306 : PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.createException(ExceptionFactory.java:73)
	at org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.create(ExceptionFactory.java:183)
	at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.createConnection(AbstractConnectProtocol.java:566)
	at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(AbstractConnectProtocol.java:1341)
	... 30 common frames omitted
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at sun.security.ssl.Alert.createSSLException(Alert.java:131)
	at sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
	at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
	at sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
	at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:654)
	at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473)
	at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369)
	at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:377)
	at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
	at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:422)
	at sun.security.ssl.TransportContext.dispatch(TransportContext.java:182)
	at sun.security.ssl.SSLTransport.decode(SSLTransport.java:152)
	at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1392)
	at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1300)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:435)
	at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.sslWrapper(AbstractConnectProtocol.java:649)
	at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.createConnection(AbstractConnectProtocol.java:532)
	... 31 common frames omitted
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:456)
	at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:323)
	at sun.security.validator.Validator.validate(Validator.java:271)
	at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:315)
	at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:223)
	at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129)
	at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:638)
	... 43 common frames omitted
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
	at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
	at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
	at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:451)
	... 49 common frames omitted