https://datahubproject.io logo
Join Slack
Powered by
Hello there, i follow the instructions to Deployin...
# troubleshoot
s
Hello there, i follow the instructions to Deploying to AWS (https://datahubproject.io/docs/deploy/aws), after login i getting a white screen and a lot Java errors on frontEnd pod
b
Hi @salmon-football-11785! It'd be great to see the logs for your datahub-frontend and datahub-gms container!
s
datahub-gms
Copy code
2022-06-09T12:35:59-03:00 15:35:59.858 [pool-10-thread-1] INFO  c.l.m.filter.RestliLoggingFilter:55 - GET /entitiesV2?ids=List(urn%3Ali%3Acorpuser%3Adatahub) - batchGet - 200 - 5ms
2022-06-09T12:35:59-03:00 15:35:59.873 [I/O dispatcher 1] WARN  org.elasticsearch.client.RestClient:65 - request [POST <http://elasticsearch-master:9200/_bulk?timeout=1m>] returned 1 warnings: [299 Elasticsearch-7.16.2-2b937c44140b6559905130a8650c64dbd0879cfb "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See <https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html> to enable security."]
2022-06-09T12:35:59-03:00 15:35:59.874 [I/O dispatcher 1] INFO  c.l.m.k.e.ElasticsearchConnector:41 - Successfully feeded bulk request. Number of events: 1 Took time ms: -1
b
Hm
Is this it?
s
datahub-frontend:
Copy code
at com.linkedin.r2.filter.transport.FilterChainClient.lambda$createWrappedClientTimingCallback$0(FilterChainClient.java:113)
	at com.linkedin.r2.filter.transport.ResponseFilter.onRestError(ResponseFilter.java:79)
	at com.linkedin.r2.filter.TimedRestFilter.onRestError(TimedRestFilter.java:92)
	at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:166)
	at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:132)
	at com.linkedin.r2.filter.FilterChainIterator.onError(FilterChainIterator.java:101)
	at com.linkedin.r2.filter.TimedNextFilter.onError(TimedNextFilter.java:48)
	at com.linkedin.r2.filter.message.rest.RestFilter.onRestError(RestFilter.java:84)
	at com.linkedin.r2.filter.TimedRestFilter.onRestError(TimedRestFilter.java:92)
	at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:166)
	at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:132)
	at com.linkedin.r2.filter.FilterChainIterator.onError(FilterChainIterator.java:101)
	at com.linkedin.r2.filter.TimedNextFilter.onError(TimedNextFilter.java:48)
	at com.linkedin.r2.filter.message.rest.RestFilter.onRestError(RestFilter.java:84)
	at com.linkedin.r2.filter.TimedRestFilter.onRestError(TimedRestFilter.java:92)
	at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:166)
	at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:132)
	at com.linkedin.r2.filter.FilterChainIterator.onError(FilterChainIterator.java:101)
	at com.linkedin.r2.filter.TimedNextFilter.onError(TimedNextFilter.java:48)
	at com.linkedin.r2.filter.message.rest.RestFilter.onRestError(RestFilter.java:84)
	at com.linkedin.r2.filter.TimedRestFilter.onRestError(TimedRestFilter.java:92)
	at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:166)
	at com.linkedin.r2.filter.FilterChainIterator$FilterChainRestIterator.doOnError(FilterChainIterator.java:132)
	at com.linkedin.r2.filter.FilterChainIterator.onError(FilterChainIterator.java:101)
	at com.linkedin.r2.filter.TimedNextFilter.onError(TimedNextFilter.java:48)
	at com.linkedin.r2.filter.transport.ClientRequestFilter.lambda$createCallback$0(ClientRequestFilter.java:102)
	at com.linkedin.r2.transport.http.common.HttpBridge$1.onResponse(HttpBridge.java:82)
	at com.linkedin.r2.transport.http.client.rest.ExecutionCallback.lambda$onResponse$0(ExecutionCallback.java:64)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 common frames omitted
Caused by: com.linkedin.r2.message.rest.RestException: Received error 404 from server for URI <http://datahub-datahub-gms:8080/dataJobs>
	at com.linkedin.r2.transport.http.common.HttpBridge$1.onResponse(HttpBridge.java:76)
	... 4 common frames omitted
datahubFrontEnd is too extensive
Caused by: com.linkedin.r2.message.rest.RestException: Received error 404 from server for URI http://datahub-datahub-gms:8080/dataJobs
is there any instructions to solve the problem ?
b
Hi can you confirm that the 
datahub-datahub-gms
pod is up and healthy?
What are the final logs for that pod? I cannot see any issues on that one
s
a lot requests like :
Copy code
2022-06-09T13:35:43-03:00 16:35:43.136 [pool-4-thread-1] WARN  org.elasticsearch.client.RestClient:65 - request [POST <http://elasticsearch-master:9200/datahubpolicyindex_v2/_search?typed_keys=true&max_concurrent_shard_requests=5&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&search_type=query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true>] returned 2 warnings: [299 Elasticsearch-7.16.2-2b937c44140b6559905130a8650c64dbd0879cfb "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See <https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html> to enable security."],[299 Elasticsearch-7.16.2-2b937c44140b6559905130a8650c64dbd0879cfb "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen tiers in place of frozen indices."]
frontEnd try to reach. :Caused by: com.linkedin.r2.message.rest.RestException: Received error 404 from server for URI http://datahub-datahub-gms:8080/dataJobs
Copy code
/ $ ping <http://datahub-datahub-gms:8080>
ping: bad address '<http://datahub-datahub-gms:8080>'
my values.yaml conf file
Copy code
datahub-frontend:
  enabled: true
  image:
    repository: linkedin/datahub-frontend-react
    tag: "latest"
  ingress:
    enabled: true
    annotations:
      <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: alb
      <http://alb.ingress.kubernetes.io/scheme|alb.ingress.kubernetes.io/scheme>: internet-facing
      <http://alb.ingress.kubernetes.io/target-type|alb.ingress.kubernetes.io/target-type>: instance
      <http://alb.ingress.kubernetes.io/certificate-arn|alb.ingress.kubernetes.io/certificate-arn>: ARN!@#!@#@!
      <http://alb.ingress.kubernetes.io/inbound-cidrs|alb.ingress.kubernetes.io/inbound-cidrs>: 0.0.0.0/0
      <http://alb.ingress.kubernetes.io/listen-ports|alb.ingress.kubernetes.io/listen-ports>: '[{"HTTP": 80}, {"HTTPS":443}]'
      <http://alb.ingress.kubernetes.io/actions.ssl-redirect|alb.ingress.kubernetes.io/actions.ssl-redirect>: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
    hosts:
      - host: #@!#@!#!@
        redirectPaths:
          - path: /*
            name: ssl-redirect
            port: use-annotation
        paths:
          - /*
datahub:
  metadata_service_authentication:
    enabled: true
Any help are welcome 🙂
b
@early-lamp-41924 Any chance you can help Rafael out?
e
Could you post the results of running the following?
Copy code
kubectl get svc -n <<namespace>>
I am suspecting either gms pod is down or gms service is not set up correctly for some reason
s
Copy code
NAME                               TYPE           CLUSTER-IP      EXTERNAL-IP                                                               PORT(S)                      AGE
datahub-acryl-datahub-actions      ClusterIP      10.100.192.69   <none>                                                                    9093/TCP                     39h
datahub-datahub-frontend           LoadBalancer   10.100.153.42   ####-<http://379995099.us-east-1.elb.amazonaws.com|379995099.us-east-1.elb.amazonaws.com>    9002:30123/TCP               39h
datahub-datahub-gms                LoadBalancer   10.100.119.44   ####-<http://1824374976.us-east-1.elb.amazonaws.com|1824374976.us-east-1.elb.amazonaws.com>   8080:31106/TCP               39h
elasticsearch-master               ClusterIP      10.100.193.15   <none>                                                                    9200/TCP,9300/TCP            39h
elasticsearch-master-headless      ClusterIP      None            <none>                                                                    9200/TCP,9300/TCP            39h
kubernetes                         ClusterIP      10.100.0.1      <none>                                                                    443/TCP                      39h
prerequisites-cp-schema-registry   ClusterIP      10.100.235.94   <none>                                                                    8081/TCP,5556/TCP            39h
prerequisites-kafka                ClusterIP      10.100.33.94    <none>                                                                    9092/TCP                     39h
prerequisites-kafka-headless       ClusterIP      None            <none>                                                                    9092/TCP,9093/TCP            39h
prerequisites-mysql                ClusterIP      10.100.142.38   <none>                                                                    3306/TCP                     39h
prerequisites-mysql-headless       ClusterIP      None            <none>                                                                    3306/TCP                     39h
prerequisites-neo4j-community      ClusterIP      None            <none>                                                                    7474/TCP,7687/TCP            39h
prerequisites-zookeeper            ClusterIP      10.100.91.171   <none>                                                                    2181/TCP,2888/TCP,3888/TCP   39h
prerequisites-zookeeper-headless   ClusterIP      None            <none>                                                                    2181/TCP,2888/TCP,3888/TCP   39h
Solved, thanks guys
b
@salmon-football-11785 Out of curiosity, how'd you get this working?
a
@salmon-football-11785 Any Change you could share what you did? i'm facing the exact problem
2 Views
Open in Slack
PreviousNext
Powered by