https://datahubproject.io logo
Join Slack
Powered by
Any idea why the update indices job would fail lik...
# troubleshoot
r
Any idea why the update indices job would fail like this?
e
Hey Alex. How are you running this job? Is this docker or kubernetes? What is the image tag of the upgrade container being run?
r
Kubernetes, this is post last helm chart upgrade to the 0.8.36 version updated via 
helm upgrade -f oidcConfig.yml datahub datahub/datahub
The state for the frontend pod is here:
Copy code
containerStatuses:
  - containerID: <containerd://4b4e1a9f67d2286383a6dc24eca111e358b97b9a449240490ff9ee7a9e659c0>d
    image: <http://docker.io/linkedin/datahub-frontend-react:v0.8.36|docker.io/linkedin/datahub-frontend-react:v0.8.36>
    imageID: <http://docker.io/linkedin/datahub-frontend-react@sha256:29a5088d961fad8f994b3b7d77c016ec89dca6616582ea8259bb80f32d0b5e9e|docker.io/linkedin/datahub-frontend-react@sha256:29a5088d961fad8f994b3b7d77c016ec89dca6616582ea8259bb80f32d0b5e9e>
    lastState: {}
    name: datahub-frontend
    ready: true
    restartCount: 0
    started: true
    state:
      running:
        startedAt: "2022-06-03T17:30:57Z"
I verified those pods are at the correct img version
Here is the cronjob spec
Copy code
apiVersion: batch/v1
kind: CronJob
metadata:
  annotations:
    <http://meta.helm.sh/release-name|meta.helm.sh/release-name>: datahub
    <http://meta.helm.sh/release-namespace|meta.helm.sh/release-namespace>: default
  creationTimestamp: "2022-06-02T00:05:02Z"
  generation: 1
  labels:
    <http://app.kubernetes.io/instance|app.kubernetes.io/instance>: datahub
    <http://app.kubernetes.io/managed-by|app.kubernetes.io/managed-by>: Helm
    <http://app.kubernetes.io/version|app.kubernetes.io/version>: 0.8.36
    <http://helm.sh/chart|helm.sh/chart>: datahub-0.2.79
  name: datahub-datahub-restore-indices-job-template
  namespace: default
  resourceVersion: "1074427"
  uid: 13fca1da-0d7c-4cd7-9563-a9d96303ef6e
spec:
  concurrencyPolicy: Allow
  failedJobsHistoryLimit: 1
  jobTemplate:
    metadata:
      creationTimestamp: null
    spec:
      template:
        metadata:
          creationTimestamp: null
        spec:
          containers:
          - args:
            - -u
            - RestoreIndices
            env:
            - name: ENTITY_REGISTRY_CONFIG_PATH
              value: /datahub/datahub-gms/resources/entity-registry.yml
            - name: DATAHUB_GMS_HOST
              value: datahub-datahub-gms
            - name: DATAHUB_GMS_PORT
              value: "8080"
            - name: DATAHUB_MAE_CONSUMER_HOST
              value: datahub-datahub-mae-consumer
            - name: DATAHUB_MAE_CONSUMER_PORT
              value: "9091"
            - name: EBEAN_DATASOURCE_USERNAME
              value: root
            - name: EBEAN_DATASOURCE_PASSWORD
              valueFrom:
                secretKeyRef:
                  key: mysql-root-password
                  name: mysql-secrets
            - name: EBEAN_DATASOURCE_HOST
              value: prerequisites-mysql:3306
            - name: EBEAN_DATASOURCE_URL
              value: jdbc:<mysql://prerequisites-mysql:3306/datahub?verifyServerCertificate=false&useSSL=true&useUnicode=yes&characterEncoding=UTF-8&enabledTLSProtocols=TLSv1.2>
            - name: EBEAN_DATASOURCE_DRIVER
              value: com.mysql.cj.jdbc.Driver
            - name: KAFKA_BOOTSTRAP_SERVER
              value: prerequisites-kafka:9092
            - name: KAFKA_SCHEMAREGISTRY_URL
              value: <http://prerequisites-cp-schema-registry:8081>
            - name: ELASTICSEARCH_HOST
              value: elasticsearch-master
            - name: ELASTICSEARCH_PORT
              value: "9200"
            - name: GRAPH_SERVICE_IMPL
              value: neo4j
            - name: NEO4J_HOST
              value: prerequisites-neo4j-community:7474
            - name: NEO4J_URI
              value: <bolt://prerequisites-neo4j-community>
            - name: NEO4J_USERNAME
              value: neo4j
            - name: NEO4J_PASSWORD
              valueFrom:
                secretKeyRef:
                  key: neo4j-password
                  name: neo4j-secrets
            image: acryldata/datahub-upgrade:v0.8.36
            imagePullPolicy: IfNotPresent
            name: datahub-upgrade-job
            resources:
              limits:
                cpu: 500m
                memory: 512Mi
              requests:
                cpu: 300m
                memory: 256Mi
            securityContext: {}
            terminationMessagePath: /dev/termination-log
            terminationMessagePolicy: File
          dnsPolicy: ClusterFirst
          restartPolicy: Never
          schedulerName: default-scheduler
          securityContext: {}
          terminationGracePeriodSeconds: 30
  schedule: '* * * * *'
  successfulJobsHistoryLimit: 3
  suspend: true
status: {}
Image is 
image: acryldata/datahub-upgrade:v0.8.36
5 Views
Open in Slack
PreviousNext
Powered by