Hello, I am Doron Podoleanu VP R&D at Velotix. We started using datahub but since it has vulnerable packing: Scanned by GCR and Snyk. This de facto prevents us from onboarding into google marketplace (and will certainly be an issue for any other audited production).

"Packages below are known to contain vulnerabilities. Please update the affected packages and resubmit the solution
Image: elasticsearch:7.17.9
  Note: CVE-2022-1471
  Package: org.yaml:snakeyaml
  Package Type: MAVEN
  Affected Version: 1.33
  Fixed Version: 2.0
Image: acryldata/datahub-actions:v0.0.12
  Note: CVE-2023-24538
  Package: go
  Package Type: GO_STDLIB
  Affected Version: 1.20.2
  Fixed Version: 1.20.3
  Note: CVE-2021-33036
  Package: org.apache.hadoop:hadoop-yarn-server-common
  Package Type: MAVEN
  Affected Version: 3.2.0
  Fixed Version: 3.2.3
  Note: CVE-2022-37865
  Package: org.apache.ivy:ivy
  Package Type: MAVEN
  Affected Version: 2.4.0
  Fixed Version: 2.5.1
  Note: CVE-2022-25168
  Package: org.apache.hadoop:hadoop-common
  Package Type: MAVEN
  Affected Version: 3.2.0
  Fixed Version: 3.2.4
  Note: CVE-2023-22946
  Package: org.apache.spark:spark-core_2.12
  Package Type: MAVEN
  Affected Version: 3.0.3
  Fixed Version: 3.4.0
  Note: CVE-2022-26612
  Package: org.apache.hadoop:hadoop-common
  Package Type: MAVEN
  Affected Version: 3.2.0
  Fixed Version: 3.2.3
  Note: CVE-2019-0204
  Package: org.apache.mesos:mesos
  Package Type: MAVEN
  Affected Version: 1.4.0
  Fixed Version: 1.4.3
Image: acryldata/datahub-kafka-setup:v0.10.2.2
  Note: CVE-2022-1471
  Package: org.yaml:snakeyaml
  Package Type: MAVEN
  Affected Version: 1.32
  Fixed Version: 2.0"

1 - I understand that it uses ES 7 and does not really run the vulnerability execution path with snakeYaml etc. It matters not. this is severe CVE and this software is going to be stopped out of any audited place (such as marketplace, etc.) 2 - Is there commitment to fix anything in that list ( I know about best efforts) 3 - Trying to run datahub without ES right now. We do not really use search. When Trying to build I get the following error:

2023-05-08T11:01:24.654+0300 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Completing Build operation 'Configure build'
2023-05-08T11:01:24.654+0300 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Configure build' completed
2023-05-08T11:01:24.660+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
2023-05-08T11:01:24.662+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] FAILURE: Build failed with an exception.
2023-05-08T11:01:24.663+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
2023-05-08T11:01:24.663+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * Where:
2023-05-08T11:01:24.663+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] Build file '/Users/podoleanu/work/datahub/buildSrc/build.gradle' line: 8
2023-05-08T11:01:24.663+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
2023-05-08T11:01:24.663+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * What went wrong:
2023-05-08T11:01:24.663+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] A problem occurred evaluating project ':buildSrc'.
2023-05-08T11:01:24.663+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] > Could not find method compile() for arguments [io.acryl:json-schema-avro:0.1.5, build_72c16s6ya15s0l3jdky658gr3$_run_closure1$_closure2@7c1b50a8] on object of type org.gradle.api.internal.artifacts.dsl.dependencies.DefaultDependencyHandler.
2023-05-08T11:01:24.664+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
2023-05-08T11:01:24.664+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * Exception is:
2023-05-08T11:01:24.664+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] org.gradle.api.GradleScriptException: A problem occurred evaluating project ':buildSrc'.
2023-05-08T11:01:24.664+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   at org.gradle.groovy.scripts.internal.DefaultScriptRunnerFactory$ScriptRunnerImpl.run(DefaultScriptRunnerFactory.java:93)
2023-05-08T11:01:24.664+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   at org.gradle.configuration.DefaultScriptPluginFactory$ScriptPluginImpl.lambda$apply$0(DefaultScriptPluginFactory.java:135)
2023-05-08T11:01:24.669+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   at org.gradle.configuration.ProjectScriptTarget.addConfiguration(ProjectScriptTarget.java:79)
2023-05-08T11:01:24.669+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   at org.gradle.configuration.DefaultScriptPluginFactory$ScriptPluginImpl.apply(DefaultScriptPluginFactory.java:138)
2023-05-08T11:01:24.670+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   at org.gradle.configuration.BuildOperationScriptPlugin$1.run(BuildOperationScriptPlugin.java:65)
2023-05-08T11:01:24.670+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
2023-05-08T11:01:24.670+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
2023-05-08T11:01:24.670+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
2023-05-08T11:01:24.670+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
2023-05-08T11:01:24.670+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
2023-05-08T11:01:24.670+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
2023-05-08T11:01:24.670+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   at org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:47)
2023-05-08T11:01:24.670+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:68)
2023-05-08T11:01:24.670+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   at org.gradle.configuration.BuildOperationScriptPlugin.lambda$apply$0(BuildOperationScriptPlugin.java:62)
2023-05-08T11:01:24.670+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   at org.gradle.configuration.internal.DefaultUserCodeApplicationContext.apply(DefaultUserCodeApplicationContext.java:44)
2023-05-08T11:01:24.670+0300 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   at org.gradle.configuration.BuildOperationScriptPlugin.apply(BuildOperationScriptPlugin.java:62)

Can I get help/answers please? Thanks!

Hey there 👋 I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: 1️⃣ There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? Yes button 2️⃣ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues? Yes button

The first issue seems to be Elastic issue, the image was released by Elastic. You have a couple of option: • Try different elastic images (i think even version 8 is having this issue) • Don't use the elastic image, use your own elastic deployment i.e. cloud provider etc and point the gms to that deployment.

ES 7->8 is not lift and shift as far as I know. Any reference to this being done previously ? About the second point I will check.

Correct es 7->8 is not a lift and shift. I am not sure why ES team has not addressed the issue. There is not much information in their forum either because they said they dont want to discuss security vul in forum. The kafka setup image is just a once of run. It is not a long running process so i suppose it is not a big deal. I am not sure about the other vulnerabilities in the action image. That one would be required if you run ingestion through UI. If you do your own ingestion through CLI, you dont need it