This message was deleted.
# troubleshooting
s
This message was deleted.
g
try adding
--no-default-remote-repositories
too
without that, your provided repo is only used for stuff that isn't found in the default repos
note; similar thread here, i wonder if you are running into the same thing? https://apachedruidworkspace.slack.com/archives/C0303FDCZEZ/p1663739533640179
a
Hi, @Gian Merlino in the logs first it's going to the repo that we mention with
-r
but even after downloading from that specified repo it again tries to download from repo.maven that is where we're getting this SocketTimeout because it's not allowed with the corporate firewall.But the thing is that if we've specified the remote repo then it should not go to that maven repo.Is there some issue with the druid ?
g
I don't see a problem in the code about this, although I haven't tried this exact setup myself
I don't see any references in our code to repo.maven.apache.org though, which is why i was wondering if your repo was somehow delegating to that one for objects it didn't have locally
If you specify no default remote repositories, and don't provide your own remote (so it'll be local only) -- do you still see it trying to contact repo.maven.apache.org?
a
yes even after providing`--no-default-remote-repositories -r "repo.xyz"` it is first going to this repo which is specified with this
-r
option and then even after downloading from this remote repo it's again going to that maven repo and also the call to that maven artifact is happening with
http
call.
g
I meant if you provide
--no-default-remote-repositories
and no
-r
— in that case it should be purely local resolution
I'm wondering if that still tries to make network calls for some reason
a
java  -Dlog4j2.level=DEBUG \
-cp "lib/*" \
-Ddruid.extensions.directory="extensions" \
-Ddruid.extensions.hadoopDependenciesDir="hadoop-dependencies" \
org.apache.druid.cli.Main tools pull-deps \
-c "org.apache.druid.extensions:druid-hdfs-storage:0.23.0" \
-h "org.apache.hadoop:hadoop-client:2.8.5" \
-h "org.apache.hadoop:hadoop-client:3.3.3"
tried this command attaching the logs below
Downloading artifact org.apache.httpcomponentshttpclientpom:4.5.13 from (https://repo1.maven.org/maven2/, releases+snapshots) 235311.199 [main] DEBUG org.eclipse.aether.internal.impl.DefaultRepositoryConnectorProvider - Using connector AetherRepositoryConnector with priority 3.4028235E38 for https://repo1.maven.org/maven2/ Downloading: https://repo1.maven.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom Downloaded: https://repo1.maven.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom (7 KB at 6.8 KB/sec) Downloaded artifact org.apache.httpcomponentshttpclientpom:4.5.13 from (https://repo1.maven.org/maven2/, releases+snapshots) Resolved artifact org.apache.httpcomponentshttpclientpom:4.5.13 from (https://repo1.maven.org/maven2/, releases+snapshots) Downloading artifact org.apache.httpcomponentshttpclientpom:4.5.13 from apache.snapshots (https://repository.apache.org/snapshots, releases+snapshots) 235312.149 [main] DEBUG org.eclipse.aether.internal.impl.DefaultRepositoryConnectorProvider - Using connector AetherRepositoryConnector with priority 3.4028235E38 for https://repository.apache.org/snapshots Downloading: https://repository.apache.org/snapshots/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom io.tesla.aether.connector.ResourceDoesNotExistException: Unable to locate resource https://repository.apache.org/snapshots/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom. Error code 404
from the above logs we can see that even after once downloading it again goes to multiple repos and it should not happen since it has already downloaded that particular file
Downloading: https://repo1.maven.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom Downloaded: https://repo1.maven.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom (7 KB at 6.8 KB/sec) Downloaded artifact org.apache.httpcomponentshttpclientpom:4.5.13 from (https://repo1.maven.org/maven2/, releases+snapshots) Resolved artifact org.apache.httpcomponentshttpclientpom:4.5.13 from (https://repo1.maven.org/maven2/, releases+snapshots) Downloading artifact org.apache.httpcomponentshttpclientpom:4.5.13 from apache.snapshots (https://repository.apache.org/snapshots, releases+snapshots) 235312.149 [main] DEBUG org.eclipse.aether.internal.impl.DefaultRepositoryConnectorProvider - Using connector AetherRepositoryConnector with priority 3.4028235E38 for https://repository.apache.org/snapshots Downloading: https://repository.apache.org/snapshots/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom io.tesla.aether.connector.ResourceDoesNotExistException: Unable to locate resource https://repository.apache.org/snapshots/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom. Error code 404 at io.tesla.aether.connector.AetherRepositoryConnector.handleResponseCode(AetherRepositoryConnector.java:298) at io.tesla.aether.connector.AetherRepositoryConnector.access$900(AetherRepositoryConnector.java:85) at io.tesla.aether.connector.AetherRepositoryConnector$GetTask.resumableGet(AetherRepositoryConnector.java:549) at io.tesla.aether.connector.AetherRepositoryConnector$GetTask.run(AetherRepositoryConnector.java:391) at io.tesla.aether.connector.AetherRepositoryConnector.get(AetherRepositoryConnector.java:232) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:535) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:436) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:262) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:239) at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:320) at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:217) at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:461) at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573) at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573) at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:261) at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:342) at io.tesla.aether.internal.DefaultTeslaAether.resolveArtifacts(DefaultTeslaAether.java:289) at org.apache.druid.cli.PullDependencies.downloadExtension(PullDependencies.java:407) at org.apache.druid.cli.PullDependencies.downloadExtension(PullDependencies.java:365) at org.apache.druid.cli.PullDependencies.run(PullDependencies.java:311) at org.apache.druid.cli.Main.main(Main.java:114) Downloaded artifact org.apache.httpcomponentshttpclientpom:4.5.13 from apache.snapshots (https://repository.apache.org/snapshots, releases+snapshots) Downloading artifact org.apache.httpcomponentshttpclientpom:4.5.13 from sigar (https://repository.mulesoft.org/nexus/content/repositories/public, releases+snapshots) 235317.594 [main] DEBUG org.eclipse.aether.internal.impl.DefaultRepositoryConnectorProvider - Using connector AetherRepositoryConnector with priority 3.4028235E38 for https://repository.mulesoft.org/nexus/content/repositories/public Downloading: https://repository.mulesoft.org/nexus/content/repositories/public/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom Downloaded: https://repository.mulesoft.org/nexus/content/repositories/public/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom (7 KB at 4.2 KB/sec) Downloaded artifact org.apache.httpcomponentshttpclientpom:4.5.13 from sigar (https://repository.mulesoft.org/nexus/content/repositories/public, releases+snapshots) Resolved artifact org.apache.httpcomponentshttpclientpom:4.5.13 from sigar (https://repository.mulesoft.org/nexus/content/repositories/public, releases+snapshots) Downloading artifact org.apache.httpcomponentshttpclientpom:4.5.13 from central (http://repo.maven.apache.org/maven2, releases) 235319.149 [main] DEBUG org.eclipse.aether.internal.impl.DefaultRepositoryConnectorProvider - Using connector AetherRepositoryConnector with priority 3.4028235E38 for http://repo.maven.apache.org/maven2
Downloading: <http://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom>
here in the last line it's making http call, even after downloading
g
what if you do that same command but add
--no-default-remote-repositories
?
a
Added this
--no-default-remote-repositories
it's failing, if we provide --no-default-remote-repositories than it's mandatory to provide the remote repo...Attaching the logs below
012650.983 [main] INFO org.apache.druid.cli.PullDependencies - Start downloading dependencies for extension coordinates: [[org.apache.druid.extensionsdruid hdfs storage0.23.0]] 012650.985 [main] INFO org.apache.druid.cli.PullDependencies - Directory [/home/ashish/apache-druid-0.23.0/extensions/druid-hdfs-storage] already exists, skipping creating a directory 012650.989 [main] INFO org.apache.druid.cli.PullDependencies - Start downloading extension [org.apache.druid.extensionsdruid hdfs storagejar:0.23.0] Resolving artifact org.apache.druid.extensionsdruid hdfs storagepom:0.23.0 Resolved artifact org.apache.druid.extensionsdruid hdfs storagepom:0.23.0 from null Missing artifact descriptor for org.apache.druid.extensionsdruid hdfs storagejar:0.23.0 012651.008 [main] DEBUG org.eclipse.aether.internal.impl.DefaultDependencyCollector - Dependency collection stats: {ConflictMarker.analyzeTime=1, ConflictMarker.markTime=0, ConflictMarker.nodeCount=1, ConflictIdSorter.graphTime=0, ConflictIdSorter.topsortTime=0, ConflictIdSorter.conflictIdCount=1, ConflictIdSorter.conflictIdCycleCount=0, ConflictResolver.totalTime=4, ConflictResolver.conflictItemCount=1, DefaultDependencyCollector.collectTime=10, DefaultDependencyCollector.transformTime=6} Resolving artifact org.apache.druid.extensionsdruid hdfs storagejar:0.23.0 Resolved artifact org.apache.druid.extensionsdruid hdfs storagejar:0.23.0 from null 012651.012 [main] ERROR org.apache.druid.cli.PullDependencies - Unable to resolve artifacts for [org.apache.druid.extensionsdruid hdfs storagejar:0.23.0 (runtime) -> [] < []]. org.eclipse.aether.resolution.DependencyResolutionException: Could not find artifact org.apache.druid.extensionsdruid hdfs storagejar:0.23.0 at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:384) ~[aether-impl-0.9.0.M2.jar:?] at io.tesla.aether.internal.DefaultTeslaAether.resolveArtifacts(DefaultTeslaAether.java:289) ~[tesla-aether-0.0.5.jar:0.0.5] at org.apache.druid.cli.PullDependencies.downloadExtension(PullDependencies.java:407) [druid-services-0.23.0.jar:0.23.0] at org.apache.druid.cli.PullDependencies.downloadExtension(PullDependencies.java:365) [druid-services-0.23.0.jar:0.23.0] at org.apache.druid.cli.PullDependencies.run(PullDependencies.java:311) [druid-services-0.23.0.jar:0.23.0] at org.apache.druid.cli.Main.main(Main.java:114) [druid-services-0.23.0.jar:0.23.0] Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not find artifact org.apache.druid.extensionsdruid hdfs storagejar:0.23.0 at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:459) ~[aether-impl-0.9.0.M2.jar:?] at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:262) ~[aether-impl-0.9.0.M2.jar:?] at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:367) ~[aether-impl-0.9.0.M2.jar:?] ... 5 more Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact org.apache.druid.extensionsdruid hdfs storagejar:0.23.0 at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:449) ~[aether-impl-0.9.0.M2.jar:?] at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:262) ~[aether-impl-0.9.0.M2.jar:?] at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:367) ~[aether-impl-0.9.0.M2.jar:?] ... 5 more Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: org.eclipse.aether.resolution.DependencyResolutionException: Could not find artifact org.apache.druid.extensionsdruid hdfs storagejar:0.23.0 at org.apache.druid.cli.PullDependencies.run(PullDependencies.java:335) at org.apache.druid.cli.Main.main(Main.java:114) Caused by: java.lang.RuntimeException: org.eclipse.aether.resolution.DependencyResolutionException: Could not find artifact org.apache.druid.extensionsdruid hdfs storagejar:0.23.0 at org.apache.druid.cli.PullDependencies.downloadExtension(PullDependencies.java:420) at org.apache.druid.cli.PullDependencies.downloadExtension(PullDependencies.java:365) at org.apache.druid.cli.PullDependencies.run(PullDependencies.java:311) ... 1 more Caused by: org.eclipse.aether.resolution.DependencyResolutionException: Could not find artifact org.apache.druid.extensionsdruid hdfs storagejar:0.23.0 at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:384) at io.tesla.aether.internal.DefaultTeslaAether.resolveArtifacts(DefaultTeslaAether.java:289) at org.apache.druid.cli.PullDependencies.downloadExtension(PullDependencies.java:407) ... 3 more Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not find artifact org.apache.druid.extensionsdruid hdfs storagejar:0.23.0 at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:459) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:262) at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:367) ... 5 more Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact org.apache.druid.extensionsdruid hdfs storagejar:0.23.0 at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:449) ... 7 more
h
Seems be a issue from druid pull-deps making use of the AetherRepositoryConnector
https://github.com/apache/druid/issues/383 : Seems to be still OPEN and cause for this
g
Wow I forgot all about that!
Couple things brought to mind First: I wonder if it's happening because some deps are missing from your organization's maven repo? perhaps you can stop it from happening by adding the deps Second: I wonder if there's some way to get aether to stop doing this? the comment on the issue suggests there is. although, if certain deps really are missing from your repo, that would make dependency resolution fail anyway (with "artifact not found" instead of "timeout")