https://pinot.apache.org/ logo
Join Slack
Powered by
mind pushing a snapshot here? I can use docker to ...
# general
a
mind pushing a snapshot here? I can use docker to get latest, but I want to play with using dependencies as flat classpath is so much faster to boot https://repository.apache.org/content/repositories/snapshots/org/apache/pinot/ I can use jitpack but it is awkward and doesn't work to get dep tree due to this: "<version>${revision}${sha1}</version>"
basically I can only use jitpack to get jars as the poms are useless as they can't resolve versions of the parent
for the curious on jitpack, this works, but there is massive amount of shade duplication which can be avoided with normal deps
Copy code
RUN mkdir classes && cd classes && \
  for ARTIFACT in pinot-distribution pinot-confluent-avro pinot-kafka-2.0; do \
  JAR=/tmp/${ARTIFACT}.jar && \
  curl -sSL <https://jitpack.io/com/github/${USER}/incubator-pinot/${ARTIFACT}/${TAG}/${ARTIFACT}-${TAG}-shaded.jar> > ${JAR} && \
  $JAVA_HOME/bin/jar -xf $JAR && rm $JAR; done
x
I think @Jack is working on rollback pom version
🙏 1
a
meanwhile if you have time or know someone who does to publish snapshot it would be handy
x
ok, one concern here is that typical SNAPSHOT dependencies are not allowed to be used pom
a
sorry can you rephrase, I didn't quite understand
to tell you what I'm doing.. I'm trying to design something that uses pom, but it also needs latest code. I want to put it together in a way so that it will work in 0.5.0 but without making my own snapshot repo. I wouldn't have to do this if jitpack worked, but that version thing is in the way. otherwise it is a bit messy to help fix startup slowness
since I see snapshot repo has been published before it would seem a way to unblock unless publishing there also re-creates the pom version problem. then agreed it is useless to deploy one
maybe if this is difficult, push a docker snapshot tag?
x
Current docker image has a tag with commit id and date
Snapshot jars are usually for testing
Most software release will have plugins/checks to prevent dependencies with SNAPSHOT as the software behavior is unpredictable
One way of doing that is to have your internal artifactory
Then you can do local release by your own tagging
E.g. 0.5.0.1, 0.5.0.2
And keep only using versions/tags
a
I understand how snapshots work though appreciate you being thorough. I was hoping you can make it easier for me to help championing you by either cutting another docker tag (last week is recent one) or republishing snapshot (some months back was most recent). if you aren't able to help this way it is ok. basically hypertrace users will see slower startup from pinot until this happens. I mainly want to help your reputation and if you now decide no more snapshot that is ok but I wont setup a repo over this. your call.
x
For docker, I can try to add a github action to build and publish docker image per master commit. Or I can change my jenkins job to daily basis (Currently we have a scheduled weekly docker build every Friday afternoon).
a
weekly is ok for docker, I didn't know the schedule. I opened an issue otherwise on the maven thing as jitpack is better than snapshot anyway thanks @Xiang Fu
x
i think once that pom version thing is fixed, then it should be easier to do so
j
I opened a PR to fix the pom versions: https://github.com/apache/incubator-pinot/pull/5945 It’d be great if you guys can review it. Thanks!
a
thanks @Jack will try later today
Open in Slack
PreviousNext
Powered by