https://linen.dev logo
Join Slack
Powered by
This message was deleted.
# troubleshooting
s
This message was deleted.
c
parsespec/firehoses have been replaced with inputSource/inputFormat https://druid.apache.org/docs/latest/ingestion/native-batch-input-sources.html#sql-input-source
v
Hi, thanks for the reply. That is the schema we have been using for our ingestion specs
\"ioConfig\": { \"type\": \"index_parallel\", \"inputSource\": { \"type\": \"sql\", \"database\": { \"type\": \"postgresql\", \"connectorConfig\": {\"connectURI\": ").Append($"\"{_jdbcPostgresConnectionString}\", \"user\": \"{userId}\", \"password\": \"{pass}\"")
c
ah, i guess 
PostgresqlFirehoseDatabaseConnector
is still used even by the input source, my bad, just latched onto firehose and assumed was using the older specs
it seems like some issue with classpath stuff since it cant find the postgres driver, but not sure off the top of my head what the issue would be
v
Ok, thanks for the hint 👍
Hey Clint, providing an update on this: we started downgrading from the most recent release and eventually downgraded back down to the original 0.21.0 version. The exception isn't being thrown on the original version we were running
Were there any subsequent releases that changed how the extension or the drivers are read/where they're read from>
Its the 0.22.0 release where the exception starts getting thrown
c
interesting, i’m not sure off the top of my head what changes might have caused that, as soon as I have some free time i’ll try to have a look at PRs between now and then for postgres. it is also probably interesting to know if there are any jar differences between the extension dir and also the lib/ dir to see if the postgres jar moved or something
👍 1
v
This is what my teammate said: Maybe you can reply Clint -- to show him the command line for the indexer:
In the classpath (-cp), it only mentions lib folder; but there is no postgresql*.jar in lib; the postgresql*.jar lives under this subfolder inside extensions:
Hi Clint, did you get a chance to take a look at this? Our analytics pipeline is kind of stuck until we can figure this out.
c
sorry for delay, got busy and hadn't had a chance yet, but am having a look now
the extensions don't need to be explicitly on classpath, by default druid looks in extensions path and loads everything in 
druid.extensions.loadlist
looking at binary packaging between 0.20.0 and 25.0.0, I see the postgres driver jar in same place for both versions, slightly newer but I wouldn't expect that to make a difference
no extras copies in lib path or anything
I don't see any real postgres changes other than version bumps, but maybe some other change is impacting, still looking through change sets
also will try to repro
i suspect copying the postgresql jar from the extension to the 
lib/
path would make things work with Druid 25.0.0, but haven’t determined which changes would have caused this problem to start happening between versions yet
v
Hi Clint, sorry for the delayed response. I was on PTO when you sent this and wasn't feeling well yesterday. I forwarded this info to my teammates when you sent it so we'll see what they say
Open in Slack
PreviousNext
Powered by