https://pinot.apache.org/ logo
Join Slack
Powered by
hellos, We tried exploring <pinot-spark-connector>...
# pinot-dev
a
hellos, We tried exploring pinot-spark-connector but it does not support https protocol as mentioned here. Can I add a github issue around this and possibly pick up implementation for it, if its not something that is there in roadmap
x
Please create the GitHub issue and feel free to pick it up!
The change for supporting https should be fairly simple
a
Hi @Xiang Fu: Created an issue here. Do you recommend I create a new HttpsClient or reuse this one with a knob to toggle between HTTP/HTTPs ?
x
Suggest to reuse it and add extra signature to the methods.
a
Sounds good. Thanks !
x
you can keep default behavior for http for backward compatibility
something like
Copy code
private val TABLE_SCHEMA_TEMPLATE = "%s://%s/tables/%s/schema"

def getTableSchema(controllerUrl: String, tableName: String, useHttps boolean): Schema = {
    val rawTableName = TableNameBuilder.extractRawTableName(tableName)
    Try {
      val uri = new URI(String.format(TABLE_SCHEMA_TEMPLATE, useHttps ? "https" : "http", controllerUrl, rawTableName))
      val response = HttpUtils.sendGetRequest(uri)
      Schema.fromString(response)
    } match {
      case Success(response) =>
        logDebug(s"Pinot schema received successfully for table '$rawTableName'")
        response
      case Failure(exception) =>
        throw PinotException(
          s"An error occurred while getting Pinot schema for table '$rawTableName'",
          exception
        )
    }
  }

def getTableSchema(controllerUrl: String, tableName: String): Schema = {
    getTableSchema(controllerUrl, tableName, false)
  }
I’m not familiar with scala syntax, treat it as pseudo code
a
Yep, only caveat is i have to carry that flag everywhere in 
HttpUtils.scala
x
true.
another way is to parse 
controllerUrl
and inspect 
scheme
, if it’s null, use 
http
so you don’t need to add a new method
a
Yep thats another option. Will look at both and check which one is simpler to implement and understand.
@Xiang Fu: I am trying to run with https on our pinot setup and for some reason, this broker API below returns 404. Do you think they are available only on certain pinot version. 
/debug/routingTable/{tableName}
Curious why does it need to call debug API's for this
For Example : If I do not provide LIMIT the api returns 404, but if I add a LIMIT it works.
x
cause routingTable is only used for debugging purpose
pinot internally don’t use this api
we only expose this api for presto and spark
a
Gotcha. Thanks
@Xiang Fu: Any pointers on how/where I can start modifying the QuickStart example to run a 
https
pinot server instead of 
http
in order to test my changes.
x
you need to start the controller by setting SslContext
a
Is it this doc ?
x
yes
👍 1
you can modify the quickstart to always start both https and http
a
Gotcha, thanks
Another follow-up. I should make those changes here, build the project to use updated configs and then run 
Quickstart
. Is that order correct or am I missing something ?
x
You can just update the config for QuickStart
No need to make config file
a
If possible once you get a chance, can you point me to which config file location in git ?
x
maybe you can try to make AuthQuickstart a https based by editing the config override
a
Sounds good
Open in Slack
PreviousNext
Powered by