https://pinot.apache.org/ logo
Join Slack
Powered by
Hi, How can I pass basic auth credentials to INSER...
# getting-started
t
Hi, How can I pass basic auth credentials to INSERT INTO statement. My Pinot controller has basic auth enabled. I tried below and got an error:
Copy code
INSERT INTO "orders" FROM FILE '<s3://xxxxxx/>'
OPTION(
  taskName=myTask-s3,
  input.fs.className=org.apache.pinot.plugin.filesystem.S3PinotFS,
  input.fs.prop.accessKey=xxxxxxx,
  input.fs.prop.secretKey=xxxxxxxxx,
  authToken='Basic xxxxxx=='
  input.fs.prop.region=us-east-1
)
I got bellow error message
Copy code
ProcessingException(errorCode:450, message:InternalError:
org.apache.pinot.sql.parsers.SqlCompilationException: Caught exception while parsing query: INSERT INTO "orders"
FROM FILE '<s3://xxxxxxxx>'
	at org.apache.pinot.sql.parsers.CalciteSqlParser.compileToSqlNodeAndOptions(CalciteSqlParser.java:136)
	at org.apache.pinot.controller.api.resources.PinotQueryResource.executeSqlQuery(PinotQueryResource.java:135)
	at org.apache.pinot.controller.api.resources.PinotQueryResource.handlePostSql(PinotQueryResource.java:103)
...
Caused by: org.apache.pinot.sql.parsers.SqlCompilationException: OPTION statement requires two parts separated by '='
	at org.apache.pinot.sql.parsers.CalciteSqlParser.extractOptionsMap(CalciteSqlParser.java:486)
	at org.apache.pinot.sql.parsers.CalciteSqlParser.compileToSqlNodeAndOptions(CalciteSqlParser.java:131)
	... 27 more)
removing the authToken line I get below error
Copy code
[
  {
    "message": "QueryExecutionError:\norg.apache.commons.httpclient.HttpException: Unable to get tasks states map. Error code 400, Error message: {\"code\":400,\"error\":\"No task is generated for table: orders, with task type: SegmentGenerationAndPushTask\"}\n\tat org.apache.pinot.common.minion.MinionClient.executeTask(MinionClient.java:123)\n\tat org.apache.pinot.core.query.executor.sql.SqlQueryExecutor.executeDMLStatement(SqlQueryExecutor.java:102)\n\tat org.apache.pinot.controller.api.resources.PinotQueryResource.executeSqlQuery(PinotQueryResource.java:145)\n\tat org.apache.pinot.controller.api.resources.PinotQueryResource.handlePostSql(PinotQueryResource.java:103)",
    "errorCode": 200
  }
]
✅ 1
a
do you have minions running?
t
Yes I have Minion working. The initial issue is because of s3 permission which I have fixed. The query executed successfully, how ever I get below error when checked the minion log. permission error was thrown while pushing the table to the controller.
Copy code
aught permanent exception while pushing table: orders_OFFLINE segment: orders_OFFLINE_0cd9507e-1ff5-4aef-922c-5869ad5c3312_0 to <http://xxxxx-controller:9000>, won't retry
org.apache.pinot.common.exception.HttpErrorStatusException: Got error status code: 403 (Forbidden) with reason: "Permission is denied for CREATE '/v2/segments' for table
I think the issue is that minio can not authenticate. I need a way to pass the basic auth, but this seems not to work due to the sql parser not ignoring the 
==
in the base64 ecoded credential
a
hmm, yeah, we personally set 
task.auth.token
in the minion's config file
but this seems not to work due to the sql parser not ignoring the 
==
in the base64 ecoded credential
yeah, that seems like a bug somebody who actually works on Pinot can confirm 😅 but in case it helps:
Copy code
segment.fetcher.auth.token=Basic <path:dev/pinot#auth_token>
task.auth.token=Basic <path:dev/pinot#auth_token>
we set those properties for our minions in a .conf file
t
I actually had had it already set. 😑
a
hmm, then maybe you dont need to pass it in your sql query?
t
Yes, and I get the error I shared above
a
ah, ok, im probably not being very helpful then, maybe a real contributor should chime in 🙂
t
You are, thanks for the suggestion!
Was my mistake. Had a small typo in my basic basic auth credential 🫣
a
glad you found it
m
Open in Slack
PreviousNext
Powered by