https://pinot.apache.org/ logo
#general
Title
# general
r

Ryan Clark

07/09/2021, 4:44 PM
I am trying to plug in an AWS Kinesis data stream, but I am unable to get authenticated because we use temporary credentials, which requires an AWS_SESSION_TOKEN to be given. Is there a way to give a session token along with the key and secret? Here is part of my table config.
Copy code
"streamConfigs": {
          "streamType": "kinesis",
          "stream.kinesis.topic.name": "stream-name",
          "region": "us-east-1",
          "accessKey": "sdfsdf",
          "secretKey": "sdfdsf",
m

Mayank

07/09/2021, 4:44 PM
@User
🙌 1
n

Neha Pawar

07/09/2021, 4:52 PM
will this AWS_SESSION_TOKEN keep changing?
if yes, setting this in the table config might not be the solution. if no, we can look into submitting a fix to incorporate this
@User do you think you can investigate how to handle this request?
k

Kartik Khare

07/09/2021, 4:57 PM
Yes, looking into it
@User since we use AWS SDKs default credentials provider you can simply export token as environment variable and it should work. Also it is recommended that accessKey and secretKey be provided as a part of credentials file or environment variables
n

Neha Pawar

07/09/2021, 5:11 PM
so set
AWS_SESSION_TOKEN
this environment variable and remove fields
accessKey
and
secretKey
from tableConfig? Would you please add this to the docs on the various ways to access https://docs.pinot.apache.org/basics/data-import/pinot-stream-ingestion/amazon-kinesis @User
k

Kartik Khare

07/09/2021, 5:18 PM
added.
thankyou 1
r

Ryan Clark

07/09/2021, 7:00 PM
Thank you! I have given my keys to the container and am getting this error now:
Copy code
"shaded.software.amazon.awssdk.core.exception.SdkClientException: Unable to load an HTTP implementation from any provider in the chain. You must declare a dependency on an appropriate HTTP implementation or pass in an SdkHttpClient explicitly to the client builder."
Given that this deployment is on EKS, I'm confused why it wouldn't automatically have credentials.
x

Xiang Fu

07/09/2021, 8:00 PM
I think this is due the shaded jar issue that amazon-kisnesis package doesn’t load the default HTTP implementation
this is the plugin issue
k

Kartik Khare

07/09/2021, 8:01 PM
Yes, seems like that. Without shading though, it might cause issues with maven enforcer.