To run pinot on kubernetes, do I need complete und...
# troubleshooting
a
To run pinot on kubernetes, do I need complete understanding of the kubernetes? If someone can guide me a little bit that would be great also what are the components I should be aware of while installing pinot on kubernetes
x
which cloud vendor are you using?
a
EKS AWS
x
then follow tutorials and ensure you use EBS storage for server and s3 for deep store
a
where are tutorials ?
is it on youtube?Is there any documentation present ?
@Xiang Fu
eks command might be outdated, you can just create your own eks then deploy pinot helmChart
a
ok ,I am learning kubernetes at the moment . May be in a day or two ill be able to run this. ill post the problems in case I face any.
Hi @Xiang Fu I wanted to iterate over this thread . We actually have started the EKS deployment of the pinot
x
👍
a
can anyone explain me why do I need the below Config ? Although I understand it but how and why would dedup metadata keep on increasing as my primary key would be the same. Didnt quite really get the concept here.
Copy code
{ 
 ...
  "dedupConfig": { 
        "dedupEnabled": true, 
        "hashFunction": "NONE",
        "dedupTimeColumn": "mtime",
        "metadataTTL": 30000
   }, 
 ...
}
x
are all the primary key in same kafka partition ?
a
If I am doing partition on a particular ID lets say user ID % 5 then my kafka partitions will be decided on the mod of primary key . So didnt quite really get what does all primary key in the same kafka partitio means
x
same PK should always go to same kafka partition
a
Yes it will
as the modulas will be same,
this is necessary for dedup right .
Just wanted to understand the above config why is it used or needed ?