Apache Pinot #troubleshooting

Join Slack

Ravi Singal

04/16/2020, 5:41 PM

what is the frequency of realtime segment fixer?

Xiang Fu

04/28/2020, 11:26 PM

so the #entries scanned is #docs * 2

Damiano

04/29/2020, 11:59 AM

is it possible to activate the debug or log somehow? or maybe run pinot with a verbose mode to see output of my custom code?

Kishore G

04/30/2020, 5:31 AM

@Neha Pawar do we support tz in SDF format

Damiano

04/30/2020, 7:57 AM

hello everyone

Dan Hill

05/01/2020, 2:08 AM

@Xiang Fu - I get the following error. Are there other options I need to enable these? I'm using presto-server-0.234.2 (the latest from the Presto download site).

Copy code

1) Configuration property 'pinot.infer-date-type-in-schema' was not used
2) Configuration property 'pinot.infer-timestamp-type-in-schema' was not used

Xiang Fu

05/01/2020, 6:19 AM

you can put options like

--memory="1g" --cpus=".5"

docker run

command

Damiano

05/03/2020, 4:04 PM

Hello, as you know i am writing a custom aggregator, i am working on aggregateGroupBySV method. could anyone explain if i must need the serialized part inside that method? i always see the IF statement that checks DataType.BYTES is it mandatory? Thanks

Subbu Subramaniam

05/05/2020, 1:43 AM

@Damiano Kishore was mentioning that you are facing some performance issues. Is that still the case? Or is that prolblem resoplved? I tried to look through this channel, but I could not find anything right away.

Neha Pawar

05/05/2020, 5:22 PM

Good to know @Ranveer Singh. Another problem is the datatype for your DateTimeFieldSpecs. It should be STRING. I was able to generate sample data - the DateTimeFieldSpecs show up correctly

Ranveer Singh

05/05/2020, 5:27 PM

@Neha Pawar, do you suggest superset for dashboard on Pinot

Jack

05/06/2020, 5:32 PM

It seems it’s because of the compiling issue on newer Java version. I can make a hot fix for that

Neha Pawar

05/07/2020, 4:02 PM

tried with a parquet file with data having similar characteristics to yours @kish. It worked fine for me 🤔

Copy code

2020/05/07 08:56:29.835 INFO [SegmentIndexCreationDriverImpl] [main] Finished building StatsCollector!
2020/05/07 08:56:29.836 INFO [SegmentIndexCreationDriverImpl] [main] Collected stats for 36000000 documents
2020/05/07 08:56:29.859 INFO [SegmentDictionaryCreator] [main] Created dictionary for INT column: m1 with cardinality: 49, range: 1 to 49
2020/05/07 08:56:29.875 INFO [SegmentDictionaryCreator] [main] Created dictionary for INT column: m2 with cardinality: 64000, range: 0 to 63999
2020/05/07 08:56:29.879 INFO [SegmentDictionaryCreator] [main] Created dictionary for LONG column: count with cardinality: 1, range: 1 to 1
2020/05/07 08:56:29.884 INFO [SegmentDictionaryCreator] [main] Using fixed bytes value dictionary for column: d1, size: 11
2020/05/07 08:56:29.884 INFO [SegmentDictionaryCreator] [main] Created dictionary for STRING column: d1 with cardinality: 1, max length in bytes: 11, range: ucdmJUDhdcv to ucdmJUDhdcv
2020/05/07 08:56:29.888 INFO [SegmentDictionaryCreator] [main] Using fixed bytes value dictionary for column: d2, size: 200000
2020/05/07 08:56:29.889 INFO [SegmentDictionaryCreator] [main] Created dictionary for STRING column: d2 with cardinality: 8000, max length in bytes: 25, range: A to zzltgSI
2020/05/07 08:56:29.897 INFO [SegmentDictionaryCreator] [main] Using fixed bytes value dictionary for column: d3, size: 570000
2020/05/07 08:56:29.898 INFO [SegmentDictionaryCreator] [main] Created dictionary for STRING column: d3 with cardinality: 38000, max length in bytes: 15, range: A to zzzqDAvrzsUvncL
2020/05/07 08:56:29.899 INFO [SegmentDictionaryCreator] [main] Using fixed bytes value dictionary for column: d4, size: 16
2020/05/07 08:56:29.899 INFO [SegmentDictionaryCreator] [main] Created dictionary for STRING column: d4 with cardinality: 4, max length in bytes: 4, range: Is to o
2020/05/07 08:56:29.900 INFO [SegmentDictionaryCreator] [main] Using fixed bytes value dictionary for column: d5, size: 910
2020/05/07 08:56:29.900 INFO [SegmentDictionaryCreator] [main] Created dictionary for STRING column: d5 with cardinality: 35, max length in bytes: 26, range: BHYTfkbxibsOoZYRInA to ysLZeqUdcMzWRYlWueR
2020/05/07 08:56:29.901 INFO [SegmentDictionaryCreator] [main] Using fixed bytes value dictionary for column: d6, size: 27
2020/05/07 08:56:29.901 INFO [SegmentDictionaryCreator] [main] Created dictionary for STRING column: d6 with cardinality: 3, max length in bytes: 9, range: GNhWQNwor to a
2020/05/07 08:56:29.902 INFO [SegmentDictionaryCreator] [main] Using fixed bytes value dictionary for column: d7, size: 35
2020/05/07 08:56:29.902 INFO [SegmentDictionaryCreator] [main] Created dictionary for STRING column: d7 with cardinality: 5, max length in bytes: 7, range: MlMtQ to z
2020/05/07 08:56:29.902 INFO [SegmentDictionaryCreator] [main] Using fixed bytes value dictionary for column: d8, size: 15
2020/05/07 08:56:29.903 INFO [SegmentDictionaryCreator] [main] Created dictionary for STRING column: d8 with cardinality: 5, max length in bytes: 3, range: cLf to x
2020/05/07 08:56:29.903 INFO [SegmentDictionaryCreator] [main] Using fixed bytes value dictionary for column: d9, size: 4553
2020/05/07 08:56:29.904 INFO [SegmentDictionaryCreator] [main] Created dictionary for STRING column: d9 with cardinality: 157, max length in bytes: 29, range: A to zwaGwwzPkH
2020/05/07 08:56:32.072 INFO [SegmentDictionaryCreator] [main] Created dictionary for LONG column: timestamp with cardinality: 29441824, range: 1587366000002 to 1587452399994
2020/05/07 08:56:32.148 INFO [SegmentIndexCreationDriverImpl] [main] Start building IndexCreator!
2020/05/07 08:59:05.704 INFO [SegmentIndexCreationDriverImpl] [main] Finished records indexing in IndexCreator!
2020/05/07 08:59:05.787 INFO [SegmentIndexCreationDriverImpl] [main] Finished segment seal!

Kishore G

05/07/2020, 4:03 PM

he is not seeing this log?

Kishore G

05/09/2020, 5:11 AM

https://github.com/apache/incubator-pinot/blob/master/pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/SingleParamMathTransformFunction.java

Dan Hill

05/09/2020, 5:22 AM

Has anyone gotten a left or full join to work with the Presto Pinot connector (and another table like MySQL)? Even if I specify 'left' or 'full', it looks like it's ignored. I filed a bug with Presto. https://github.com/prestodb/presto/issues/14508

Damiano

05/09/2020, 5:37 PM

Copy code

protected IntermediateResultsBlock getNextBlock() <--- here?

Damiano

05/11/2020, 10:21 AM

Hello everybody, is there a way to create an aggregator and return more than one column as result? I have many aggregators to implements and most of them have similar implementations so i thought to share their logic inside ONE aggregator, something like: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-stats-aggregation.html

srisudha

05/20/2020, 5:52 PM

Probably the FAQ needs to be updated isn't it? @Neha Pawar and @Jackie

Jackie

05/21/2020, 6:06 PM

This line can be ignored as we do not have config for

brokerResource

Copy code

2020/05/21 16:35:04.476 WARN [ConfigAccessor] [ZkClient-EventThread-27-zookeeper.test.svc.cluster.local:2181/pinot] No config found at /test-views/CONFIGS/RESOURCE/brokerResource

Kishore G

05/22/2020, 8:18 PM

tried it on quickstart and seems to work

Copy code

select playerID,league playerName, sum(hits) from baseballStats group by playerID, league, playerName
select league,playerID, playerName, sum(hits) from baseballStats group by league,playerID, playerName

srisudha

05/23/2020, 1:44 AM

I will give a try with 500 mb and higher and check it out

Dan Hill

05/24/2020, 4:58 PM

What's the small amount of memory that a local Pinot can use? I started up a local broker, server and controller and zookeeper and it seems like it needs a 1g of memory. How can I reduce the size of this?

Dan Hill

05/24/2020, 6:25 PM

I was trying to get a separate dev Pinot instance running that my local jobs can use. If I try to get a local Presto to hit a remote Pinot (using a service address), I get an exception that looks like Presto is getting passed the pinot-broker's k8 address. I'm guessing the controller is returning the k8 address to Presto. What's a good way around this?

Copy code

Query 20200524_182130_00003_5izra failed: <http://java.net|java.net>.UnknownHostException: pinot-broker-0.pinot-broker-headless.pinot-metrics-dev.svc.cluster.local: Name or service not known

Kishore G

06/01/2020, 9:37 PM

also after starting the server, you need to wait for health checks to make sure everything is loaded before moving on to next server

👍 1

Dan Hill

06/02/2020, 12:13 AM

Hmm, what's the recommended structure for calling LaunchDataIngestionJob? I'm trying to run it with 300 million rows and I'm hitting a tar size issue. I can split the DataIngestion calls but I'm curious as to what is recommended.

Copy code

java.lang.RuntimeException: entry size '14879990781' is too big ( > 8589934591 )
	at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumber(TarArchiveOutputStream.java:623) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-ed26e8589fe5f91d2876d417aebf23575010cc76]

Logs

kish

06/03/2020, 10:54 PM

Hi: Is row groups supported in parquet file format ingestion?

Kishore G

06/04/2020, 9:31 PM

Copy code

pinot.broker.timeoutMs

❤️ 1

Xiang Fu

06/08/2020, 12:46 AM

sudo docker run --rm -ti -e JAVA_OPTS="-Xms8G -Dlog4j2.configurationFile=/opt/pinot/conf/pinot-admin-log4j2.xml  -Dplugins.dir=/opt/pinot/plugins" -v /home/ec2-user/metrics:/home/ec2-user/metrics   -v /home/ec2-user/events:/home/ec2-user/events   --name pinot-data-ingestion-job   apachepinot/pinot:latest LaunchDataIngestionJob   -jobSpecFile ~/metrics/pinot/loadtest/kubernetes/aws-dev-batch-job-spec.yaml

👍 1

Xiang Fu

06/08/2020, 8:09 PM

please read this for more detailed configs: https://zookeeper.apache.org/doc/r3.4.3/zookeeperAdmin.html#sc_advancedConfiguration

❤️ 1