Xiang Fu
sudo docker run --rm -ti -e JAVA_OPTS="-Xms8G -Dlog4j2.configurationFile=/opt/pinot/conf/pinot-admin-log4j2.xml -Dplugins.dir=/opt/pinot/plugins" -v /home/ec2-user/metrics:/home/ec2-user/metrics -v /home/ec2-user/events:/home/ec2-user/events --name pinot-data-ingestion-job apachepinot/pinot:latest LaunchDataIngestionJob -jobSpecFile ~/metrics/pinot/loadtest/kubernetes/aws-dev-batch-job-spec.yaml
Dan Hill
06/08/2020, 3:46 AMFinished constructing star-tree, got 2 tree nodes and 0 records under star-node
Finished creating aggregated documents, got 2 aggregated records
Finished building star-tree in 50ms
Finished building 2 star-trees in 219ms
Computed crc = 222171945, based on files [/tmp/pinot-1591585815037/output/metrics_OFFLINE_1589501075305_1589587134620_0/v3/columns.psf, /tmp/pinot-1591585815037/output/metrics_OFFLINE_1589501075305_1589587134620_0/v3/index_map, /tmp/pinot-1591585815037/output/metrics_OFFLINE_1589501075305_1589587134620_0/v3/metadata.properties, /tmp/pinot-1591585815037/output/metrics_OFFLINE_1589501075305_1589587134620_0/v3/star_tree_index, /tmp/pinot-1591585815037/output/metrics_OFFLINE_1589501075305_1589587134620_0/v3/star_tree_index_map]
Driver, record read time : 36
Driver, stats collector time : 0
Driver, indexing time : 31
Tarring segment from: /tmp/pinot-1591585815037/output/metrics_OFFLINE_1589501075305_1589587134620_0 to: /tmp/pinot-1591585815037/output/metrics_OFFLINE_1589501075305_1589587134620_0.tar.gz
Size for segment: metrics_OFFLINE_1589501075305_1589587134620_0, uncompressed: 420.96K, compressed: 76.55K
Trying to create instance for class org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner
Initializing PinotFS for scheme file, classname org.apache.pinot.spi.filesystem.LocalPinotFS
Start pushing segments: [/tmp/metrics/segments/metrics_OFFLINE_1589501075305_1589587134620_0.tar.gz]... to locations: [org.apache.pinot.spi.ingestion.batch.spec.PinotClusterSpec@275bf9b3] for table metrics
Pushing segment: metrics_OFFLINE_1589501075305_1589587134620_0 to location: <http://a257ed0c56c7b4715a084a2134e4c50b-1388897056.us-east-1.elb.amazonaws.com:9000> for table metrics
Sending request: <http://a257ed0c56c7b4715a084a2134e4c50b-1388897056.us-east-1.elb.amazonaws.com:9000/v2/segments?tableName=metrics> to controller: pinot-controller-0.pinot-controller-headless.pinot-loadtest.svc.cluster.local, version: Unknown
Response for pushing table metrics segment metrics_OFFLINE_1589501075305_1589587134620_0 to location <http://a257ed0c56c7b4715a084a2134e4c50b-1388897056.us-east-1.elb.amazonaws.com:9000> - 200: {"status":"Successfully uploaded segment: metrics_OFFLINE_1589501075305_1589587134620_0 of table: metrics"}
Xiang Fu
Xiang Fu
Xiang Fu
200: {"status":"Successfully uploaded segment: metrics_OFFLINE_1589501075305_1589587134620_0 of table: metrics"}
Dan Hill
06/08/2020, 6:08 AM