This message was deleted.
# general
s
This message was deleted.
b
Copy code
{
  "type": "kafka",
  "spec": {
    "dataSchema": {
      "dataSource": "metrics-kafka",
      "timestampSpec": {
        "column": "timestamp",
        "format": "auto"
      },
      "dimensionsSpec": {
        "dimensions": [],
        "dimensionExclusions": [
          "timestamp",
          "value"
        ]
      },
      "metricsSpec": [
        {
          "name": "count",
          "type": "count"
        },
        {
          "name": "value_sum",
          "fieldName": "value",
          "type": "doubleSum"
        },
        {
          "name": "value_min",
          "fieldName": "value",
          "type": "doubleMin"
        },
        {
          "name": "value_max",
          "fieldName": "value",
          "type": "doubleMax"
        }
      ],
      "granularitySpec": {
        "type": "uniform",
        "segmentGranularity": "HOUR",
        "queryGranularity": "NONE"
      }
    },
    "ioConfig": {
      "topic": "metrics",
      "inputFormat": {
        "type": "json"
      },
      "consumerProperties": {
        "bootstrap.servers": "localhost:9092"
      },
      "taskCount": 1,
      "replicas": 1,
      "taskDuration": "PT1H"
    },
    "tuningConfig": {
      "type": "kafka",
      "maxRowsPerSegment": 5000000
    }
  }
}
j
@Bibek Sahoo The one which you shared is of supervisorSpec not about tasks, I am aware of supervisorSpec but currently interested in taskSpec to create kafka ingestion task... ThanksBTW!
b
For kafka, we do not submit task, We submit supervisor and task gets created automatically. For batch we submit task. I will check if i have some sample running task.
j
No problem, @Bibek Sahoo I have tasks running already which were created by supervisor for Kafka data ingestion, I just wanted to know If tasks can be created independent of supervisor for kafka but seems like it's not the case for Kafka data ingestion Thank You so much!!!
b
Thats Correct
🙌 1