Does anyone have suggestions, how I can avoid this error?

greptimedb-1     | 2025-01-02T19:04:28.357368Z ERROR LocalManager::submit_root_procedure: common_procedure::local::runner: Failed to execute procedure metasrv-procedure::CreateFlow-6fde612e-38ef-4ad9-9d80-f1f80ad2fbfe, retry: false err=0: Failed to execute procedure due to external error
greptimedb-1     | 1: Failed to operate on datanode: peer-0(), at src/common/meta/src/ddl/utils.rs:37:18
greptimedb-1     | 2: External error, at src/flow/src/adapter/flownode_impl.rs:40:10
greptimedb-1     | 3: Not implement in flow: Aggregated function with multiple arguments is not supported, at src/flow/src/transform/aggr.rs:131:20
g

Is it expected that the JSON field type is returned as a byte array in the dashboard?

I have raised a PR to fix an issue in Safari for the dashboard: https://github.com/GreptimeTeam/dashboard/pull/492

Greetings, I'm trying to run GreptimeDB cluster in Nomad using podman. Everything is working, except scaling. When I take out one node - cluster seems healthy, but I cant execute queries. SELECT 1 works, bust SELECT something from table - isnt. In frontend server logs I see that it cant connect to RegionServer (because it is down). But it should be migrate regions and not try to query that node May I ask you to help me understand what I am doing wrong? Metaserver config

enable_region_failover=true

        [meta]
        enable_region_failover=true

        [storage]
        type = "S3"
        endpoint = "http://${s3_endpoint}"
        bucket = "${s3_bucket}"
        region = "us-east-1"
        root = "/greptimedb"
        access_key_id = "${s3_access_key_id}"
        secret_access_key = "${s3_access_key}"

        [wal]
        provider = "kafka"
        broker_endpoints = [{{ range nomadService "kafka" }}"{{ .Address }}:{{ .Port }}",{{end}}]
        max_batch_bytes = "1MB"
        consumer_wait_timeout = "100ms"
        backoff_init = "500ms"
        backoff_max = "10s"
        backoff_base = 2
        backoff_deadline = "5mins"
        overwrite_entry_start_id = false

Datanode config

[storage]
        type = "S3"
        endpoint = "http://${s3_endpoint}"
        bucket = "${s3_bucket}"
        region = "us-east-1"
        root = "/greptimedb"
        access_key_id = "${s3_access_key_id}"
        secret_access_key = "${s3_access_key}"

        cache_path = "/data/greptimedb_cache"
        cache_capacity = "100GiB"

        [wal]
        provider = "kafka"
        broker_endpoints = [{{ range nomadService "kafka" }}"{{ .Address }}:{{ .Port }}",{{end}}]
        max_batch_bytes = "1MB"
        consumer_wait_timeout = "100ms"
        backoff_init = "500ms"
        backoff_max = "10s"
        backoff_base = 2
        backoff_deadline = "5mins"
        overwrite_entry_start_id = false

        [[region_engine]]
        [region_engine.mito]
        write_cache_size = "20G"
        sst_meta_cache_size = "256MB"
        vector_cache_size = "512MB"
        page_cache_size = "512MB"
        selector_result_cache_size = "512MB"

        enable_write_cache = true

        write_cache_path = "/data/greptimedb_write_cache"

        [region_engine.mito.index]
        staging_size = "2GB"

Hi team, I have not an issue, but rather question regarding RANGE queries Im trying to execute something like this

select
	greptime_timestamp,
	client_request_host,
	count(*) RANGE '10s' FILL LINEAR
from
	logs.cloudflare
WHERE
	greptime_timestamp between '2025-04-01 00:00:00' and '2025-04-02 00:00:00'

But I've got an error

SQL Error [1815] [HY000]: (conn=8) (PlanQuery): Failed to plan SQL: Error during planning: Projection references non-aggregate values: Expression logs.cloudflare.greptime_timestamp could not be resolved from available columns: count(*)

What am I doing wrong? I tried multiple combinations - none successful

hey there, I’m running the following PromQL and it works correctly on it’s own:

max_over_time(power[30s]) > 250

When I try to add a condition to filter for weekends using

day_of_week()

, the query fails. I have tried several variations, including:

max_over_time(power[30s]) > 250 and (day_of_week() == 0 or day_of_week() == 6)
max_over_time(power[30s]) * (day_of_week() == 0 or day_of_week() == 6) > 250

All of these attempts result in the same error:

Query 1 ERROR at Line 1: : (PlanQuery): Internal error during building DataFusion plan: No field named asset. Valid fields are .time.

Am I doing something wrong or is there a limitation in the PromQL implementation that prevents this from working?

Hey folks, Is this issue open to work on? https://github.com/GreptimeTeam/greptimedb/issues/3004 since this is a year old, I am not sure this is still a priority.

hey there again, we’ve been experiencing a substantial performance degradation over time in our PromQL queries in queries that take place over a large period of time (7d+) and wanted to ask for some guidance. We’re using a custom metrics engine table with around 27 million rows (15 logical tables). GreptimeDB is hosted in standalone mode on a single node with 2GB of memory and 1 CPU core. Memory usage doesn’t seems to be the issue as we have around 1.4GB free, but CPU usage is a bit high at around 40-60% during query execution. For storage we are using a S3 compatible object storage. are there best practices we should follow to maintain query performance at this scale for metrics tables? Any tips on optimizing performance (storage layout, compaction, indexing, etc.) would be much appreciated

Hello, I need a bit of assistance. We have an S3 object storage that operates with a self-signed certificate. I would like to configure it with Greptime. Is there a way to bypass SSL validation?

Hi. I want to set up a digitaloceanspaces bucket as storage. However, when trying to create a table I get this error:

error sending request for url (<http://169.254.169.254/latest/api/token>): client error (Connect): tcp connect error: Connection refused (os error 111): Connection refused (os error 111)

Is there any configuration option I am missing? I already tried changing 'endpoint' and 'enable_virtual_host_style' without luck. It seems that GreptimeDB's S3 client library is hardcoded to try AWS metadata service (http://169.254.169.254/latest/api/token) even when I provide explicit credentials and a non-AWS endpoint.

Hi. I am trying to use dlt hub (https://dlthub.com/) to manage the data loading and I want to use it to load data into tables in GrepTimeDB. I did some tests but it is failing due to some specific rules of GrepTimeDB (i.e. "Reserved words need to be quoted: Column names like version and schema need to be quoted with backticks", and also "Missing time index constraint: GrepTimeDB requires a time index constraint for all tables"). Do you know of any integrations or examples that solve this issues? Thanks in advance.

Hi. We are trying to ingest some opentelemetry data into GreptimeDB. And I see that, for some metrics, tables are created, but there are 0 records in them Also I see lots of errors in logs (frontend node)

2025-06-30T14:23:05.657552899-04:00 stdout F 2025-06-30T18:23:05.657468Z ERROR metrics{protocol="otlp" request_type="metrics"}:metrics:handle_row_inserts:alter_table_inner: meta_client::client::procedure: An error occurred in gRPC, status: status: Cancelled, message: "Timeout expired", details: [], metadata: MetadataMap { headers: {} }
2025-06-30T14:23:05.658180268-04:00 stdout F 2025-06-30T18:23:05.658131Z ERROR servers::error: Failed to handle HTTP request  err=0: Execute gRPC query error, at src/frontend/src/instance/otlp.rs:69:14
2025-06-30T14:23:05.658180268-04:00 stdout F 1: Table operation error, at src/frontend/src/instance/grpc.rs:423:14
2025-06-30T14:23:05.658180268-04:00 stdout F 2: Failed to execute ddl, at /greptimedb/src/operator/src/statement/ddl.rs:1202:14
2025-06-30T14:23:05.658180268-04:00 stdout F 3: External error, at src/meta-client/src/client.rs:262:14
2025-06-30T14:23:05.658180268-04:00 stdout F 4: Timeout expired, at src/meta-client/src/error.rs:190:23

May I ask for advice how to debug it, and maybe how to solve this issue?

Hi, We're experiencing an issue with data ingestion into an existing table using Vector with the identity pipeline. The data format is simple JSON Lines, and the setup has been working flawlessly until today. The table currently holds around 7 billion records. Suddenly, Vector began receiving HTTP 400 responses during ingestion. There are no relevant logs on the frontend node, datanode, or metasrv—even when running in debug mode. Other pipelines ingesting into similar tables are functioning without issues. Interestingly, if I configure the same pipeline to ingest into a newly created table, it works as expected. Could you please advise on how to troubleshoot or resolve this issue?

Hi, Sorry for bothering you. But I have another question I'm ingesting OpenTelemetry using grafana alloy and I'm getting

sending queue is full

errors on alloy side. Alloy is basically wrapper around opentelemetry-collector On greptime frontend side I see no errors. May I ask you for an advice on how to debug this? Is seems like frontend node is backing-off or something. But no logs for that events

👋 Hello, team!

Hi, Not an issue, but rather question. We are using greptimedb with Grafana (standard promQL connector). Grafana have this new(-ish) feature - drilldown or metrics explorer. I partially working with greptimedb datasource, but I see an error in Grafana UI

Matcher operator =~ is not supported for __name__

I guess its a part for their metric search query. Are there any plans to support this in the future?

Hi. I'm installing new cluster using v0.16.0. I have same configuration as for v0.15.5, except postgresql store backend for metasrv. Frontend node is closing connection when I'm trying to access it (via any protocol - mysql or http). Same config with v0.15 image is working okay. Do I need to change something somewhere? What can be a cause of this issue?

I'm having some difficulty figuring out my query to do averaging over a time period to down sample my data. I was expecting to be able to do

$__interval

but that doesn't seem to work. I get

"Invalid SQL syntax: sql parser error: not a valid duration string: $__interval"

. I'm guessing this is more of a me issue than something is broken but any advice would be appreciated. Edit: I did try hardcoding '1ms', and '100ms' into it and that did work so it seems like something isn't working with the substitution. The whole was looking something like this:

SELECT
    ts,
    AVG(temperature) RANGE $__interval
FROM
    "public"."sensor_data"
WHERE
    ts >= $__fromTime AND ts <= $__toTime
ALIGN
    $__interval
ORDER BY
    ts ASC

I'm writing the opentelemetry traces to the database using greptime_trace_v1 pipeline as described in the docs. The traces data is getting properly distributed across the 16 partitions as defined in the opentelemetry_traces table. I've verified the parquet files in a given partition storing traces data that begins with the same letter. But when I query using trace_id in the where clause, it's scanning all the partition thereby taking a lot of time for returning result. The output of [explain select trace_id,timestamp,service_name from opentelemetry_traces WHERE trace_id='5c5f055730a1a04b1644f9e664406462' and timestamp >= '2025-09-28 140000' and timestamp < '2025-09-28 140200'] is as below - logical_plan MergeScan [is_placeholder=false] physical_plan MergeScanExec: peers=[5098126180352(1187, 0), 5098126180353(1187, 1), 5098126180354(1187, 2), 5098126180355(1187, 3), 5098126180356(1187, 4), 5098126180357(1187, 5), 5098126180358(1187, 6), 5098126180359(1187, 7), 5098126180360(1187, 😎, 5098126180361(1187, 9), 5098126180362(1187, 10), 5098126180363(1187, 11), 5098126180364(1187, 12), 5098126180365(1187, 13), 5098126180366(1187, 14), 5098126180367(1187, 15), ] CREATE TABLE IF NOT EXISTS

opentelemetry_traces

(

timestamp

TIMESTAMP(9) NOT NULL,

timestamp_end

TIMESTAMP(9) NULL,

duration_nano

BIGINT UNSIGNED NULL,

parent_span_id

STRING NULL SKIPPING INDEX WITH(granularity = '10240', type = 'BLOOM'),

trace_id

STRING NULL SKIPPING INDEX WITH(granularity = '10240', type = 'BLOOM'),

span_id

STRING NULL,

span_kind

STRING NULL,

span_name

STRING NULL,

span_status_code

STRING NULL,

span_status_message

STRING NULL,

trace_state

STRING NULL,

scope_name

STRING NULL,

scope_version

STRING NULL,

service_name

STRING NULL SKIPPING INDEX WITH(granularity = '10240', type = 'BLOOM'),

span_attributes.thread.name

STRING NULL, =======more rows======= TIME INDEX (

timestamp

), PRIMARY KEY (

service_name

) ) PARTITION ON COLUMNS (

trace_id

) ( trace_id < '1', trace_id >= 'f', trace_id >= '1' AND trace_id < '2', trace_id >= '2' AND trace_id < '3', trace_id >= '3' AND trace_id < '4', trace_id >= '4' AND trace_id < '5', trace_id >= '5' AND trace_id < '6', trace_id >= '6' AND trace_id < '7', trace_id >= '7' AND trace_id < '8', trace_id >= '8' AND trace_id < '9', trace_id >= '9' AND trace_id < 'a', trace_id >= 'a' AND trace_id < 'b', trace_id >= 'b' AND trace_id < 'c', trace_id >= 'c' AND trace_id < 'd', trace_id >= 'd' AND trace_id < 'e', trace_id >= 'e' AND trace_id < 'f' ) ENGINE=mito WITH( append_mode = 'true', table_data_model = 'greptime_trace_v1', ttl = '2years' ) Please guide.

Hi, We are trying to build Grafana dashboards using promql. Thing is - we ingest metrics in greptime using opentelemetry and vector.dev, so they end up looking like this:

CREATE TABLE IF NOT EXISTS "kubelet_volume_stats_inodes" (
  "ts" TIMESTAMP(3) NOT NULL,
  "endpoint" STRING NULL,
  "k8s_cluster" STRING NULL,
  "metrics_path" STRING NULL,
  "namespace" STRING NULL,
  "node" STRING NULL,
  "persistentvolumeclaim" STRING NULL,
  "resource.http.scheme" STRING NULL,
  "resource.k8s_cluster" STRING NULL,
  "resource.namespace" STRING NULL,
  "resource.net.host.name" STRING NULL,
  "resource.net.host.port" STRING NULL,
  "resource.server.address" STRING NULL,
  "resource.server.port" STRING NULL,
  "resource.service.instance.id" STRING NULL,
  "resource.service.name" STRING NULL,
  "resource.url.scheme" STRING NULL,
  "scope.name" STRING NULL,
  "scope.version" STRING NULL,
  "service" STRING NULL,
  "val" DOUBLE NULL,
  TIME INDEX ("ts"),
  PRIMARY KEY ("endpoint", "k8s_cluster", "metrics_path", "namespace", "node", "persistentvolumeclaim", "resource.http.scheme", "resource.k8s_cluster", "resource.namespace", "resource.net.host.name", "resource.net.host.port", "resource.server.address", "resource.server.port", "resource.service.instance.id", "resource.service.name", "resource.url.scheme", "scope.name", "scope.version", "service")
)

ENGINE=mito
;

How can we query these metrics using labels with periods. Something like

kubelet_volume_stats_inodes{resource.http.scheme="http"}

is not valid query. And replacing

.

with

_

or

__

is not working either. May I ask - how should we query those metrics using promql? Thank you in advance. Greoptimedb v1.0.0-beta1

Hi, I am still have this problem reproducible at my laptop. Let me know if any details you need to be able reproduce it at your end. I am running greptimedb standalone on my laptop in kind cluster. The table now has 1,202,640 rows, but I was facing the same problem on Friday with just 600,000. I am able to query this table with simple queries, but the moment I am adding ORDER BY the database crashes with 2025-12-22T142430.453876Z ERROR common_telemetry:panic hook panicked at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-buffer-56.2.0/src/buffer/scalar.rs7052: offset overflow backtrace= 0: common_telemetry:panic hookset panic hook:{{closure}}

1: std::panicking::panic_with_hook
   2: std::panicking::panic_handler::{{closure}}
   3: std::sys::backtrace::__rust_end_short_backtrace
   4: __rustc::rust_begin_unwind
   5: core::panicking::panic_fmt
   6: core::option::expect_failed
   7: arrow_buffer::buffer::scalar::ScalarBuffer<T>::new
   8: <arrow_array::array::byte_array::GenericByteArray<T> as arrow_array::array::Array>::slice
   9: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter
  10: query::part_sort::PartSortStream::can_stop_early
  11: query::part_sort::PartSortStream::split_batch
  12: <query::part_sort::PartSortStream as futures_core::stream::Stream>::poll_next
  13: query::window_sort::WindowedSortStream::poll_next_inner
  14: <query::window_sort::WindowedSortStream as futures_core::stream::Stream>::poll_next
  15: datafusion_common_runtime::trace_utils::trace_future::{{closure}}
  16: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
  17: tokio::runtime::task::core::Core<T,S>::poll
  18: tokio::runtime::task::harness::Harness<T,S>::poll
  19: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
  20: tokio::runtime::context::scoped::Scoped<T>::set
  21: tokio::runtime::context::runtime::enter_runtime
  22: tokio::runtime::scheduler::multi_thread::worker::run
  23: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
  24: tokio::runtime::task::core::Core<T,S>::poll
  25: tokio::runtime::task::harness::Harness<T,S>::poll
  26: tokio::runtime::blocking::pool::Inner::run
  27: std::sys::backtrace::__rust_begin_short_backtrace
  28: core::ops::function::FnOnce::call_once{{vtable.shim}}
  29: std::sys::thread::unix::thread::new::thread_start
  30: <unknown>
  31: <unknown>
 panic.file="/root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-buffer-56.2.0/src/buffer/scalar.rs" panic.line=70 panic.column=52
Well, this is embarrassing.

GreptimeDB had a problem and crashed. To help us diagnose the problem you can send us a crash report.

Hi there! I wonder if there is workaround for the below behavior. I'm trying to add a column (an attribute one) to the existing Flow, and got the below error:

Failed to generate query for flow=1082 err=0: Datafusion error: Plan("Expect the last column in table to be timestamp column, found column ip_address with type String(StringType { size_type: Utf8 })")

@Nagato Yuzuru joined #issues. They’re also new to GreptimeDB Community.

@Navin shrinivas joined #issues. They’re also new to GreptimeDB Community.

hi all, I'm trying to replace my influxdb 1.x setup with greptimedb, but I'm getting consistently very slow queries when querying with promql and I can't figure it out. almost any query with promql takes pretty consistently 17-19 seconds to complete, while almost any sql query completes within some 100-300ms typically in my setup I'm storing mainly sensor data (eg. temperature, humidity, etc), inserting that to greptimedb using the influxdb api, and querying from grafana by having greptimedb added as a propetheus data source (as I'm most familiar with promql so I'd prefer using it for my dashboards). I have verified I'm getting pretty much exact same performance characteristics through grafana and through the greptimedb builtin dashboard UI, so it's not a grafana issue it seems to be more or less same with any kind of query, but for example using promql:

ruuvi_measurements{__database__="ruuvi",__field__="temperature"}

takes rather consistently that 17-19 seconds if I query using a week long timeframe, one day, or even just that last 1 hour of data, and then for example this:

SELECT
  "name",
  "dataFormat",
  "mac",
  AVG("temperature") AS avg_temperature
FROM ruuvi."ruuvi_measurements"
WHERE "greptime_timestamp" >= NOW() - INTERVAL '7 day'
GROUP BY "name", "dataFormat", "mac"
ORDER BY "name";

completes within that 100-300ms even with a week worth of data, and unsurprisingly it's pretty much equally fast with only last 1 hour of data. I have not tuned many settings in my config as I figured the defaults are probably quite good starting point. I am using s3 compatible object storage (backblaze b2) as the main storage backend if that makes any difference. I also tried enabling slow query logging if it would reveal any additional insights, but it seems not, as the queries appear in that log as expected is this kind of performance expected with promql? if not, what can I do to troubleshoot the issue further?

Hi everyone, We’re currently using GreptimeDB 1.0 in distributed mode and have been running into some issues with the write cache. We’ve noticed that disk usage keeps growing until the disks become completely full (100% usage) on both our datanode and frontend nodes. Our current configuration looks like this:

[[region_engine]]
[region_engine.mito]
write_cache_size = "10G"
write_cache_ttl = "1h"
sst_meta_cache_size = "256MB"
vector_cache_size = "512MB"
page_cache_size = "512MB"
selector_result_cache_size = "512MB"
enable_write_cache = true
write_cache_path = "/data/greptimedb_write_cache"

[region_engine.mito.index]
staging_size = "2GB"

Even though

write_cache_size

is set to

10G

, the

write_cache_path

directory continues to grow and eventually reaches around 200 GB. I was hoping someone could help me understand: • Is it expected for the write cache to grow beyond the configured

write_cache_size

? • Are there additional settings we should tune to better control disk usage? • Is there any configuration related to disk usage for frontend nodes? I wasn’t able to find anything in the documentation. Any suggestions or insights would be greatly appreciated. Thanks in advance!

Hi! Before submitting an issue, I decided to check if this is not expected behavior. Starting version 1.0.1 (1.0.2 shows same behavior) we noticed, that mysql interface does not expect arguments for LIMIT. We use golang mysql module github.com/go-jet/jet/v2/mysql And for the query

SELECT limit_param_repro.item_id AS "out_item_id",
     limit_param_repro.sample_score AS "out_sample_score"
FROM limit_param_repro
LIMIT ?;

we have 1 argument:

args (1): []interface {}{1}

and we get the below error:

run query: sql: expected 0 arguments, got 1

Hi! I have a question about authorization. If we use static_user_provider we cant start flownode. It fails with error:

The request does not have valid authentication credentials, source: 0: Not found http or grpc authorization header, at src/client/src/error.rs:173:1
2026-07-17T13:26:30.462954537-04:00 stderr F Error: 0: Failed to start flownode, at src/cmd/src/flownode.rs:96:37
2026-07-17T13:26:30.462954537-04:00 stderr F 1: Invalid request: Frontend <> rejected unauthenticated flownode probe; ensure frontend internal_grpc is advertised to metasrv, at src/flow/src/batching_mode/frontend_client.rs:244:38
2026-07-17T13:26:30.462954537-04:00 stderr F 2: Failed to do Flight get, code: The request does not have valid authentication credentials
2026-07-17T13:26:30.462954537-04:00 stderr P 3: Not found http or grpc authorization header, at src/client/src/error.rs:173:1

Greptimedb 1.1.2 Can you advice correct config for flownode please?

:::writing{variant="email" id="58321" subject="Grafana GreptimeDB plugin: intermittent "column not found" errors"} Hi, I have another issue, this time I believe it may be related to the Grafana plugin. I installed the latest version of the GreptimeDB plugin on the latest version of Grafana. I have a logs table populated through an identity pipeline, where JSON fields are parsed into columns, and the pipeline itself appears to be working as expected. However, I’m seeing inconsistent behavior on the Grafana side. When I query the table, I sometimes receive a 500 error indicating that a column cannot be found. If I simply refresh the Grafana page and run the same query again, I get the expected results. The error then comes back again later, so the behavior is not consistent. Here is an example of the query:

{
  "request": {
    "url": "api/ds/query?ds_type=info8fcc-greptimedb-datasource&requestId=SQR123",
    "method": "POST",
    "data": {
      "queries": [
        {
          "refId": "A",
          "hide": false,
          "datasource": {
            "type": "info8fcc-greptimedb-datasource",
            "uid": "Greptimedb"
          },
          "editorType": "sql",
          "format": 1,
          "meta": {
            "builderOptions": {
              "columns": [],
              "database": "",
              "limit": 1000,
              "meta": {},
              "mode": "list",
              "queryType": "table",
              "table": ""
            },
            "timezone": "America/New_York"
          },
          "pluginVersion": "2.1.4",
          "queryType": "table",
          "rawSql": "SELECT greptime_timestamp as time, \"message.traceId\", \"message.spanId\", \"message.error.message\", \"message.message\", FROM logs.\"k8s_app_logs\" WHERE greptime_timestamp BETWEEN $__fromTime and $__toTime AND level = 'ERROR' ORDER BY greptime_timestamp DESC LIMIT 1000",
          "datasourceId": 7,
          "intervalMs": 5000,
          "maxDataPoints": 1066
        }
      ],
      "from": "1787087854703",
      "to": "1787094539531"
    },
    "hideFromInspector": false
  },
  "response": {
    "results": {
      "A": {
        "error": "greptime http 400: {\"code\":3000,\"error\":\"Failed to plan SQL: No field named \\\"message.traceId\\\". Valid fields are logs.k8s_app_logs.greptime_timestamp, logs.k8s_app_logs.k8s_cluster, logs.k8s_app_logs.message, logs.k8s_app_logs.namespace, logs.k8s_app_logs.observed_timestamp, logs.k8s_app_logs.level.\",\"execution_time_ms\":3}",
        "errorSource": "downstream",
        "status": 500,
        "frames": [],
        "refId": "A"
      }
    }
  }
}

The interesting part is that the error says the parsed fields such as

message.traceId

do not exist, while the identity pipeline is successfully creating/populating these fields. There is also another related issue when I use the MySQL connector. In that case, I don't get an error, but instead of returning the field value, Grafana displays the column name itself as a string. For example, if I query:

SELECT "k8s_app_logs.k8s_cluster" FROM ...

the result is literally

k8s_app_logs.k8s_cluster

rather than the value stored in that field. Could you please take a look and let me know if this is a known issue with the Grafana plugin, or if there is something I may be missing in the configuration? Thanks! :::

Hi, Sorry I have large number of questions I created table for logs. We send logs via vector and it works great. Greptime 1.1.2 and vector 0.56 (latest I think) When I create json column type for message (message is valid json) - vector can't ingest data - I see lots of errors. But if message is just string - it works perfectly fine. Maybe its known issue and you know how to make it work? Thank you in advance