I have data merge sql errors after upgrading from ...
# openobserve
c
I have data merge sql errors after upgrading from 0.90.3 to 0.91.5 : `2026-08-11T123417.172846442+00:00 ERROR openobserve:jobcompactor [COMPACTOR::JOB] run data merge error: SqlxError# error occurred while decoding column "num": mismatched types; Rust type
i64
(as SQL type
INT8
) is not compatible with SQL type `INT4``
s
This looks like a type mismatch in the compactor introduced after upgrading to v0.91.5. Could you share the full error/stack trace so we can investigate?
c
that's all there is... just keeps repeating every 12 seconds
👀 1
In case it is useful, these are log entries from the db upgrade
Aug 11 13:25:14 oo openobserve[59557]: 2026-08-11T12:25:14.894842380+00:00 INFO openobserve::migration: DB_SCHEMA_VERSION mismatch : expected 45, found 41; running db upgrade
Aug 11 132514 oo openobserve[59557]: 2026-08-11T122514.897161196+00:00 INFO sqlx:postgresnotice relation "meta" already exists, skipping Aug 11 132514 oo openobserve[59557]: 2026-08-11T122514.903596544+00:00 INFO infra:file listpostgres [POSTGRES] Table file_list already partitioned, ensuring partitions Aug 11 132514 oo openobserve[59557]: 2026-08-11T122514.921540211+00:00 INFO infra:file listpostgres [POSTGRES] Table file_list_history already partitioned, ensuring partitions Aug 11 132514 oo openobserve[59557]: 2026-08-11T122514.930170118+00:00 INFO infra:file listpostgres [POSTGRES] Table file_list_dump_stats already partitioned, ensuring partitions Aug 11 132514 oo openobserve[59557]: 2026-08-11T122514.955373456+00:00 WARN infra:file listpostgres [POSTGRES] partition create blocked by DEFAULT overlap: file_list_p_20260712 Aug 11 132514 oo openobserve[59557]: 2026-08-11T122514.955587105+00:00 INFO sqlx:postgresnotice relation "file_list_dump_stats_p_20260712" already exists, skipping Aug 11 132514 oo openobserve[59557]: 2026-08-11T122514.956072933+00:00 WARN infra:file listpostgres [POSTGRES] partition create blocked by DEFAULT overlap: file_list_p_20260713 Aug 11 132514 oo openobserve[59557]: 2026-08-11T122514.956182398+00:00 INFO sqlx:postgresnotice relation "file_list_dump_stats_p_20260713" already exists, skipping Aug 11 132514 oo openobserve[59557]: 2026-08-11T122514.956530136+00:00 WARN infra:file listpostgres [POSTGRES] partition create blocked by DEFAULT overlap: file_list_p_20260714 Aug 11 132514 oo openobserve[59557]: 2026-08-11T122514.956622393+00:00 INFO sqlx:postgresnotice relation "file_list_dump_stats_p_20260714" already exists, skipping (and a few more of these, then) Aug 11 132514 oo openobserve[59557]: 2026-08-11T122514.970869215+00:00 INFO sqlx:postgresnotice relation "stream_stats" already exists, skipping Aug 11 132514 oo openobserve[59557]: 2026-08-11T122514.980597991+00:00 INFO infra:file listpostgres [POSTGRES] Skipping file column widen for file_list: already VARCHAR(1024) Aug 11 132514 oo openobserve[59557]: 2026-08-11T122514.981714637+00:00 INFO infra:file listpostgres [POSTGRES] Skipping file column widen for file_list_history: already VARCHAR(1024) Aug 11 132514 oo openobserve[59557]: 2026-08-11T122514.982466668+00:00 INFO infra:file listpostgres [POSTGRES] Skipping file column widen for file_list_deleted: already VARCHAR(1024) Aug 11 132514 oo openobserve[59557]: 2026-08-11T122514.983193932+00:00 INFO infra:file listpostgres [POSTGRES] Skipping file column widen for file_list_dump_stats: already VARCHAR(1024) Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.003329642+00:00 INFO infra:file listpostgres [POSTGRES] Widened account column for file_list to VARCHAR(128) Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.012066345+00:00 INFO infra:file listpostgres [POSTGRES] Widened account column for file_list_history to VARCHAR(128) Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.019696919+00:00 INFO infra:file listpostgres [POSTGRES] Widened account column for file_list_deleted to VARCHAR(128) Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.019980805+00:00 INFO sqlx:postgresnotice relation "pipeline" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.020992080+00:00 INFO sqlx:postgresnotice relation "pipeline_org_idx" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.021167946+00:00 INFO sqlx:postgresnotice relation "pipeline_org_src_type_stream_params_idx" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.021262186+00:00 INFO sqlx:postgresnotice relation "scheduled_jobs" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.023151150+00:00 INFO sqlx:postgresnotice relation "schema_history" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.023300604+00:00 INFO sqlx:postgresnotice relation "distinct_value_fields" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.023399546+00:00 INFO sqlx:postgresnotice relation "short_urls" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.023498927+00:00 INFO sqlx:postgresnotice relation "seaql_migrations" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.023589574+00:00 INFO sqlx:postgresnotice relation "seaql_migrations" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.023639160+00:00 INFO sqlx:postgresnotice relation "seaql_migrations" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.024034097+00:00 INFO sqlx:postgresnotice relation "seaql_migrations" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.024038792+00:00 INFO sea_orm_migration:migrator Applying 0 pending migrations Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.024075180+00:00 INFO sqlx:postgresnotice relation "seaql_migrations" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.024110595+00:00 INFO sqlx:postgresnotice relation "seaql_migrations" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.024147266+00:00 INFO sqlx:postgresnotice relation "seaql_migrations" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.024541121+00:00 INFO sqlx:postgresnotice relation "seaql_migrations" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.024545404+00:00 INFO sea_orm_migration:migrator Applying all pending migrations Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.024568914+00:00 INFO sqlx:postgresnotice relation "seaql_migrations" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.024594215+00:00 INFO sqlx:postgresnotice relation "seaql_migrations" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.024619261+00:00 INFO sqlx:postgresnotice relation "seaql_migrations" already exists, skipping Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.024746083+00:00 INFO sea_orm_migration:migrator Applying migration 'm20260506_000001_create_org_ingestion_tokens_table' Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.025872331+00:00 INFO sea_orm_migration:migrator Migration 'm20260506_000001_create_org_ingestion_tokens_table' has been applied Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.025971707+00:00 INFO sea_orm_migration:migrator Applying migration 'm20260401_00001_create_org_s3_table' Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.026433733+00:00 INFO sea_orm_migration:migrator Migration 'm20260401_00001_create_org_s3_table' has been applied Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.026486539+00:00 INFO sea_orm_migration:migrator Applying migration 'm20260520_000001_create_providers_table' Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.027613979+00:00 INFO sea_orm_migration:migrator Migration 'm20260520_000001_create_providers_table' has been applied Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.027659956+00:00 INFO sea_orm_migration:migrator Applying migration 'm20260520_000002_create_score_configs_table' Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.028946452+00:00 INFO sea_orm_migration:migrator Migration 'm20260520_000002_create_score_configs_table' has been applied Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.028993047+00:00 INFO sea_orm_migration:migrator Applying migration 'm20260520_000003_create_scorers_table' Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.030380312+00:00 INFO sea_orm_migration:migrator Migration 'm20260520_000003_create_scorers_table' has been applied Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.030428727+00:00 INFO sea_orm_migration:migrator Applying migration 'm20260520_000004_create_online_eval_jobs_table' Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.031625152+00:00 INFO sea_orm_migration:migrator Migration 'm20260520_000004_create_online_eval_jobs_table' has been applied Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.031672765+00:00 INFO sea_orm_migration:migrator Applying migration 'm20260520_000005_drop_eval_templates_table' Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.032183736+00:00 INFO sea_orm_migration:migrator Migration 'm20260520_000005_drop_eval_templates_table' has been applied Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.032222560+00:00 INFO sea_orm_migration:migrator Applying migration 'm20260604_000001_add_kind_to_pipeline' Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.032398981+00:00 INFO sea_orm_migration:migrator Migration 'm20260604_000001_add_kind_to_pipeline' has been applied Aug 11 132515 oo openobserve[59557]: 2026-08-11T122515.059821605+00:00 INFO openobserve:migration DB upgrade completed to version 45
Anything else I can get you?
s
Allow me sometime, i will try to reproduce and see what went wrong here
👍 1
h
can you keep upgrade to v0.92.0?
if you still got this error, please ping me
c
Ok will try that shortly
👍 1
@Hengfei 0.92.0 is different. Compaction does now appear to be running - it has caught up on quite a few. A few queries I've tried seem ok. I haven't re-enabled ingest yet. I do see these errors appearing quite a bit though:
Copy code
Aug 12 13:21:52 oo openobserve[3790]: 2026-08-12T12:21:52.141942414+00:00 ERROR search_service::cluster::flight: [trace_id 019ff5ebff8a7ab08af0226b1c537c0a] flight->search: create physical plan error: Optimizer rule 'simplify_expressions' failed
Aug 12 13:21:52 oo openobserve[3790]: caused by
Aug 12 13:21:52 oo openobserve[3790]: Arrow error: Cast error: Cannot cast string '1.0' to value of Int64 type
Aug 12 13:21:52 oo openobserve[3790]: 2026-08-12T12:21:52.141948622+00:00 ERROR search_service::cluster::flight: [trace_id 019ff5ebff8a7ab08af0225ee7bc2bbd] flight->search: create physical plan error: Optimizer rule 'simplify_expressions' failed
Aug 12 13:21:52 oo openobserve[3790]: caused by
Aug 12 13:21:52 oo openobserve[3790]: Arrow error: Cast error: Cannot cast string '1.0' to value of Int64 type
h
it looks some schema got problem. it should be string, but it try to convert to Int64.
you can get all the context of this keyword
019ff5ebff8a7ab08af0226b1c537c0a
c
sorry, not sure what you want me to do with that...
h
if you want to track the error, need you help to get all the related logs. then we can help.
c
sorry, just realised what you meant.... will send you the logs for that trace by pm
👍 1