Hi. We run SR 4.0.0 and hit an issue when added new MV that is based on an Iceberg table. At some point few CNs started to crash during sync with

[1770846486.520][thread: 140215171376704] je_mallctl execute dontdump success
[1770846486.520][thread: 140215171376704] je_mallctl execute purge success
    @     0x7f88c94e18c0 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x1268bf)
    @     0x7f88c944fac3 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x94ac2)
    @          0xdde3125 starrocks::Thread::supervise_thread(void*)
    @          0xddec588 starrocks::ThreadPool::dispatch_thread()
    @          0xcb28acc starrocks::MemtableFlushTask::run()
    @          0xcb26f6a starrocks::FlushToken::_flush_memtable(starrocks::MemTable*, starrocks::SegmentPB*, bool, long*)
    @          0xd5baee7 starrocks::MemTable::flush(starrocks::SegmentPB*, bool, long*)
    @          0xd948b7c starrocks::lake::SpillMemTableSink::flush_chunk(starrocks::Chunk const&, starrocks::SegmentPB*, bool, long*)
    @          0xd675993 starrocks::lake::HorizontalGeneralTabletWriter::flush(starrocks::SegmentPB*)
    @          0xd69eb65 starrocks::lake::HorizontalPkTabletWriter::flush_segment_writer(starrocks::SegmentPB*)
    @          0xd1e1b2f starrocks::SegmentWriter::finalize(unsigned long*, unsigned long*, unsigned long*)
    @          0xd1e138b starrocks::SegmentWriter::finalize_columns(unsigned long*)
    @          0xd221ef7 starrocks::FlatJsonColumnWriter::finish()
    @          0xd1eb98c starrocks::StringColumnWriter::finish()
    @          0xd1e89cd starrocks::ScalarColumnWriter::finish()
    @     0x7f88c93fd520 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x4251f)
    @     0x7f88ca5fbffe JVM_handle_linux_signal
    @     0x7f88ca5fb586 PosixSignals::chained_handler(int, siginfo_t*, void*) [clone .part.0]
    @         0x11942048 google::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*)
    @     0x7f88c9454ee8 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x99ee7)
*** SIGSEGV (@0x44) received by PID 28 (TID 0x7f86637b6640) LWP(2105) from PID 68; stack trace: ***
PC: @          0xd1e89cd starrocks::ScalarColumnWriter::finish()
*** Aborted at 1770846486 (unix time) try "date -d @1770846486" if you are using GNU date ***
query_id:00000000-0000-0000-0000-000000000000, fragment_instance:00000000-0000-0000-0000-000000000000

The issue has gone after I dropped this MV. Weird thing, that there is another MV that uses the same Iceberg table as source and has no sync issue. "bad" MV had extra boolean column added. Is it a known issue? Is there anything I can do to make it work?

Hi @Andrei Harbunou This issue seems to be related to Flat JSON, and adding the boolean appears to have actually affected the flat JSON. I’m not sure if this solution will be perfect, but I think it’s worth trying, so I’d like to share it: 1. Set

enable_flat_json

from

true

to

false

- Global Changes(Workaround 1) 2. Just Inavtive Specifiec MV

ALTER MATERIALIZED VIEW <db_name>.<mv_name> INACTIVE

- Just avoid issue 3. Rewrite MV Definition to CAST Boolean Column - Workaround 2 4. If flat_json feature very importent to you - Upgrade to version 4.0.1 or 4.0.2 (several Flat JSON related bugs have been fixed) For Number 3 approach is look like this. Instead of:

CREATE MATERIALIZED VIEW <mv_name>
   REFRESH ASYNC
   AS SELECT ..., bool_col, ...
   FROM iceberg_catalog.iceberg_db.iceberg_table;

Use:

CREATE MATERIALIZED VIEW <mv_name>
   REFRESH ASYNC
   AS SELECT ..., CAST(bool_col AS TINYINT) AS bool_col, ...
   FROM iceberg_catalog.iceberg_db.iceberg_table;

thanks! will try to apply your suggestions. we don't use json type at all, so it is weird to see this in logs.

Oh… if you’re not using FlatJson, that’s odd. Still, some of your logs appear to contain this line.

0xd221ef7 starrocks::FlatJsonColumnWriter::finish()

this is why I'm confused

speechless rocky Me either, can you share more details about your MV Table(how to create it)? I'll check more deeper.

let me try to reproduce issue, so far I hit it with one MV only. I will let you know if find anything.

Thank you for your testing! I'll very helpful.

does upgrade to 4.0.5 can help?

@Peter Pang unless another issue is fixed https://starrocks.slack.com/archives/C02FACZSNJV/p1767747167388909. otherwise I can't migrate