HI I am getting this error on cn current cn versi...
# questions-and-troubleshooting
n
HI I am getting this error on cn current cn version : 3.3.19
Copy code
*** Aborted at 1761719895 (unix time) try "date -d @1761719895" if you are using GNU date ***
PC: @          0x9cee060 google::protobuf::RepeatedPtrField<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::empty() const
*** SIGSEGV (@0x20) received by PID 27 (TID 0xfffe85300040) from PID 32; stack trace: ***
    @     0xffffae2153dc (/usr/lib/aarch64-linux-gnu/libc.so.6+0x853db)
    @          0x9932bf8 google::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*)
    @     0xffffaf4f7850 ([vdso]+0x84f)
    @          0x9cee060 google::protobuf::RepeatedPtrField<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::empty() const
    @          0x69c94b0 starrocks::lake::UpdateManager::publish_primary_compaction(starrocks::TxnLogPB_OpCompaction const&, long, starrocks::TabletMetadataPB const&, starrocks::lake::Tablet const&, starrocks::DynamicCache<unsigned long, starrocks::lake::LakePrimaryIndex>::Entry*,
    @          0x83ee878 std::_Function_handler<starrocks::Status (), starrocks::lake::PrimaryKeyTxnLogApplier::apply(starrocks::TxnLogPB const&)::{lambda()#2}>::_M_invoke(std::_Any_data const&)
    @          0x83ef660 starrocks::lake::PrimaryKeyTxnLogApplier::check_and_recover(std::function<starrocks::Status ()> const&)
I20251029 06:38:15.793957 281468878127168 <http://service.cc:56]|service.cc:56]> Add shard id: 4710662, fs type: S3, fs key: d91d1094-f5ea-466a-81d6-1cae10a87e7e, file path: <s3://prod-eu.unifyapps.com/starrocks/reporting/27520c45-9b2c-44c3-a268-3f6461ee97c5/db10561/10854/4710660>, cache_enable: 1, cache_ttl: 0, async_write_back: 0, hash_code: 2192
    @          0x83f2258 starrocks::lake::PrimaryKeyTxnLogApplier::apply(starrocks::TxnLogPB const&)
    @          0x83e9fb8 starrocks::lake::publish_version(starrocks::lake::TabletManager*, long, long, long, std::span<starrocks::TxnInfoPB const, 18446744073709551615ul>)
    @          0x83dd508 starrocks::LakeServiceImpl::publish_version(google::protobuf::RpcController*, starrocks::PublishVersionRequest const*, starrocks::PublishVersionResponse*, google::protobuf::Closure*)::{lambda()#1}::operator()() const
    @          0x8515d40 std::_Function_handler<void (), starrocks::ConcurrencyLimitedThreadPoolToken::submit(std::shared_ptr<starrocks::Runnable>, std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >)::{lambda()#1}>::
    @          0x851cd14 starrocks::ThreadPool::dispatch_thread()
    @          0x85135ec starrocks::Thread::supervise_thread(void*)
    @     0xffffae210398 (/usr/lib/aarch64-linux-gnu/libc.so.6+0x80397)
    @     0xffffae279e9c (/usr/lib/aarch64-linux-gnu/libc.so.6+0xe9e9b)
[1761719895.851][thread:281468621291584] je_mallctl execute purge success
[1761719895.851][thread:281468621291584] je_mallctl execute dontdump success
/opt/starrocks/cn_entrypoint.sh: line 187:    27 Segmentation fault      (core dumped) $STARROCKS_HOME/bin/start_cn.sh $addition_args
k
can you report the issue on github?
n
is there any way to take backup of data?
also this is failing within few seconds after startup
@Kevin Cai can someone please help here? https://github.com/StarRocks/starrocks/issues/64723
we need data backup
k
it's a little bit hard if the coredump can't be generated. The root cause identification will be taken time.
n
can I write any script to copy data from s3 and ingest to new starrocks cluster?
is that possible
k
yes, you can use bin/meta_tool.sh to dump the data into human readable format and regenerate it to csv or parquet format , and then ingest into a different cluster.
n
can please help with sample command or documentation?
@Kevin Cai is this it possible to retreive data from s3? since this is shared-data cn
k
sr itself doesn't have the ability to pull the data back from remote because FE set the latest version to pull from the tablet and the tablet is causing the cn continuously crashing.
n
no i mean using bin/meta_tool.sh
Copy code
./bin/meta_tool.sh \
  --operation=dump_segment_data \
  --file=27520c45-9b2c-44c3-a268-3f6461ee97c5/db10861/13505/13982/data/0000000000237d61_b922684d-0293-445d-b0f0-727b1a2382c6.dat \
  > segment_dump.tsv
is this correct ?
Copy code
<s3://starrocks-bucket/starrocks/reporting/27520c45-9b2c-44c3-a268-3f6461ee97c5/db10861/13505/13982/data/0000000000237d61_b922684d-0293-445d-b0f0-727b1a2382c6.dat>
datafile is stored at above location
k
need to pull the data file back to local, and then provide it the local fs path to meta_tool.