@Rocky I am running starrocks 4.0.10 on k8s in shared mode. I set

enable_json_flat=false

in the cluster. I ingested data from parquet files into tables where most data in in a column called

source

of type

json

Field    |Type          |Null|Key
---------+--------------+----+-----
id       |varchar(65533)|NO  |true
timestamp|bigint        |NO  |true
_index   |varchar(65533)|NO  |false
score    |double        |YES |false
source   |json          |YES |false

When running a simple query such as:

SELECT get_json_string(source, 'config_id')
  FROM source_containers
  limit 10;

I get he following errors in the CN nodes:

W20260517 13:39:57.025548 139870366082624 lake_connector.cpp:506] failed to find column in schema: source
W20260517 13:39:57.025595 139870366082624 lake_connector.cpp:590] failed to find column in schema: source
W20260517 13:39:57.025575 139870292653632 lake_connector.cpp:506] failed to find column in schema: source
W20260517 13:39:57.025634 139870292653632 lake_connector.cpp:590] failed to find column in schema: source

Furthermore, the queries are very slow

It appears you are encountering an issue specific to StarRocks 4.0.x in Shared-Data (Lake) mode when the new Flat JSON V2 engine is partially disabled. The error

failed to find column in schema: source

in

lake_connector.cpp

typically indicates a mismatch between the query plan (which expects a standard JSON column) and the physical storage layer's schema mapping. In StarRocks 4.0, JSON handling was re-architected to use Flat JSON by default. 1. Why this is happening In version 4.0, the "Flat JSON" feature is no longer just an optimization but is integrated into the core execution path for JSON. * The Error: By setting

enable_json_flat=false

, you are forcing the engine to treat the

source

column as a traditional binary JSON blob. However, the

lake_connector

(which handles shared-storage reads) may be attempting to use the V2 execution paths which expect a specific schema structure (either flattened sub-columns or a different metadata mapping) that isn't being generated or recognized correctly with that flag off. * Performance: Queries are slow because disabling Flat JSON in v4.0 bypasses the SIMD-optimized vectorized execution and "Zone Map" pruning that usually makes JSON queries 3–15x faster in this version. 2. Recommended Solution Unless you have a specific requirement to disable flattening, you should re-enable Flat JSON. This is the native way StarRocks 4.0 handles JSON data. Step 1: Enable Flat JSON at the cluster/session level

sql
-- Enable the BE to process flattened JSON
ADMIN SET FRONTEND CONFIG ("enable_json_flat" = "true"); -- Ensure FE is aware
-- In StarRocks 4.0, this is usually a BE config, ensure it's not set to false in be.conf

Step 2: Ensure Optimizer V2 Rewrites are enabled These variables are usually

true

by default in 4.0, but check them to ensure the query is optimized:

sql
SET GLOBAL cbo_json_v2_rewrite = true;
SET GLOBAL cbo_json_v2_dict_opt = true;

Step 3: Table-level configuration In v4.0, you can explicitly enable flattening for your specific table to ensure the

lake_connector

handles the schema correctly:

sql
ALTER TABLE source_containers SET ("flat_json.enable" = "true");

Note: After enabling this, new data will be flattened immediately. Existing data will be flattened during background compaction or if you trigger a manual compaction. 3. If you MUST keep

enable_json_flat=false

If your use case requires disabling flattening (e.g., highly unpredictable schemas that cause "schema explosion"), the "failed to find column" error in

lake_connector

is likely a bug in the fallback path for shared-data mode in version 4.0.10. Workaround: Try setting

enable_vectorized_json_load = false

or checking if there are hidden columns being queried. However, the most stable path in 4.0 is to use the default settings. Summary 1. Remove

enable_json_flat=false

from your configuration (let it default to

true

). 2. Run

OPTIMIZE TABLE source_containers

(or wait for compaction) to transform the physical storage into the format the 4.0

lake_connector

expects. 3. This should resolve the "failed to find column" error and significantly improve query performance. If the error persists after enabling Flat JSON, it may indicate that the metadata in your StarOS/Shared-storage layer has become desynced. In that case, please check the FE logs for

Master

node metadata errors. References • using_starrocks/Flat_json.md