@Rocky what config changes to make to allow more than 100MB in json data for streaming load. The client cannot send ignore_json_size header.

To allow JSON Stream Loads larger than 100MB without using the

ignore_json_size

header, you need to modify the BE (Backend) configuration. The specific parameter controlling this limit is

streaming_load_max_batch_size_mb

. 1. Update the BE Configuration File To make the change permanent, modify the

be.conf

file on every Backend node: 1. Open

be.conf

(typically located in the

conf/

directory of your BE installation). 2. Add or update the following line (for example, to set it to 500MB):

properties
   streaming_load_max_batch_size_mb = 500

3. Restart the BE service for the changes to take effect. ────────── 2. Update Dynamically (No Restart Required) Since this parameter is mutable, you can apply the change immediately without restarting the BE by using the

update_config

API. Run this command for each BE node in your cluster:

bash
curl -X POST http://<be_host>:<be_http_port>/api/update_config?streaming_load_max_batch_size_mb=500

_Replace

<be_host>

with the BE IP/hostname and

<be_http_port>

with the

be_http_port

(default is

8040

)._ Summary of Relevant Parameters *

streaming_load_max_batch_size_mb

: Specifically limits the size of JSON format data for Stream Loads. The default is

100

(MB). *

streaming_load_max_mb

: Limits the overall file size for all Stream Loads (CSV, Parquet, etc.). The default is

102400

(100GB) in v3.0+. If your total JSON file is extremely large (e.g., >10GB), you may need to check this as well, though usually, it is already high enough. Note: When increasing this limit, ensure your BE nodes have sufficient memory, as parsing very large JSON objects can be memory-intensive. Also, ensure the

stream_load_default_timeout_second

(an FE config) is long enough to accommodate the transfer of larger files. References • administration/management/BE_configuration.md • faq/loading/Stream_load_faq.md • sql-reference/sql-statements/loading_unloading/STREAM_LOAD.md • loading/StreamLoad.md • sql-reference/sql-statements/loading_unloading/BROKER_LOAD.md • administration/management/FE_configuration.md