http://starrocks.io logo
Join Slack
Powered by
<@U0A71G31CDV> I am getting the following error wh...
# questions-and-troubleshooting
d
@Rocky I am getting the following error when running starrocks using test containers on the CI pipeline.
Copy code
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table replication num should be less than or equal to the number of available backends. You can change this default by setting the replication_num table properties. Current available backends: [], backends without enough disk space: [10001], table=test_table, replication_num=1, default_replication_num=1
✅ 1
r
This error occurs because StarRocks has a default "high water mark" for disk space. By default, it requires at least 100 GB of free space on each Backend (BE) node. Most CI environments (like GitHub Actions) provide significantly less disk space, causing the FE to mark the BE as "unavailable" immediately. To fix this in your Testcontainers setup, you need to lower the disk reserve thresholds in both the Frontend (FE) and Backend (BE) configurations. 1. Fix via Environment Variables If you are using the official StarRocks Docker images, you can typically pass these configurations as environment variables. Set the following variables for your container: * For the FE: * 
FE_CONF_storage_usage_hard_limit_reserve_bytes = 104857600
(Sets reserve to 100MB) * 
FE_CONF_storage_usage_soft_limit_reserve_bytes = 104857600
* For the BE: * 
BE_CONF_storage_flood_stage_left_capacity_bytes = 104857600
(Sets reserve to 100MB) _(Note: Depending on your specific image version, the prefix might be 
STARROCKS_FE_
or similar. Check your image documentation if the 
FE_CONF_
prefix does not work.)_ 2. Fix via SQL (Dynamic Configuration) If the cluster starts but fails at the table creation step, you can try to update these settings dynamically via a SQL client before running your tests:
Copy code
sql
-- Lower the FE's hard and soft reserve limits
ADMIN SET FRONTEND CONFIG ("storage_usage_hard_limit_reserve_bytes" = "104857600");
ADMIN SET FRONTEND CONFIG ("storage_usage_soft_limit_reserve_bytes" = "104857600");

-- Lower the BE's flood stage limit (via the BE HTTP API or be.conf)
-- This is often easier to set via environment variables as shown above.
Why this happens * FE Config: 
storage_usage_hard_limit_reserve_bytes
(Default: 100 GB). If the BE reports less available space than this, the FE will not allow new tablets (tables) to be created on it. * BE Config: 
storage_flood_stage_left_capacity_bytes
(Default: 100 GB). If the actual disk space is below this, the BE will reject load and write operations. Recommendation for CI In resource-constrained environments like CI pipelines, it is best practice to set these values to a small number (e.g., 100 MB or 1 GB) to prevent "insufficient disk space" false positives. You should also ensure the 
replication_num
is set to 
1
if you are only running a single BE container. References • administration/Meta_recovery.md • developers/build-starrocks/Build_in_docker.md • faq/operation_maintenance_faq.md • faq/Deploy_faq.md • administration/management/monitoring/alert.md • developers/development-environment/dev-env-setup.md • developers/mac-compile-run-test.md
d
image is 4.0 @Rocky what is the env variable to set?
r
For StarRocks 4.0 Docker images, the recommended environment variables to resolve the "insufficient disk space" error in CI environments are prefixed with 
FE_CONF_
 and 
BE_CONF_
. These variables allow you to override the default 100 GB disk reserve limit, which is much larger than the space typically available in CI runners (like GitHub Actions). Environment Variables to Set Set these in your Testcontainers configuration: For the Frontend (FE): * `FE_CONF_storage_usage_hard_limit_reserve_bytes`: 
104857600
(100 MB) * `FE_CONF_storage_usage_soft_limit_reserve_bytes`: 
104857600
(100 MB) For the Backend (BE): * `BE_CONF_storage_flood_stage_left_capacity_bytes`: 
104857600
(100 MB) Why this works The Docker entrypoint script in version 4.0 images scans for environment variables starting with 
FE_CONF_
or 
BE_CONF_
. It strips the prefix, converts the remaining string to lowercase, and appends it to the 
fe.conf
or 
be.conf
file automatically. Additional CI Tips 1. Single Replica: Ensure your table creation DDL includes 
PROPERTIES ("replication_num" = "1")
. 2. Wait Strategy: CI environments can be slow. Ensure your Testcontainers "wait strategy" looks for the FE MySQL port (
9030
) and verify the BE is "Alive" by running 
SHOW BACKENDS
before starting tests. 3. Root Password: If you have changed the root password in your image, you may need to set 
MYSQL_PWD
for the BE to register itself successfully with the FE in version 4.0.
d
I'm using the 
starrocks/allin1-ubuntu:4.0.0
does that change anything? @Rocky
r
Yes, using the 
starrocks/allin1-ubuntu:4.0.0
image actually makes things simpler, but you still use the same environment variable pattern. Because this "all-in-one" image runs both the Frontend (FE) and Backend (BE) in a single container, it is designed to recognize the prefixes to route configuration to the correct service. For version 4.0.0, set these environment variables in your Testcontainers configuration: 1. Environment Variables (Recommended) Set these to lower the disk reserve from the default 100 GB to 100 MB: * For the FE component: * 
FE_CONF_storage_usage_hard_limit_reserve_bytes=104857600
* 
FE_CONF_storage_usage_soft_limit_reserve_bytes=104857600
* For the BE component: * 
BE_CONF_storage_flood_stage_left_capacity_bytes=104857600
The entrypoint script in the 
allin1
image will automatically strip the prefixes and append these settings to the internal 
/data/deploy/starrocks/fe/conf/fe.conf
and 
/data/deploy/starrocks/be/conf/be.conf
files. 2. SQL Alternative (Frontend only) If you cannot easily change environment variables in your environment, you can run this SQL command immediately after connecting to StarRocks to fix the FE side of the error:
Copy code
sql
ADMIN SET FRONTEND CONFIG ("storage_usage_hard_limit_reserve_bytes" = "104857600");
Why this is happening in your CI * The Problem: StarRocks is built for production environments where it expects large disks. By default, it refuses to create tables if it sees less than 100 GB of free space. * The CI Reality: Most CI runners (like GitHub Actions) have total disk sizes of 20GB–30GB. Since 20GB < 100GB, the FE marks the BE as "unavailable" for new data, resulting in your error. * The Fix: Lowering these "reserve" limits to a small value (like 100MB) tells StarRocks it's okay to operate on the smaller disks found in CI. One final check for Testcontainers Ensure you have the following in your DDL for any table you create in CI, as the 
allin1
container only has one backend:
Copy code
sql
PROPERTIES ("replication_num" = "1")
_(Your error message shows you are already using 
replication_num=1
, so the disk space fix above is the final piece of the puzzle.)_
Open in Slack
PreviousNext
Powered by