https://linen.dev logo
Join Slack
Powered by
If I'm getting errors like `2026-05-06T15:09:29.53...
# general
d
If I'm getting errors like 
2026-05-06T15:09:29.532431Z ERROR on_query{protocol="mysql"}: servers::mysql::writer: Failed to handle mysql query, code: EngineExecuteQuery, error: Too many files to read concurrently: 1528, max allowed: 384, db: public
, would running manual compaction help?
e
Yes. You can also adjust the 
max_concurrent_scan_files
option. • https://github.com/GreptimeTeam/greptimedb/blob/main/config/datanode.example.toml#L525-L526 • https://github.com/GreptimeTeam/greptimedb/blob/main/config/standalone.example.toml#L26-L27 I think the default restriction may be too small. Could you share the vCPU number of your machine? It’d be helpful if you provide the result of `select * from information_schema.ssts_manifest`; We can inspect the files via the SQL. If you have many table, you can also specify the table id.
d
I had just found the max_concurrent_scan_files setting. 🙂 I did try compaction, but I think I might have targeted the wrong table, so it didn't seem to make a difference. I'm running the ssts manifest query now.
I have 8 vcpu. Host is running vmware esxi 7 with Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz cpus.
The ssts_manifest query is still spinning, but it's loaded up rows. 114433 of them.... Firefox complained about that for a bit....
Does the spinning icon in the dashboard's "Run query" button ever just get stuck spinning? I'm not seeing any load on my host, and the rows are loaded, so is it actually finished?
e
I guess so. You can also export the result as CSV. The drop down list of the “Run query” button provide an export button.
d
Ok, the csv exported file is 33mb. What info do you need out of it?
e
The average files num, total rows and file size per day for each table. Actually you can use SQL to compute that. If you have more than one table, better to query the table id from 
information_schema.tables
. Besides enlarging file concurrency, maybe we can adjust the swcs compaction time window to a smaller one. The default time window is 1 day. Seems there are too many small files. https://docs.greptime.com/user-guide/deployments-administration/manage-data/compaction/#strict-window-compaction-strategy-swcs-and-manual-compaction You can also upload the file so we can analyze that later. Maybe we can improve our compaction strategy and concurrent file limit.
d
Would that info be in the metrics if I had the metric scraped? I haven't yet set up my monitoring instance, so I don't have the metrics scraped yet. But that is something I want to do soon. So if it would be an easier way to analyze this, I can push it up.
Anyway, now I'm wiating for FF to close the initial query tab.... Then I'll see about getting the data you asked for. I'm guessing most of it is in the tables table?
From the csv file I pulled this info out:
Copy code
Total rows: 38766659639
Min time: 2026-02-09 23:10:53.587109
Max time: 2026-05-06 15:44:01.368976
Days covered: 85
Average rows per day: 456078348
And if it's a row per file, we have 114,431 files.
e
Are you importing the data between 02-09 and 05-06 to the database? I guess it may take some time to merge the small files for each day. You can filter by min_ts to reduce the result set size. For example, if we only take the result in latest 6 days.
Copy code
select * from information_schema.ssts_manifest where min_ts > '2026-05-01 00:00:00';
d
Smaller time frame queries do work. I just have some data I need to be able to use large time frames on.
So, I'm applying my settings wrong. I just got an 
laneintel-db  | 5: Context("parallelize_scan", External(0: Too many files to read concurrently: 433, max allowed: 384, at src/mito2/src/read/series_scan.rs:311:14)))
error when I should have the limit set o 1800.
Copy code
default_timezone = "America/Los_Angeles"
max_in_flight_write_bytes = "8GB"
## The maximum current queries allowed to be executed. Zero means unlimited.
## NOTE: This setting affects scan_memory_limit's privileged tier allocation.
## When set, 70% of queries get privileged memory access (full scan_memory_limit).
## The remaining 30% get standard tier access (70% of scan_memory_limit).
max_concurrent_queries = 0
## Enable telemetry to collect anonymous usage data. Enabled by default.
enable_telemetry = false
## The data storage options.
[storage]
## The working home directory.
data_home = "/greptimedb_data"
## The storage type used to store the data.
## - `File`: the data is stored in the local file system.
## - `S3`: the data is stored in the S3 object storage.
## - `Gcs`: the data is stored in the Google Cloud Storage.
## - `Azblob`: the data is stored in the Azure Blob Storage.
## - `Oss`: the data is stored in the Aliyun OSS.
type = "File"
[logging]
dir = "/greptimedb_data/logs"
log_format = "json"
[[region_engine]]
[region_engine.mito]
max_concurrent_scan_files = 1800
That's my /etc/greptime/custom.toml file. Docker's inspect says this is my ENV for the service:
Copy code
GREPTIMEDB_STANDALONE__LOGGING__LEVEL=info",
"PATH=/greptime/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"TARGET_BIN=greptime",
"MALLOC_CONF=prof:true,prof_active:false"
Where am I messing up?
I figured it out. Wasn't actually loading the config file properly and there were issues with formatting.
e
Smaller time frame queries do work. I just have some data I need to be able to use large time frames on.
Oh, I meant that you can adjust the time range of the 
ssts_manifest
query if there are too many rows. Maybe you have already get the full result set in the CSV. Could you share the CSV? I’d like to check why there are so many files haven’t been merged. If you don’t mind, you can also tar the db logs and share it to me. Maybe you can DM me if you don’t want to share them in the channel.
d
DM'd you a zip with the results from wednesday.
Ok, so, let's check my expectations. Would you expect a dashboard in grafana to load 30 days worth of data from greptime's mysql endpoint? Where I have 17 visualizations that each run their own query? This would be with a working version of the configuration I posted above, and the GF_DATAPROXY_TIMEOUT var set to 300 for grafana. Oh, and my vm has 8vcpu, and 24gb of ram which the greptime docker service limited to 20gb and 4.0 cpu. (Thought I had increased that to 8 when I upped my vcpu count...) I so see all 8 cores get hit when running queries, so I need to go refresh my memory how that setting works....
e
DM’d you a zip with the results from wednesday.
Thanks. 30 days may be a large time frame to query and may take some time if you run 17 queries together. You can run each query to check the response time and resource usage. If the response time is high, we can further enlarge the proxy timeout.
13 Views
Open in Slack
PreviousNext
Powered by