@Rocky we're evaluating a new database for our platform and StarRocks is really appealing, but we're a little torn between Timeplus and StarRocks. A big benefit of TP for us is query streaming (for powering automation services in our product) - any recommendations on how we'd tackle this?

That's a great question. While both are excellent for real-time data, they approach "streaming" from different directions. Timeplus is a dedicated stream processor (push-based), whereas StarRocks is a high-performance OLAP database (pull-based, but optimized for sub-second freshness). To achieve "query streaming" or power automation services with StarRocks, we typically recommend three strategies depending on your latency needs: 1. The Flink-StarRocks "Golden Combo" This is the most common industry architecture for your specific use case. * How it works: You use Apache Flink as your stream processing engine to handle the "continuous queries" and automation triggers (e.g., if value > X, call webhook). * The Benefit: Flink handles the complex stateful streaming logic (windows, watermarks), and then sinks the processed data into StarRocks for high-speed user-facing analytics and historical correlation. This gives you the "push" for automation (via Flink) and the "pull" for analytics (via StarRocks) from the same data stream. 2. Asynchronous Materialized Views (Auto-Refresh) If your "streaming" requirement is more about keeping calculated results fresh for an application to pick up, StarRocks' Asynchronous Materialized Views (MVs) are built for this. * How it works: You can define an MV with

REFRESH ASYNC

. StarRocks can automatically trigger a refresh whenever the base table data changes. * Automation Strategy: Since the MV maintains the "latest state" incrementally, your automation service can query this MV via the HTTP SQL API or a standard JDBC connection. Because StarRocks is built for massive concurrency, your service can "poll" this fresh state at very high frequencies (sub-second) without impacting system performance. 3. StarRocks as the High-Concurrency Serving Layer In many "automation" scenarios, the bottleneck isn't the stream, but the ability to join real-time events with historical context (e.g., "Is this event's value 20% higher than this user's 30-day average?"). * The StarRocks Advantage: StarRocks excels at real-time joins. You can ingest millions of events per second via Routine Load (from Kafka) and have your automation service run complex SQL queries against that live data. * Recommendation: Instead of a long-running "continuous query," your service can use a "triggered pull" approach. When an event arrives in your message bus, the automation service queries StarRocks to get the full context (live event + history) and makes a decision in milliseconds. Summary: Which to choose? * Choose Timeplus if your primary requirement is a push-based event-trigger system with minimal interest in long-term historical analytics or complex multi-table joins. * Choose StarRocks if you need a unified platform where the same data powering your automation is also powering your real-time dashboards and ad-hoc analytics. If you are already leaning toward StarRocks for its performance, I'd suggest looking at the *Flink Connector*—it’s the standard way StarRocks users bridge the gap between "stream processing" and "analytical serving." References • sql-reference/http_sql_api.md • administration/http_interface.md • release_notes/release-3.3.md • administration/management/monitoring/Monitor_and_Alert.md • using_starrocks/Colocate_join.md

Can StarRocks ingest via HTTP?

@Derry Holt - make sure to tag @Rocky for an answer. Check out this DeepWiki page https://deepwiki.com/search/can-stock-rocks-ingest-via-htt_42a2e6f5-0c5e-4908-8761-432fe0ece11a?mode=fast Yes, StarRocks can ingest data via HTTP through its Stream Load feature, which provides an HTTP-based synchronous loading method for importing data from local file systems or streaming data sources [1](#0-0) . ## HTTP Ingestion Methods ### Stream Load Stream Load is the primary HTTP-based ingestion method in StarRocks. It uses HTTP PUT requests to load data and supports CSV and JSON formats [2](#0-1) . Basic syntax:

bash
curl --location-trusted -u <username>:<password> -XPUT <url>

URL format:

http://<fe_host>:<fe_http_port>/api/<database_name>/<table_name>/_stream_load

[3](#0-2) ### HTTP Interface Endpoints StarRocks provides HTTP interfaces on both FE and BE nodes for Stream Load operations [4](#0-3) : • FE endpoint:

PUT /api/{db}/{table}/_stream_load

• BE endpoint:

PUT /api/{db}/{table}/_stream_load

## Key Features • Synchronous execution: Jobs run synchronously and return results upon completion [5](#0-4) • Compression support: Since v3.2.7, supports GZIP, BZIP2, LZ4_FRAME, and ZSTD compression for JSON data [6](#0-5) • Multiple data formats: Supports CSV and JSON file formats [7](#0-6) • Data transformation: Supports data transformation during loading and UPSERT/DELETE operations [8](#0-7) ## Notes • Stream Load requires INSERT privileges on target tables [9](#0-8) • Materialized views are automatically updated when data is loaded via Stream Load [10](#0-9) • HTTP chunked transfer encoding is recommended for better performance [11](#0-10) ### Citations File: docs/en/sql-reference/sql-statements/loading_unloading/STREAM_LOAD.md (L7-10)

markdown
# STREAM LOAD

StarRocks provides the loading method HTTP-based STREAM LOAD to help you load data from a local file system or a streaming data source. After you submit a load job, StarRocks synchronously runs the job, and returns the result of the job after the job finishes. You can determine whether the job is successful based on the job result. For information about the application scenarios, limits, principles, and supported data file formats of Stream Load, see [Loading from a local file system via Stream Load](../../../loading/StreamLoad.md#loading-from-a-local-file-system-via-stream-load).

File: docs/en/sql-reference/sql-statements/loading_unloading/STREAM_LOAD.md (L12-14)

markdown

Since v3.2.7, Stream Load supports compressing JSON data during transmission, reducing network bandwidth overhead. Users can specify different compression algorithms using parameters `compression` and `Content-Encoding`. Supported compression algorithms including GZIP, BZIP2, LZ4_FRAME, and ZSTD. For more information, see [data_desc](#data_desc).

File: docs/en/sql-reference/sql-statements/loading_unloading/STREAM_LOAD.md (L18-20)

markdown
>
> - After you load data into a StarRocks table by using Stream Load, the data of the materialized views that are created on that table is also updated.
> - You can load data into StarRocks tables only as a user who has the INSERT privilege on those StarRocks tables. If you do not have the INSERT privilege, follow the instructions provided in [GRANT](../account-management/GRANT.md) to grant the INSERT privilege to the user that you use to connect to your StarRocks cluster.

File: docs/en/sql-reference/sql-statements/loading_unloading/STREAM_LOAD.md (L33-34)

markdown

• You can use chunked transfer encoding, as demonstrated in this topic. If you do not choose chunked transfer encoding, you must input a `Content-Length` header field to indicate the length of content to be transferred, thereby ensuring data integrity.

File: docs/en/sql-reference/sql-statements/loading_unloading/STREAM_LOAD.md (L58-60)

markdown
```Plain
http://<fe_host>:<fe_http_port>/api/<database_name>/<table_name>/_stream_load

**File:** docs/zh/loading/StreamLoad.md (L17-18)
```markdown
• Stream Load 支持 CSV 和 JSON 文件格式。如果您想从少量文件中导入数据，并且每个文件的大小不超过 10 GB，建议使用此方法。
• Broker Load 支持 Parquet、ORC、CSV 和 JSON 文件格式（从 v3.2.3 开始支持 JSON 文件格式）。如果您想从大量文件中导入数据，并且每个文件的大小超过 10 GB，或者文件存储在网络附加存储（NAS）设备中，建议使用此方法。**从 v2.5 开始支持使用 Broker Load 从本地文件系统导入数据。**

File: docs/zh/loading/StreamLoad.md (L25-26)

markdown
Stream Load 和 Broker Load 都支持在数据导入时进行数据转换，并支持在数据导入时通过 UPSERT 和 DELETE 操作进行数据更改。有关更多信息，请参见 [在导入时转换数据](../loading/Etl_in_loading.md) 和 [通过导入更改数据](../loading/Load_to_Primary_Key_tables.md)。

File: docs/zh/loading/StreamLoad.md (L37-40)

markdown
## 通过 Stream Load 从本地文件系统导入

Stream Load 是一种基于 HTTP PUT 的同步导入方法。在您提交导入作业后，StarRocks 会同步运行该作业，并在作业完成后返回结果。您可以根据作业结果判断作业是否成功。

File: docs/en/administration/http_interface.md (L11-14)

markdown
| Request Method      | Request Path                                                    | Description                                                                                                          |
|---------------------|-----------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------|
| PUT                 | `/api/{db}/{table}/_stream_load`                                | Stream Load operation, see [Stream Load](../loading/StreamLoad.md) for details.                                      |
| POST/PUT            | `/api/transaction/{txn_op}`                                     | Stream Load transaction interface, see [Stream Load Transaction Interface](../loading/Stream_Load_transaction_interface.md) for details. |

File: docs/zh/sql-reference/sql-statements/loading_unloading/STREAM_LOAD.md (L8-10)

markdown
## 功能

Stream Load 是一种基于 HTTP 协议的同步导入方式，支持将本地文件或数据流导入到 StarRocks 中。您提交导入作业以后，StarRocks 会同步地执行导入作业，并返回导入作业的结果信息。您可以通过返回的结果信息来判断导入作业是否成功。有关 Stream Load 的应用场景、使用限制、基本原理、以及支持的数据文件格式等信息，请参见[使用 Stream Load 从本地导入](../../../loading/StreamLoad.md#使用-stream-load-从本地导入)。

Hi there, I'm Rocky! The information you found on DeepWiki is spot-on. If you're building automation services and want to stick to a RESTful/HTTP-first architecture, StarRocks fits right in. While the other tool focuses on pushing query results out, StarRocks provides a high-performance HTTP-based data highway both for getting data in and getting results out. Here’s how you can leverage StarRocks' HTTP capabilities to tackle your automation requirements: 1. Ingestion: Stream Load (HTTP PUT) As your wiki link noted, Stream Load is the primary way to ingest data via HTTP. It is designed for exactly the kind of high-frequency, small-to-medium batch ingestion that powers product-led automation. * Synchronous Feedback: When your automation service sends an HTTP PUT request, StarRocks responds immediately with a JSON result. This is crucial for automation because your service knows instantly if the data was committed or if it needs to retry. * Efficiency: It’s significantly faster than standard SQL

INSERT

statements because it bypasses the SQL parsing layer and goes straight to the data nodes (BE). 2. Retrieval: HTTP SQL API (HTTP POST) To power the "query" part of your automation without managing complex JDBC connection pools, StarRocks (starting from v3.2) offers a native HTTP SQL API. * The Workflow: Your automation service can trigger a query via a standard

POST

request to

http://<fe_host>:<fe_http_port>/api/v1/sql

. * JSON Results: It returns results in a clean JSON format, making it easy for your microservices to parse "triggers" (e.g., Is the inventory count < 10?) and take action. 3. "Query Streaming" Workaround: The Micro-Polling Strategy Since StarRocks doesn't "push" query results to a client in the same way a dedicated stream processor might, automation developers typically use a Micro-Polling pattern with Asynchronous Materialized Views (MVs): * Step A: Define an MV that calculates your automation triggers (e.g., identifying users who haven't logged in for 3 days). Set it to

REFRESH ASYNC

so StarRocks keeps it fresh as data arrives via Stream Load. * Step B: Your automation service polls this MV via the HTTP SQL API every second (or even sub-second). * Why this works: Because StarRocks is built for massive concurrency and the MV has pre-calculated the result, these polls are incredibly "cheap" and return in milliseconds, effectively giving you a near-real-time "stream" of automation triggers. Summary for your Platform * Ingest: Use

HTTP PUT

(Stream Load) to feed your events into StarRocks. * Process: Use

Asynchronous Materialized Views

to handle the heavy lifting and logic in the background. * Act: Use the

HTTP SQL API

to pull triggers into your automation service. This setup gives you the scalability of a world-class OLAP database while keeping your service-to-service communication entirely over HTTP. References • loading/StreamLoad.md • sql-reference/sql-statements/loading_unloading/STREAM_LOAD.md • sql-reference/http_sql_api.md • using_starrocks/async_mv/Materialized_view.md • administration/http_interface.md References • integrations/streaming/rising_wave.md • data_source/External_table.md • faq/Others.md • unloading/Export.md • sql-reference/http_sql_api.md

Ah, thanks @Tony Wasson!