Hi Team,
Could you please share a sample logical t...
# generals
Shubham Kumar
10/22/2025, 4:19 AMHi Team,
Could you please share a sample logical table configuration?
I’m currently using the configuration below for my logical table, but it only fetches data from the realtime table the offline table data is not being fetched.
Copy code
{
"tableName": "logicalTable",
"physicalTableConfigMap": {
"user_stream_REALTIME": {},
"user_batch_OFFLINE": {}
},
"refOfflineTableName": "user_batch_OFFLINE",
"refRealtimeTableName": "user_stream_REALTIME",
"brokerTenant": "DefaultTenant",
"timeBoundaryConfig": {
"boundaryStrategy": "min",
"parameters": {
"function": "min"
}
}
}Shubham Kumar
10/22/2025, 4:21 AMIf I create the logical table using only the offline table, it will be able to fetch data from that table
Shubham Kumar
10/22/2025, 5:18 AM@Abhishek Bafna @Mayank could you please help me here?
m
Mayank
10/22/2025, 5:26 AM
Why do you need logical table?
s
Shubham Kumar
10/22/2025, 6:00 AMI’m doing a POC for my use case
m
Mayank
10/22/2025, 1:25 PM
I meant what requirement in your use case makes you need logical tables?
s
Shubham Kumar
10/22/2025, 3:45 PMwe need logical tables so that we can query a single table while Pinot automatically fetches data from both the REALTIME and OFFLINE tables, performs the union internally, and returns the combined results. This simplifies our queries and gives us a unified view of both streaming and batch data.
Shubham Kumar
10/22/2025, 3:48 PMUpdated config
Copy code
{
"tableName": "logicalTable",
"physicalTableConfigMap": {
"user_stream_REALTIME": {},
"user_batch_OFFLINE": {}
},
"refOfflineTableName": "user_batch_OFFLINE",
"refRealtimeTableName": "user_stream_REALTIME",
"brokerTenant": "DefaultTenant",
"timeBoundaryConfig": {
"boundaryStrategy": "min",
"parameters": {
"includedTables": [
"user_batch_OFFLINE"
]
}
}
}m
Mayank
10/22/2025, 5:05 PM
That is not logical table, that is hybrid table (I suspected that is what you need and not logical table, hence I asked the question)
s
Shubham Kumar
10/22/2025, 5:17 PMthe recent release (1.4.0) introduces the concept of a logical table, so I’d appreciate your help with it
m
Mayank
10/22/2025, 5:27 PM
The reason for designing logical tables was for a scale where you have multi-million segments within a single table, and you start to hit some of the limits due to that. What's your data size, how many segments do you have?
➕ 1
s
Shubham Kumar
10/22/2025, 5:49 PM7000 segments and 3 TB of data for 10 days of ingestion
m
Mayank
10/22/2025, 5:49 PM
Yep, you don't need logical tables.
s
Shubham Kumar
10/22/2025, 5:51 PMBut data and segments will grow based on the retention period
m
Mayank
10/22/2025, 5:54 PM
What I am saying is unless you go into millions of segments range, you don't need logical table. You can either stay using real-time table, or use the hybrid table, by periodically offloading data to offline table using managed offline flow.
➕ 1