I am not getting correct data after upsert.
2nd r...
# troubleshootingp
Piyush Chauhan
08/24/2022, 5:03 AMI am not getting correct data after upsert.
2nd row was already there I wanted to update it. But when I sent a new kafka message with same
idexception_countupdated_atm
Mayank
08/24/2022, 5:11 AM
Which version? There have been some recent bug fixes that will be available in 0.11.
p
Piyush Chauhan
08/24/2022, 5:50 AMScreenshot from 2022-08-24 11-19-37.png
Piyush Chauhan
08/24/2022, 5:51 AM@Mayank should we move to upgrading to 0.11?
m
Mayank
08/24/2022, 1:06 PM
What’s your upsert config
p
Piyush Chauhan
08/25/2022, 10:56 AM Copy code
{
"mode": "PARTIAL",
"partialUpsertStrategies": {
"exception_count": "OVERWRITE",
},
"hashFunction": "NONE"
}m
Mayank
08/25/2022, 2:29 PM
@Kartik Khare ^^
k
Kartik Khare
08/25/2022, 2:32 PMHi, can you also share the primary key from the schema?
Also, was the upsert enabled after inserting the first record?
Kartik Khare
08/25/2022, 2:32 PMBut when I query using the Java client I get on the 1 record i.e. older record whose updated at value 1660654106.p
Piyush Chauhan
08/26/2022, 11:14 AM1. id is the primary key in the schema.
2. Upsert was enabled before inserting both the records.
3. In the cluster manager I can see 3 servers
k
Kartik Khare
08/26/2022, 11:17 AMok, what I believe that the upsert state in multiple servers is different for some reason.
Can you try running the query multiple times in pinot UI and check if you always get same results
you can also add $hostName to the queries to check which server the records are coming from
p
Piyush Chauhan
08/26/2022, 11:31 AMI ran query multiple times. Getting same result
Piyush Chauhan
08/26/2022, 11:32 AMcan you share the query syntax(any doc would be the best) of adding hostname in the query?
k
Kartik Khare
08/26/2022, 11:34 AMOk. Can you tell me how are you querying using the java client and does it also return the same result?
syntax is
select id, reference_number, updated_at, $hostName, $segmentName, $docId from table
p
Piyush Chauhan
08/29/2022, 4:46 AMThis is the query result.
Piyush Chauhan
08/29/2022, 4:46 AM Copy code
private ResultSet runQuery(String query) {
Request pinotClientRequest = new Request("sql", query);
try {
ResultSetGroup pinotResultSetGroup = pinotConnection.execute(pinotClientRequest);
return pinotResultSetGroup.getResultSet(0);
} catch (PinotClientException e) {
log.error("Query Error, Query:{}", query, e);
throw GenericExceptionFactory.getGenericException(VisibilityErrorCode.DB_ERROR,
"DB Query Failed");
}
}Piyush Chauhan
08/29/2022, 4:46 AMAbove is the Java Code snippet that we are using.
@Kartik Khare
Piyush Chauhan
08/29/2022, 5:10 AMJava Client is only returning old record.
k
Kartik Khare
08/29/2022, 6:58 AMNo, I meant the creating connection part as well as
querys
Sid
09/01/2023, 8:05 AM@Kartik Khare this currently happens to the pinot setup I have. server-1 shows different result compared to server 3.
the segments are currently in consuming state.