Koorous Vargha
01/10/2024, 12:20 AMcom.linkedin.venice.endToEnd.DaVinciClientMemoryLimitTest > testDaVinciMemoryLimitShouldFailLargeDataPush
. Is there a specific part in the code where the in memory db should be cleared?Koorous Vargha
01/10/2024, 3:19 AMKoorous Vargha
01/11/2024, 11:47 PMElijah Grimaldi
01/16/2024, 5:30 PMZac Policzer
01/17/2024, 5:39 AMZac Policzer
02/02/2024, 9:40 PMZac Policzer
02/02/2024, 9:44 PMFelix GV
02/07/2024, 4:21 PMFelix GV
02/07/2024, 4:47 PMKoorous Vargha
02/09/2024, 5:04 AMElijah Grimaldi
02/09/2024, 5:12 PMKoorous Vargha
02/21/2024, 6:10 PMreadyToServe
parameter
2. Get rid of TransformedRecord
3. Have put
return the transformed value instead of TransformedRecord
4. Remove versionNumber
as function parameters and instead have the store version be a part of the class, and for each store version there will be a different instance of the transformer. We also need to specify in the Java doc that it needs to be thread safe.
5. Mention in the interface that schema evolution is possible.
6. Need to handle a case where previous store version is retired. Possibly handled by reingesting from Kafka. Does this need to already be defined in the interface or do we depend on the user to define the behavior?
7. put
should throw an exception in the case of an error for easier debugging.
Please let me know if I missed anything.Koorous Vargha
02/22/2024, 11:46 PMversionNumber
in the transformer shouldn't we change it from an interface to an abstract class?Koorous Vargha
02/22/2024, 11:47 PMTransformedRecord
, what return type should delete
have as it's currently returning TransformedRecord
. Is void
appropriate?Elijah Grimaldi
02/23/2024, 6:41 PMKoorous Vargha
02/24/2024, 12:31 AMversionNumber
is persisted in the class in my latest commit?Koorous Vargha
03/09/2024, 9:40 PMDaVinciRecordTransformer
for each new version as discussed in the call.
But I'm running into an issue of how to get the current store version before the backend is initialized here?Zac Policzer
03/09/2024, 10:02 PMZac Policzer
03/09/2024, 10:12 PMKoorous Vargha
03/20/2024, 5:14 PMKoorous Vargha
03/21/2024, 3:26 AMprocessKafkaDataMessage
in StoreIngestionTask
. Here's my test. What is the correct way to test processKafkaDataMessage
? Thanks.Zac Policzer
03/21/2024, 6:58 PMKoorous Vargha
03/23/2024, 4:32 AMKoorous Vargha
03/27/2024, 6:41 PMKoorous Vargha
03/27/2024, 6:42 PMZac Policzer
03/27/2024, 9:32 PMKoorous Vargha
03/27/2024, 9:33 PMFelix GV
03/27/2024, 9:46 PMZac Policzer
04/20/2024, 12:06 AMKoorous Vargha
04/24/2024, 4:52 PM