1 new commit pushed to

<https://github.com/linkedin/venice/tree/main|main>

by haoxu07

<https://github.com/linkedin/venice/commit/2edcc80a4232dc47f9c24df0c47ded92170a7a75|2edcc80a>

- [vpj] Avoid using D2Client Fetching Latest KME Schema in VPJ. (#2072) linkedin/venice

#2130 [controller] integrate otel metrics for admin spark server Pull request opened by pthirun ## Problem Statement Current controller request metrics do not have any method to separate between url, which is required to supply controller SLO by command. ## Solution Migrate controller metrics to generic gauges. This will allow the use of OTel and the dimensions it supports. ### Code changes • Added new code behind a config. If so list the config names and their default values in the PR description. • Introduced new log lines. • Confirmed if logs need to be rate limited to avoid excessive logging. ### Concurrency-Specific Checks Both reviewer and PR author to verify • Code has no race conditions or thread safety issues. • Proper synchronization mechanisms (e.g.,

synchronized

,

RWLock

) are used where needed. • No blocking calls inside critical sections that could lead to deadlocks or performance degradation. • Verified thread-safe collections are used (e.g.,

ConcurrentHashMap

,

CopyOnWriteArrayList

). • Validated proper exception handling in multi-threaded code to avoid silent thread termination. ## How was this PR tested? Existing tests • New unit tests added. • New integration tests added. • Modified or extended existing tests. • Verified backward compatibility (if applicable). ## Does this PR introduce any user-facing or breaking changes? • No. You can skip the rest of this section. • Yes. Clearly explain the behavior change and its impact. linkedin/venice

#2131 [server] Improve logging in readGlobalRtDivState Pull request opened by lluwm ## Problem Statement Improve the logging in readGlobalRtDivState to contain the

key

string (thus

brokerUrl

) when exception happens. ### Code changes • Added new code behind a config. If so list the config names and their default values in the PR description. • Introduced new log lines. • Confirmed if logs need to be rate limited to avoid excessive logging. ### Concurrency-Specific Checks Both reviewer and PR author to verify • Code has no race conditions or thread safety issues. • Proper synchronization mechanisms (e.g.,

synchronized

,

RWLock

) are used where needed. • No blocking calls inside critical sections that could lead to deadlocks or performance degradation. • Verified thread-safe collections are used (e.g.,

ConcurrentHashMap

,

CopyOnWriteArrayList

). • Validated proper exception handling in multi-threaded code to avoid silent thread termination. ## How was this PR tested? • New unit tests added. • New integration tests added. • Modified or extended existing tests. • Verified backward compatibility (if applicable). ## Does this PR introduce any user-facing or breaking changes? • No. You can skip the rest of this section. • Yes. Clearly explain the behavior change and its impact. linkedin/venice

1 new commit pushed to

<https://github.com/linkedin/venice/tree/main|main>

by lluwm

<https://github.com/linkedin/venice/commit/dc1a9e8073747f0dea532a785a8a487eceba5c24|dc1a9e80>

- [server] Improve logging in readGlobalRtDivState (#2131) linkedin/venice

#2132 [dvc] Add heartbeat monitoring service for Da Vinci Pull request opened by sixpluszero ## [dvc] Add heartbeat monitoring service for Da Vinci This PR adds heartbeat monitoring support for Da Vinci. With this support, Da Vinci will be able to switch to measure time lag, instead of offset lag when performing ready-to-serve check. ## Solution ### Code changes • Added new code behind a config. If so list the config names and their default values in the PR description. • Introduced new log lines. • Confirmed if logs need to be rate limited to avoid excessive logging. ### Concurrency-Specific Checks Both reviewer and PR author to verify • Code has no race conditions or thread safety issues. • Proper synchronization mechanisms (e.g.,

synchronized

,

RWLock

) are used where needed. • No blocking calls inside critical sections that could lead to deadlocks or performance degradation. • Verified thread-safe collections are used (e.g.,

ConcurrentHashMap

,

CopyOnWriteArrayList

). • Validated proper exception handling in multi-threaded code to avoid silent thread termination. ## How was this PR tested? • New unit tests added. • New integration tests added. • Modified or extended existing tests. • Verified backward compatibility (if applicable). ## Does this PR introduce any user-facing or breaking changes? • No. You can skip the rest of this section. • Yes. Clearly explain the behavior change and its impact. linkedin/venice

#2133 [controller] Fix minor bugs in validation flow for DeferredVersionSwapService Pull request opened by misyel ## Problem Statement 1. Currently the validation flow uses the store config's target region as the region to validate. This is not correct as this field is not always set 2. getChildDataCenterControllerUrlMap is used to get the list of regions, but this field is not always set 3. We do not wait for dvc ingestion to complete before performing validation 4. When roll forward controller client call times out, it is possible that the child successfully rolled forward. This case needs to be handled in DeferredVersionSwapService as well so VPJ stops polling ## Solution 1. Use the version config's target region as the target region to validate for validation flow 2. Use getControllerClientMap() as this will use d2 map and url map 3. Check execution status before performing validation so that dvc ingestion is complete before performing validation 4. Add check for if all child regions are online then mark parent as online to stop vpj polling to account for roll forward controller client timeouts ### Code changes • Added new code behind a config. If so list the config names and their default values in the PR description. • Introduced new log lines. • Confirmed if logs need to be rate limited to avoid excessive logging. ### Concurrency-Specific Checks Both reviewer and PR author to verify • Code has no race conditions or thread safety issues. • Proper synchronization mechanisms (e.g.,

synchronized

,

RWLock

) are used where needed. • No blocking calls inside critical sections that could lead to deadlocks or performance degradation. • Verified thread-safe collections are used (e.g.,

ConcurrentHashMap

,

CopyOnWriteArrayList

). • Validated proper exception handling in multi-threaded code to avoid silent thread termination. ## How was this PR tested? • New unit tests added. • New integration tests added. • Modified or extended existing tests. • Verified backward compatibility (if applicable). Existing tests ## Does this PR introduce any user-facing or breaking changes? • No. You can skip the rest of this section. • Yes. Clearly explain the behavior change and its impact. linkedin/venice

1 new commit pushed to

<https://github.com/linkedin/venice/tree/main|main>

by gaojieliu

<https://github.com/linkedin/venice/commit/525348c9fc6ec2b33b1ba7adb16413a3dfd5e8ef|525348c9>

- [compat][da-vinci] Added key urn compression support in DaVinci (#2074) linkedin/venice

#2134 [dvc][test] Add support for Version Specific Da Vinci and refactor DaVinciBackendTest Pull request opened by kvargha ## Problem Statement 1. There isn't a way to subscribe to a specific store version on the Da Vinci Client. We need this to perform ETL on Venice Kafka Version Topics. Once version specific Da Vinci is added, we will build a CDC client on top of it using DVRT. 2. A test in

DaVinciBackendTest

, is leveraging a lot of reflection because it is difficult to test

DaVinciBackend

. ## Solution 1. Added support for version specific DaVinci. It will ignore version swaps, and not crash if the Kafka topic gets deleted on version retirement. However, on the next restart it will crash because the Venice store version has been deleted. 2. Refactored

DaVinciBackendTest

, so it's easier to inject dependencies, eliminating the need of reflection. ### Code changes • Added new code behind a config. If so list the config names and their default values in the PR description. • Introduced new log lines. • Confirmed if logs need to be rate limited to avoid excessive logging. ### Concurrency-Specific Checks Both reviewer and PR author to verify • Code has no race conditions or thread safety issues. • Proper synchronization mechanisms (e.g.,

synchronized

,

RWLock

) are used where needed. • No blocking calls inside critical sections that could lead to deadlocks or performance degradation. • Verified thread-safe collections are used (e.g.,

ConcurrentHashMap

,

CopyOnWriteArrayList

). • Validated proper exception handling in multi-threaded code to avoid silent thread termination. ## How was this PR tested? • New unit tests added. • New integration tests added. • Modified or extended existing tests. • Verified backward compatibility (if applicable). ## Does this PR introduce any user-facing or breaking changes? • No. You can skip the rest of this section. • Yes. Clearly explain the behavior change and its impact. linkedin/venice

1 new commit pushed to

<https://github.com/linkedin/venice/tree/main|main>

by sushantmane

<https://github.com/linkedin/venice/commit/0d783da75865307c47842c3cbe9d54db5fec6265|0d783da7>

- [test] Implement parallel shutdown for Venice test infrastructure to improve performance (#2119) linkedin/venice

#2136 [admin-tool] Disable Log Fix Pull request opened by KaiSernLim ## Problem Statement The

--disable-log

option seems to still leak some logs and I think it's because the

PubSubClientsFactory

is initialized before

LogConfigurator.disableLog()

on line

226

. ## Solution Move

PubSubClientsFactory

to be created after logging is disabled. ## How was this PR tested? CI ## Does this PR introduce any user-facing or breaking changes? • No. You can skip the rest of this section. linkedin/venice

#2137 [server][dvc][test] Delete deprecated PubSubPositionDeserializer::getPositionFromWireFormat API Pull request opened by sushantmane ## [server][dvc][test] Delete deprecated PubSubPositionDeserializer::getPositionFromWireFormat API Remove the deprecated static

PubSubPositionDeserializer::getPositionFromWireFormat

method to avoid accidental usage. This API only supported a limited set of default position types, which could lead to incorrect behavior. Callers should instead use a custom

PubSubPositionDeserializer

to handle all position types properly. ### Code changes • Added new code behind a config. If so list the config names and their default values in the PR description. • Introduced new log lines. • Confirmed if logs need to be rate limited to avoid excessive logging. ### Concurrency-Specific Checks Both reviewer and PR author to verify • Code has no race conditions or thread safety issues. • Proper synchronization mechanisms (e.g.,

synchronized

,

RWLock

) are used where needed. • No blocking calls inside critical sections that could lead to deadlocks or performance degradation. • Verified thread-safe collections are used (e.g.,

ConcurrentHashMap

,

CopyOnWriteArrayList

). • Validated proper exception handling in multi-threaded code to avoid silent thread termination. ## How was this PR tested? • New unit tests added. • New integration tests added. • Modified or extended existing tests. • Verified backward compatibility (if applicable). ## Does this PR introduce any user-facing or breaking changes? • No. You can skip the rest of this section. • Yes. Clearly explain the behavior change and its impact. linkedin/venice

#2138 [client] ClusterStats migration to OTel Pull request opened by lluwm ## Problem Statement OTel metrics migration for ClusterStats. ## Solution This PR migrates the following Tehuti metrics to OTel metrics: `blocked_instance_count`/`unhealthy_instance_count`/`overloaded_instance_count` ->

instance.error_count

version_update_failure

->

version.update_failure_count

current_version

->

version.current_number

We also adds new unit tests for code coverage. ### Code changes • Added new code behind a config. If so list the config names and their default values in the PR description. • Introduced new log lines. • Confirmed if logs need to be rate limited to avoid excessive logging. ### Concurrency-Specific Checks Both reviewer and PR author to verify • Code has no race conditions or thread safety issues. • Proper synchronization mechanisms (e.g.,

synchronized

,

RWLock

) are used where needed. • No blocking calls inside critical sections that could lead to deadlocks or performance degradation. • Verified thread-safe collections are used (e.g.,

ConcurrentHashMap

,

CopyOnWriteArrayList

). • Validated proper exception handling in multi-threaded code to avoid silent thread termination. ## How was this PR tested? • New unit tests added. • New integration tests added. • Modified or extended existing tests. • Verified backward compatibility (if applicable). ## Does this PR introduce any user-facing or breaking changes? • No. You can skip the rest of this section. • Yes. Clearly explain the behavior change and its impact. linkedin/venice

1 new commit pushed to

<https://github.com/linkedin/venice/tree/main|main>

by lluwm

<https://github.com/linkedin/venice/commit/32b85063a2fe797d33ae8e649ee6615840d5a029|32b85063>

- [client] Add Otel metrics for clients (#2095) linkedin/venice

1 new commit pushed to

<https://github.com/linkedin/venice/tree/main|main>

by sixpluszero

<https://github.com/linkedin/venice/commit/79c97797d1002babe018708a0d353b68054acea6|79c97797>

- [controller][client] Do not track stats for system store readers used in controller (#2099) linkedin/venice

#2139 [server][da-vinci-client] Add per consumer pool rate metrics Pull request opened by majisourav99 ## Problem Statement Currently adaptive throttler metrics only expose static throttler rate limit of the various consumer pool ingestions throttler which is not very useful as from the log we can infer the current throttler limit. ## Solution This PR replaces those metrics with actual ingestion rate of various pool. ### Code changes • Added new code behind a config. If so list the config names and their default values in the PR description. • Introduced new log lines. • Confirmed if logs need to be rate limited to avoid excessive logging. ### Concurrency-Specific Checks Both reviewer and PR author to verify • Code has no race conditions or thread safety issues. • Proper synchronization mechanisms (e.g.,

synchronized

,

RWLock

) are used where needed. • No blocking calls inside critical sections that could lead to deadlocks or performance degradation. • Verified thread-safe collections are used (e.g.,

ConcurrentHashMap

,

CopyOnWriteArrayList

). • Validated proper exception handling in multi-threaded code to avoid silent thread termination. ## How was this PR tested? • New unit tests added. • New integration tests added. • Modified or extended existing tests. • Verified backward compatibility (if applicable). ## Does this PR introduce any user-facing or breaking changes? • No. You can skip the rest of this section. • Yes. Clearly explain the behavior change and its impact. linkedin/venice

1 new commit pushed to

<https://github.com/linkedin/venice/tree/main|main>

by KaiSernLim

<https://github.com/linkedin/venice/commit/98a8fc0057a61b4049b115e7efea4236ec829502|98a8fc00>

- [admin-tool] Disable Log Fix (#2136) linkedin/venice

#2140 Add assets for Venice logos Pull request opened by nisargthakkar ## Problem Statement I had these Venice logo assets on my laptop. Adding them to the repo for more persistent storage ## Solution Add assets on the repo to prevent them from getting lost ### Code changes • Added new code behind a config. If so list the config names and their default values in the PR description. • Introduced new log lines. • Confirmed if logs need to be rate limited to avoid excessive logging. ### Concurrency-Specific Checks Both reviewer and PR author to verify • Code has no race conditions or thread safety issues. • Proper synchronization mechanisms (e.g.,

synchronized

,

RWLock

) are used where needed. • No blocking calls inside critical sections that could lead to deadlocks or performance degradation. • Verified thread-safe collections are used (e.g.,

ConcurrentHashMap

,

CopyOnWriteArrayList

). • Validated proper exception handling in multi-threaded code to avoid silent thread termination. ## How was this PR tested? • New unit tests added. • New integration tests added. • Modified or extended existing tests. • Verified backward compatibility (if applicable). ## Does this PR introduce any user-facing or breaking changes? • No. You can skip the rest of this section. • Yes. Clearly explain the behavior change and its impact. linkedin/venice

1 new commit pushed to

<https://github.com/linkedin/venice/tree/main|main>

by nisargthakkar

<https://github.com/linkedin/venice/commit/0b86bcc4797c164d9c20807ccddd61b34f6fc505|0b86bcc4>

- Add assets for Venice logos (#2140) linkedin/venice

#2142 [controller] Check cluster specific multi-region configs for batch push eligibility Pull request opened by huangminchn ## Problem Statement Currently controller is picking configs from a random cluster when checking push job eligibility. ## Solution This PR fixed the issue by specifying the right cluster name when retrieving configs. ### Code changes • Added new code behind a config. If so list the config names and their default values in the PR description. • Introduced new log lines. • Confirmed if logs need to be rate limited to avoid excessive logging. ### Concurrency-Specific Checks Both reviewer and PR author to verify • Code has no race conditions or thread safety issues. • Proper synchronization mechanisms (e.g.,

synchronized

,

RWLock

) are used where needed. • No blocking calls inside critical sections that could lead to deadlocks or performance degradation. • Verified thread-safe collections are used (e.g.,

ConcurrentHashMap

,

CopyOnWriteArrayList

). • Validated proper exception handling in multi-threaded code to avoid silent thread termination. ## How was this PR tested? • New unit tests added. • New integration tests added. • Modified or extended existing tests. • Verified backward compatibility (if applicable). ## Does this PR introduce any user-facing or breaking changes? • No. You can skip the rest of this section. • Yes. Clearly explain the behavior change and its impact. linkedin/venice

1 new commit pushed to

<https://github.com/linkedin/venice/tree/main|main>

by sixpluszero

<https://github.com/linkedin/venice/commit/b4a98e39740766c90f0f2f3ec3bdde11d6f489c0|b4a98e39>

- [dvc] Add heartbeat monitoring service for Da Vinci (#2132) linkedin/venice

1 new commit pushed to

<https://github.com/linkedin/venice/tree/main|main>

by huangminchn

<https://github.com/linkedin/venice/commit/ffa323f5095f053fe56242d9b06f7212b06116f4|ffa323f5>

- [controller] Check cluster specific multi-region configs for batch push eligibility (#2142) linkedin/venice

#2143 [vpj] Cleanup left-over D2 schema reader configs in VPJ Pull request opened by nisargthakkar ## Problem Statement Recently, we switched the implementation of the schema reader in VPJ. However some configs were left over This change removes those unnecessary configs ## Solution This change removes some unnecessary configs ### Code changes • Added new code behind a config. If so list the config names and their default values in the PR description. • Introduced new log lines. • Confirmed if logs need to be rate limited to avoid excessive logging. ### Concurrency-Specific Checks Both reviewer and PR author to verify • Code has no race conditions or thread safety issues. • Proper synchronization mechanisms (e.g.,

synchronized

,

RWLock

) are used where needed. • No blocking calls inside critical sections that could lead to deadlocks or performance degradation. • Verified thread-safe collections are used (e.g.,

ConcurrentHashMap

,

CopyOnWriteArrayList

). • Validated proper exception handling in multi-threaded code to avoid silent thread termination. ## How was this PR tested? • New unit tests added. • New integration tests added. • Modified or extended existing tests. • Verified backward compatibility (if applicable). ## Does this PR introduce any user-facing or breaking changes? • No. You can skip the rest of this section. • Yes. Clearly explain the behavior change and its impact. linkedin/venice

#2144 bump tehuti version to 0.12.4 Pull request opened by pthirun ## Problem Statement Tehuti version is out of date, upgrading to 0.12.4 ## Solution Bump tehuti version ### Code changes • Added new code behind a config. If so list the config names and their default values in the PR description. • Introduced new log lines. • Confirmed if logs need to be rate limited to avoid excessive logging. ### Concurrency-Specific Checks Both reviewer and PR author to verify • Code has no race conditions or thread safety issues. • Proper synchronization mechanisms (e.g.,

synchronized

,

RWLock

) are used where needed. • No blocking calls inside critical sections that could lead to deadlocks or performance degradation. • Verified thread-safe collections are used (e.g.,

ConcurrentHashMap

,

CopyOnWriteArrayList

). • Validated proper exception handling in multi-threaded code to avoid silent thread termination. ## How was this PR tested? • New unit tests added. • New integration tests added. • Modified or extended existing tests. • Verified backward compatibility (if applicable). ## Does this PR introduce any user-facing or breaking changes? • No. You can skip the rest of this section. • Yes. Clearly explain the behavior change and its impact. linkedin/venice

1 new commit pushed to

<https://github.com/linkedin/venice/tree/main|main>

by eldernewborn

<https://github.com/linkedin/venice/commit/7398e12ce98c85bf76e54fe2a7c4aca0d3452243|7398e12c>

- bump tehuti version to 0.12.4 (#2144) linkedin/venice

1 new commit pushed to

<https://github.com/linkedin/venice/tree/main|main>

by sushantmane

<https://github.com/linkedin/venice/commit/ba39e96300c49fa82e362c8a46b440048d47b93d|ba39e963>

- [server][dvc][cc] Read checkpoint state from PubSubPosition field with offset fallback (#2123) linkedin/venice

#2145 [admin-tool][compat] Remove D2 usage from Admin tool Pull request opened by nisargthakkar ## Problem Statement We are migrating away from using D2 for controller and will rely on URL-routing and DNS-based service discovery. This change removes D2 usage from Admin tool ## Solution This change removes D2 usage from Admin tool ### Code changes • Added new code behind a config. If so list the config names and their default values in the PR description. • Introduced new log lines. • Confirmed if logs need to be rate limited to avoid excessive logging. ### Concurrency-Specific Checks Both reviewer and PR author to verify • Code has no race conditions or thread safety issues. • Proper synchronization mechanisms (e.g.,

synchronized

,

RWLock

) are used where needed. • No blocking calls inside critical sections that could lead to deadlocks or performance degradation. • Verified thread-safe collections are used (e.g.,

ConcurrentHashMap

,

CopyOnWriteArrayList

). • Validated proper exception handling in multi-threaded code to avoid silent thread termination. ## How was this PR tested? • New unit tests added. • New integration tests added. • Modified or extended existing tests. • Verified backward compatibility (if applicable). ## Does this PR introduce any user-facing or breaking changes? • No. You can skip the rest of this section. • Yes. Clearly explain the behavior change and its impact. • Some admin tool commands will not work with Venice setups that only use D2 to communicate from Parent to Child controllers linkedin/venice

1 new commit pushed to

<https://github.com/linkedin/venice/tree/main|main>

by sushantmane

<https://github.com/linkedin/venice/commit/0e0a7d35beb21b8fa64fd2fe5f06613a455e0518|0e0a7d35>

- [server][dvc][test] Delete deprecated PubSubPositionDeserializer::getPositionFromWireFormat API (#2137) linkedin/venice

1 new commit pushed to

<https://github.com/linkedin/venice/tree/main|main>

by nisargthakkar

<https://github.com/linkedin/venice/commit/c51b673e07ddd379b4f12e693232b41297de4732|c51b673e>

- [vpj] Cleanup left-over D2 schema reader configs in VPJ (#2143) linkedin/venice

1 new commit pushed to

<https://github.com/linkedin/venice/tree/main|main>

by lluwm

<https://github.com/linkedin/venice/commit/ffdfe6846417dcc2d1d32a5afba9e40a93f1aec5|ffdfe684>

- [client] ClientStats metrics OTEL migration (#2102) linkedin/venice

#2141 [fc] Update the Fast client batch get config to be on par with thin client batch get config Pull request opened by ymuppala on <!date^1758144539^{date_short}|2025-09-17T21:28:59Z> ## Problem Statement As part of migrating users to fast client, we want to have the same batch get limits on fast client that are similar to the thin client. ## Solution This change makes the defaults the same between fast client and thin client. ### Code changes • Added new code behind a config. If so list the config names and their default values in the PR description. • Introduced new log lines. • Confirmed if logs need to be rate limited to avoid excessive logging. ### Concurrency-Specific Checks Both reviewer and PR author to verify • Code has no race conditions or thread safety issues. • Proper synchronization mechanisms (e.g.,

synchronized

,

RWLock

) are used where needed. • No blocking calls inside critical sections that could lead to deadlocks or performance degradation. • Verified thread-safe collections are used (e.g.,

ConcurrentHashMap

,

CopyOnWriteArrayList

). • Validated proper exception handling in multi-threaded code to avoid silent thread termination. ## How was this PR tested? • New unit tests added. • New integration tests added. • Modified or extended existing tests. • Verified backward compatibility (if applicable). ## Does this PR introduce any user-facing or breaking changes? • No. You can skip the rest of this section. • Yes. Clearly explain the behavior change and its impact. linkedin/venice