venice

Hi all, :wave: I am Madhusudhan, you can call me Madhu. Gone through the presentation of "Open-Sourcing Venice", wanted to learn more about it and also contribute by starting with minor things :slightly_smiling_face:.

Hello ! I am Enrico, I would like to test Venice

Hi! I’m Ridwan. Here to learn and explore Venice!

Great to have you onboard Ridwan! Let us know if you have any questions :smile:

Hi! I'm Dave from DataStax. Here to learn Venice and then do performance testing.

Let us know if you have any questions :smile:

Will do. Just dipping my toe in to the Venice canals now. Hope to be a gondolier soon although I'll need autotune if I start to sing.

Hi everybody!  I’m Ryan from DataStax / Kaskada. Excited to be a part of the Venice community.

Hi all :slightly_smiling_face: I'm Eric Hare, I'm a software engineer at DataStax, and I'm beginning development of a Python client / API for venice. I was wondering if anyone can share any information on the Router HTTP API, if any documentation / information is available, so I can get started. Thanks a lot in advance :slightly_smiling_face:

We don't have a lot of documentation on this as historically within LinkedIn we don't want users to get too smart for their own good :sweat_smile:.  We guide them towards our official client to make support easier.  That said, if you play around with the query script that I believe is part of the docker demo, it prints the uri of the request being made and you can observe the steps.  First a CLUSTER_DISCOVERY request is made to find the hosting cluster to a given store, and then all requests get sent to the returned uri.  Gets are either single key get (which will be a hash of the key in the URI with a get method), a post for batch get where the body of the request is a list of keys, or a read compute request (where I forget how that's set).  Results returned will be in avro binary and need deserialization on the client side

There are some python/java library wrappers we've historically been interested in for doing the kind of work you're proposing, but just haven't really gotten too.  We don't have a lot of great python experience in the team so the drawbacks on some of these are a bit amorphous to us without taking the time to dig deep

But the draw would be simple maintenance (where most of the complexity lives in one codebase).

Though if you feel this is the wrong direction I certainly won't look a gift horse in the mouth :joy:

<@U03USK369J5> thank you so much, this is very helpful. No worries about the documentation, i know what you mean about people getting too smart for their own good :laughing: been there before too.

I'm still very early in learning about Venice so i dont have strong opinions on a lot of it, but im going to play around more with it - if there's been any work / investigation into those python wrappers that is worth sharing i'd happily take a look before getting started - i have basically no Java experience but Python i'm very comfortable with and I definitely think creating a basic Python client wont be a difficult task, but with that said i'm still so early in learning about Venice that i'll qualify it by saying that i have more reading to do :slightly_smiling_face: I'll take a look at the query script in the docker demo and go from there for now!

I begun documenting the REST spec…

<https://github.com/linkedin/venice/blob/93169cf561044f04303e0c440c2151e6f67df175/docs/dev_guide/router_rest_spec.md>

Still incomplete, but what’s there should be correct at least

FYI, I pushed up some more details on that branch

Here: <https://github.com/linkedin/venice/pull/424>

Also visible here: <https://felixgv.github.io/venice/docs/dev_guide/router_rest_spec>

I’ve opened up the PR for review. I’m done documenting single gets and batch gets. Planning to leave Read Compute out of scope for now, but if you’re interested in implementing that as well, it could be done.

The spec documentation is now merged: <https://venicedb.org/docs/dev_guide/router_rest_spec|https://venicedb.org/docs/dev_guide/router_rest_spec>

Please take a look and let us know if you have any comments or questions

BTW, we've been discussing with <@U03USKUPRJM> that perhaps some router-side enhancements could make it easier to develop non-Java clients, such as converting the responses to JSON rather than needing the client to deal with Avro. These are just ideas ATM, and we have no plan to execute them, but if you're interested in it we're open to reviewing/merging PRs along those lines.

Hey All, my name is Murad, I am ex-LinkedIn employee, very happy to be part of this community again. :slightly_smiling_face:
:raised_hands: :wave:

Hi folks, I’m Nicolò and I’m a software engineer at Datastax. Excited to follow and contribute to the Venice adoption !

Hi, I am Dejan, working as a lead engineer at LiveIntent. We've been using Voldemort for a few years (in its read-only "flavor") and currently evaluating Venice as its replacement.

Hi, my name is Jonathan, and I am a currently an undergraduate computer science major. I’m pretty new to open source projects so I’m just here to learn and help contribute in any way I can.

Hi My name is Zicheng Liu, and I am currently a Lead engineer at Pure Storage. I am pretty into the *Streaming Processing area* overall and would like to explore the open source community within this domain. I would like to participate in the discussion and contribute whatever it would be valuable to the community

Welcome! Do you know <@U03UDSURQDR>? He was also at Pure Storage before coming to LinkedIn to work in the Venice team :smile:

I guess you missed each other by ~6 months, based on your LinkedIn profiles :smile: …

&gt; Welcome! Do you know <@U03UDSURQDR>? He was also at Pure Storage before coming to LinkedIn to work in the Venice team :smile:
Wow ! it is nice to know more Pure alumni here at LinkedIn. I did know some other than Salil

Hey <@U05QBJVPHQE> welcome! nice to connect to a Puritan :slightly_smiling_face:

Hi !! My name is Vincent, and I am a data infrastructure engineer at Arkea. We used for several years Voldemort Project as principal cache server for critical apps (We push data every night from our Z IBM platform. Last year I worked in upgrade technicaly Voldemort (Java 17 / Gradle / Library ...), and then I discovered Venice, so perhaps Voldemort replacement ...

If you use Voldemort Build and Push then Venice can be a drop-in replacement for that. In addition to batch ingestion, Venice also supports ingesting from streams, as well as hybrid (batch+stream, also known as Lambda and Kappa architectures).

We have Docker images and some folks from the community also published Helm charts if you prefer. Let us know if you need any more info