venice

Great to have you onboard Ridwan! Let us know if you have any questions :smile:

Let us know if you have any questions :smile:

Will do. Just dipping my toe in to the Venice canals now. Hope to be a gondolier soon although I'll need autotune if I start to sing.

We don't have a lot of documentation on this as historically within LinkedIn we don't want users to get too smart for their own good :sweat_smile:.  We guide them towards our official client to make support easier.  That said, if you play around with the query script that I believe is part of the docker demo, it prints the uri of the request being made and you can observe the steps.  First a CLUSTER_DISCOVERY request is made to find the hosting cluster to a given store, and then all requests get sent to the returned uri.  Gets are either single key get (which will be a hash of the key in the URI with a get method), a post for batch get where the body of the request is a list of keys, or a read compute request (where I forget how that's set).  Results returned will be in avro binary and need deserialization on the client side

There are some python/java library wrappers we've historically been interested in for doing the kind of work you're proposing, but just haven't really gotten too.  We don't have a lot of great python experience in the team so the drawbacks on some of these are a bit amorphous to us without taking the time to dig deep

But the draw would be simple maintenance (where most of the complexity lives in one codebase).

Though if you feel this is the wrong direction I certainly won't look a gift horse in the mouth :joy:

<@U03USK369J5> thank you so much, this is very helpful. No worries about the documentation, i know what you mean about people getting too smart for their own good :laughing: been there before too.

I'm still very early in learning about Venice so i dont have strong opinions on a lot of it, but im going to play around more with it - if there's been any work / investigation into those python wrappers that is worth sharing i'd happily take a look before getting started - i have basically no Java experience but Python i'm very comfortable with and I definitely think creating a basic Python client wont be a difficult task, but with that said i'm still so early in learning about Venice that i'll qualify it by saying that i have more reading to do :slightly_smiling_face: I'll take a look at the query script in the docker demo and go from there for now!

I begun documenting the REST spec…

<https://github.com/linkedin/venice/blob/93169cf561044f04303e0c440c2151e6f67df175/docs/dev_guide/router_rest_spec.md>

Still incomplete, but what’s there should be correct at least

FYI, I pushed up some more details on that branch

Here: <https://github.com/linkedin/venice/pull/424>

Also visible here: <https://felixgv.github.io/venice/docs/dev_guide/router_rest_spec>

I’ve opened up the PR for review. I’m done documenting single gets and batch gets. Planning to leave Read Compute out of scope for now, but if you’re interested in implementing that as well, it could be done.

The spec documentation is now merged: <https://venicedb.org/docs/dev_guide/router_rest_spec|https://venicedb.org/docs/dev_guide/router_rest_spec>

Please take a look and let us know if you have any comments or questions

BTW, we've been discussing with <@U03USKUPRJM> that perhaps some router-side enhancements could make it easier to develop non-Java clients, such as converting the responses to JSON rather than needing the client to deal with Avro. These are just ideas ATM, and we have no plan to execute them, but if you're interested in it we're open to reviewing/merging PRs along those lines.

Hey All, my name is Murad, I am ex-LinkedIn employee, very happy to be part of this community again. :slightly_smiling_face:
:raised_hands: :wave:

Hi folks, I’m Nicolò and I’m a software engineer at Datastax. Excited to follow and contribute to the Venice adoption !

Welcome! Do you know <@U03UDSURQDR>? He was also at Pure Storage before coming to LinkedIn to work in the Venice team :smile:

I guess you missed each other by ~6 months, based on your LinkedIn profiles :smile: …

&gt; Welcome! Do you know <@U03UDSURQDR>? He was also at Pure Storage before coming to LinkedIn to work in the Venice team :smile:
Wow ! it is nice to know more Pure alumni here at LinkedIn. I did know some other than Salil

Hey <@U05QBJVPHQE> welcome! nice to connect to a Puritan :slightly_smiling_face:

If you use Voldemort Build and Push then Venice can be a drop-in replacement for that. In addition to batch ingestion, Venice also supports ingesting from streams, as well as hybrid (batch+stream, also known as Lambda and Kappa architectures).

We have Docker images and some folks from the community also published Helm charts if you prefer. Let us know if you need any more info

Hi, my name is Yanick, I'm from Québec, Canada. I'm always interested in open source projects, and got curious about this one. I will follow for a while, but hopefully will be able to bring a few contributions.

Nice to meet you Yanick!  Always happy to meet potential contributors!

I imagine it might take a moment for you to get acquainted with the project.  Take some time to browse <https://venicedb.org/>.  Once you've gotten acquainted, we keep a list of good first time contributor bugs listed here: <https://github.com/linkedin/venice/issues>

And if you want to chat, we've of course have this slack, and we meet every other Wednesday on zoom (join <#C0567KESECQ|> for more info)

Yeah. I have watched both Felix's panel and the quick demo on YouTube, but this was not enough to get a concrete idea of the components and code. Looking at the issues and PRs is a good idea. Thanks

Hello, I'm a Brazilian developer, Backend software engineer, I like technology and innovation, I'm always improving myself, I like networking.

Hi, I am Senior Lead Data Scientist at LiveIntent. I have prior engineering experience working with high volume systems in advertising  / marketing space. Me, together with <@U05PTAKHMJL> and <@U05QNPM485C>, are currently trying to figure out production setup of Venice that would work for us.

Great to have you here Kacper! Looking forward to working together on this!

Hi everyone! Found Venice from the InfoQ podcast and found it really interesting. I would be really interested to work on a good first issue

You can view some open issues here... <https://github.com/linkedin/venice/issues>

But before going heads down into one of them, it may be good to double check here just to make sure the issue is not stale or to clarify any unclear details.

cc <@U03USK369J5> and <@U067HQNDH0A> who have some context on these issues.

Welcome!

Before you start diving into an issue, please reference our <https://venicedb.org/docs/dev_guide/how_to/workspace_setup|workspace setup guide>.

I'll be able to look through the issues on Friday to see if we have a good first one to start with.

That sounds perfect  <@U067HQNDH0A> thank you!

I'm thinking about this <https://github.com/linkedin/venice/issues/656|bug>. We could add retry with exponential backoff <https://github.com/linkedin/venice/blob/6918083b5de67740f0ae6e43276b83ab5fcf2fa2/internal/venice-common/src/main/java/com/linkedin/venice/utils/HelixUtils.java#L230-L243|here>?

What do you guys think <@U03USK369J5> <@U03T1CD7XUJ>?

Hey <@U08KDGAS3TL>,

We have a bi-weekly sync-up meeting that occurs on Mondays 9am PST. The next one is this coming <https://venicedb.slack.com/archives/C0567KESECQ/p1743782418497699|Monday>.

Also, if you're unable to make it, no worries!

If you have any questions, please don't hesitate to ask.

Hi Folks :slightly_smiling_face: I've also found Venice from the InfoQ podcast. I'm a sophomore student in Computer Science at the University of Sherbrooke, Canada, and intern in software development at Flare. I joined the slack out of curiosity and hopeful to learn more about all and anything regarding databases and big data.

Cheers!

Tomorrow 9am PST we're doing our contributor sync up on zoom.  If you have time, pop in to say hi!

Here are a couple resources I recommend to learn more about Venice:
1. <https://www.youtube.com/watch?v=pJeg4V3JgYo>
2. <https://www.linkedin.com/blog/engineering/open-source/open-sourcing-venice-linkedin-s-derived-data-platform>
    a. This is essentially what was talked about in the video but in text-form
Also, if you would like to contribute, please reference our <https://venicedb.org/docs/dev_guide/how_to/workspace_setup|workspace setup guide>.

No problem. Please don’t hesitate to ask any questions 

Hi all - I'm a LinkedIn employee and will probably end up making a sprinkling of contributions to make Venice a bit more configurable for our internal needs.

Welcome!  Eager to hear what you have in mind!