👋 Hey all, I’m trying to look into possibly using airbyte for a few different things. One of the most impactful ones could potentially be a lakehouse -> DynamoDB connection. It seems like both the sources and sinks needed for this are available, but I was curious on the DynamoDB destination / connecter. 1. What’s the max performance we should expect, Assuming no partitioning or provisioning concerns? Our current solution is able to surpass over 15000 item writes / second and was hoping for something within that order of magnitude. The connector only mentioned provisioning concerns, but at the scale I’m looking at it may depend more on the implementation than the table settings. 2. It seems like the destination output schema is relatively fixed with airbyte id = PK, sync time = sort key, then a json blob of the actual data, etc. For performance reasons we need to use our own schema. Is this something that is generally supported in other airbyte conenctors? 3. I’ve noticed theres several options for building our own connecters. I see that java is one of the options, I was wondering if other JVM are supported as well (EG: Scala, Kotlin). Also, probably outside of scope of the question but I’d love the ability to write one in Rust 😉 Thanks for taking the time to read and consider my noob questions 🙂

Hello Zachary Paden, it's been a while without an update from us. Are you still having problems or did you find a solution?

Hey there Sajarin, I ended up not pursuing airbyte for this at the time being, but, I suppose in general my questions still stand. 1. Do all airbyte connectors require a certain pre-defined schema? Or is it just that the some connectors are limited in this way? 2. What sort of performance can one expect regularly? (I’d imagine it quite depends on which connector).

Hey Zachary, most airbyte connectors require a certain predefined schema and performance is a touch question to answer because it really depends on your use case and your ingestion load. We recommend experimenting with Airbyte to see if it’d be a right fit for you, even if you’ve decided not to pursue Airbyte for the time being

Thanks Saj! We’re keeping airbyte in mind for some other use cases coming up that we believe will line up with the features and capabilities of airbyte super well 🙂 Thanks for the insight!