❓Have you encountered any performance issue with Kedro? i.e. config/dataset takes forever to load. We are starting to look into opportunities to improve performance of Kedro but need to get some more clarity what's actually slow. The 1st step for this is create something internally we can run in CI to inform ourselves.

69.5 % of respondents of the Stack Overflow survey visit SO at least a few times per week https://survey.stackoverflow.co/2024/community#1-frequency-of-visiting-stack-overflow comparatively, 43.9 % use Slack for synchronous communication https://survey.stackoverflow.co/2024/technology#1-synchronous-tools what's your preferred method to find help about programming questions? (not only about Kedro) stackoverflow Stack Overflow, S tool-specific Slack, chatgpt ChatGPT (or other AI tools), ➕ Other (please leave comment!)

Hi all! In the interest of building the best end-to-end Python ELT experience, a couple of us from the Kedro team (myself and @Juan Luis) chatted with two people from the dlt team about how we can best integrate the two. We explored a few options and have some initial ideas, enumerated in https://github.com/kedro-org/kedro/issues/4057, but would love to get the community's input (especially if you use both libraries, or have previously explored dlt!).

@Alexey Gravanov has left the channel

🚀 Help Us Improve Kedro! 🚀 We’re conducting a short survey to learn more about how you deploy Kedro pipelines and the challenges you face. Your feedback will directly contribute to enhancing Kedro’s documentation and deployment tools. 👉 Take the Survey Thank you for your valuable insights! Best regards, The Kedro Team

Hi all! I'm planning to spend some time on improving the integration between Kedro and Ibis this winter, and would love your thoughts on priorities and if I've missed anything in this list (especially if you've used or have considered using Kedro and Ibis together): https://github.com/kedro-org/kedro/issues/4190

Hi everyone! I'd love to get your thoughts on a new feature we're experimenting with. We're adding the option for users to change the flowchart orientation from top-to-bottom to left-to-right. Do you think this flexibility would be useful in your workflow? Your feedback will be super valuable as we work on creating this feature!

Hi Kedro community! Is anyone here a user of the LambdaDataset and if so, what do you use it for? We're thinking of removing the dataset, but would like to know its usage before doing it.

what if you could see the progress of the pipeline visually, and also immediately spot failing nodes? any early thoughts?

Hi Team, Would it be helpful if you could quickly visualize a Kedro pipeline without creating a full blown Kedro project during your project exploration journey in jupyter notebook ? Appreciate any ideas or thoughts on this feature we would like to ship in the future KedroViz release. Thank you

thoughts? https://github.com/kedro-org/kedro/discussions/4453

Hi Team! I have a PR(https://github.com/kedro-org/kedro-plugins/pull/1008/files) to add

s3://

support to spark dataset along with

s3a://

on a user request. I know s3a is recommended for spark but would love to take your viewpoint on would you really want that and for what use case? is it for EMRFS?

When your Kedro projects scale, have you faced challenges such as: • 🪢 High dependency between development and production? • 🚧 Difficulty in making changes without impacting the entire pipeline? • ⏳ Large dependency footprints, with long installation times? • 🧶 Monolithic testing, requiring full pipeline tests in CI/CD? • ✍️ Multiple teams using the same pipeline code but with different configurations? We've developed a tool called Alloy that helps data science teams scale their applications by building consumable Kedro pipelines from reusable Python packages, enforcing best development practices. We are currently seeking users interested in trialing Alloy to support our research on potentially open-sourcing it as a sister product to Kedro. Please react with 👍 if you'd be interested in learning more

Alloy-overview.png

Hi Team, Would anyone be interested in helping us QA and provide feedback on the Support filtering of pipelines on Kedro-Viz flowchart in VSCode feature before we ship it. I’d be happy to do a quick demo on my setup and would really appreciate your thoughts. Your feedback will help us refine the feature before release. GIF added in 🧵. Let me know if you're interested—thank you!

Hey, just noticed a small grammar issue in the "Visualise the project" section. The sentence "To install it your virtual environment" seems to be missing a word. Maybe it should be "To install it in your virtual environment"?

Hey kedro-vscode users ! We're adding two exciting features to the VS Code Kedro extension: ✅ Expand/Collapse Pipelines on kedro-viz ✅ Filter Pipelines on kedro-viz If you're a VS Code Kedro user and have 10 minutes for a quick call, we'd love to get your feedback! Just reply to this message, and we'll reach out. Alternatively, if the GIFs help you understand the features and you don’t have time for a call, feel free to leave your feedback in a message. Thanks! 🚀

Hi everyone! Happy Monday! K We’ve recently added documentation for using Delta tables and Iceberg tables and their versioning capabilities with Kedro. They’ll be out with the next release but you can already see them in the

latest

version of the docs here and here. While

kedro-datasets

offers various connectors to interact with Delta tables, it doesn’t have support for Iceberg tables currently. We’d like to hear from the Kedro community about what they’d like to see! If you’ve worked with Iceberg tables and Kedro before or would like to in the future, leave your comments under this issue!

📢 Calling all Kedro users! Help us improve the Kedro VS Code Extension Kvscode We're looking for Kedro users to participate in a 20-30 minute user testing session for our latest Kedro VS Code extension walkthrough. You can install the Kedro VS Code Extension here: https://marketplace.visualstudio.com/items?itemName=kedro.Kedro. Why join? • Share feedback on the setup experience of Kedro in VS Code. • Help us identify and fix any pain points in the extension. • Get an early look at improvements and shape the future of Kedro tooling! How to participate? • No prior experience with the extension required! • Simply walk through the setup steps and tell us what works (or doesn't). • Sign up here: Please react with 👍 if you'd be interested in participating Your feedback is invaluable—thank you for helping us build a better Kedro experience!

👋 Hey everyone! We’d love your help validating a new guide on node grouping methods in Kedro: pipelines, tags, and namespaces. 🔍 What we’re asking: Please read through the node grouping guide, then share your feedback on: • What’s clear vs. confusing? • Any pain points or inconsistencies? • How easy is it to understand the differences between pipelines, tags, and namespaces? 🧪 Think of it like a quick "bash test" — try out each method and let us know where things don’t make sense. 💬 This is a self-serve task — no supervision needed. Just reply directly in this thread with your thoughts and findings! If you're interested in a follow-up session to test out namespace-based grouping, give this message a 👍 and we’ll reach out soon. Thanks in advance for your insights — they’re super valuable in shaping the Kedro experience!

In case you haven't seen the news! 📣 K Quick reminder to fill in our SHORT survey, it'll only take 2 mins 🙂🙏 Survey closes in a few days! thankyou bufo pat https://kedro-org.slack.com/archives/C03RKAQ0MGQ/p1742992371097649

Kedro 1.0 When comes？

Very specific question, but would be happy to talk more to/learn from anybody who's facing these challenges: • Are there people who are primarily developing transformation code outside of Databricks (i.e. using Kedro, I assume not too many people using alternatives in here 😉), and trying to better orchestrate that? • More specifically, are you trying to do this not in Databricks (i.e. not using Databricks Workflows), but using Dagster or Airflow?

@Max Pardoe has left the channel

At first I was dealing something with namespace and I figure I need something quick to check if nodes are connected as expected so I make a ASCII graph. Then I figure it doesn't take too much to make this work with Mermaid and this is now the newer version of it. Is something like this useful? (The solid yellow one are node, black one are raw inputs, green are output and blue are intermediate input/output). This will render in notebook without any extra dependencies

I’ve made a little proposal here about natively using Pydantic for parameters, please give us your thoughts?

<!here> @AnselmAdrian recently contributed a new runner workbench component that lets you edit parameters, launch runs, monitor jobs with live logs/history, and give stakeholders visibility into experiments - all directly in Kedro-Viz. Below is GIF of the solution (sorry abt the res, you can see original video on the PR) and the PR where you can explore it in detail (see also [issue #2483] for context). We’d love your thoughts: how useful would this be in your workflows? Do you already solve this today with another tool? Please share feedback either on the PR/issue or here on Slack 🙌

@channel 📣 Our very short 1 min survey is closing tomorrow ⏰ please share your thoughts with us if you haven't already! THANK YOU 🎉