📣 The 0.0.1 (beta) version of Kedro VSCode extension is out! It works for 0.19+ only. If you encounter any bug, report it here! Note: • Make sure you select the proper Python Interpreter in VSCode • 0.19+ only • It works mostly for a standard

spaceflights

structure Create a new repo and open it in a VScode workspace:

kedro new -n spaceflights --tools=none --example=yes

VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=kedro.Kedro

Calling Apache Airflow users!! 🎉 We’re looking to improve our kedro-airflow plugin, which allows you to create deployment-ready DAGs for Apache Airflow! We want to know: • Please react with A below if you have used

apache-airflow

to orchestrate your Kedro projects before. • Please react with K below if you have used

kedro-airflow

to create DAGs for deployment on Apache Airflow. Faced any hurdles? 🚧 Drop a comment in this thread to share your pain points 🔴 you’ve encountered in using / trying to use the

kedro-airflow

plugin.

hi everyone, a heads up - we're working towards making Kedro telemetry opt-out (hence enabled by default). see our detailed writeup here https://github.com/kedro-org/kedro-plugins/issues/715 if you have any thoughts, feel free to share them

Hello friends! We are exploring ways to make DataCatalog API more friendly, particularly the interactive workflow. This is designed to work with interactive workflow so you can hit

Tab

to get the dataset name without typing out the full thing. What do you think?

given that the new hot chapter in the Databricks v Snowflake race seems to be data catalogs, I opened this issue to start collecting some thoughts https://github.com/kedro-org/kedro-devrel/issues/141 if you have any, please feel free to chime in

Hey Kedro team, just wanted to share that I recently found dataset factories to be a super cool feature, and say thanks for that 😁. I'm doing a data ingestion & processing pipeline inspired by this article (@datajoely), where I have

int

layer as a typed/concatenated mirror of

raw

, then

pri

and

feat

etc. And while my

raw

datasets definitions are quite long and differ from dataset to dataset, e.g. like this:

raw_notifications_multisheet:
  type: pandas.ExcelDataset
  filepath: data/01_raw/...xlsx
  load_args:
    sheet_name: null
    dtype:
      Order: str
      Equipment: str
  <<: *raw_layer

It takes me just 3 dataset definitions to capture an arbitrary number of

int

,

pri

and

feat

layer datasets, all of which I just want to save as a parquet file.

"int_{dataset}":
  type: pandas.ParquetDataset
  filepath: data/02_intermediate/int_{dataset}.parquet
  <<: *intermediate_layer

"pri_{dataset}":
  type: pandas.ParquetDataset
  filepath: data/03_primary/pri_{dataset}.parquet
  <<: *primary_layer

"feat_{dataset}":
  type: pandas.ParquetDataset
  filepath: data/04_feature/feat_{dataset}.parquet
  <<: *feature_layer

If not dataset factories, the catalog YAML would have been incredibly long, or at best I would have to use a jinja for loop, which requires knowing all datasets in advance of the run.

do you use Kedro with LLMs? would love to hear your thoughts at https://github.com/kedro-org/kedro/discussions/3979

|| What is your view on versioning? || (Survey Link) Hello, this is Elijah from the Kedro team! K We are now looking at user stories, goals, and pain points of Kedro dataset versioning and other versioning tools. Would love to hear your thoughts in this 5 mins survey! Survey Link: https://www.surveys.online/jfe/form/SV_6zcS99SPPdtkWYC Your input will help us make Kedro even more awesome. Click the link below and let us know how you're using versioning in Kedro! Thanks a ton! 🙌 P.S. Completing this survey might just make you a superhero in the eyes of our user-research team. 🦸‍♂️🦸‍♀️

Is there any people want to use the Kedro VSCode extension (support 0.19+) but you cannot do this because you are stuck in older Kedro version? https://kedro-org.slack.com/archives/C03RKAQ0MGQ/p1720190568459939

❓Have you encountered any performance issue with Kedro? i.e. config/dataset takes forever to load. We are starting to look into opportunities to improve performance of Kedro but need to get some more clarity what's actually slow. The 1st step for this is create something internally we can run in CI to inform ourselves.

69.5 % of respondents of the Stack Overflow survey visit SO at least a few times per week https://survey.stackoverflow.co/2024/community#1-frequency-of-visiting-stack-overflow comparatively, 43.9 % use Slack for synchronous communication https://survey.stackoverflow.co/2024/technology#1-synchronous-tools what's your preferred method to find help about programming questions? (not only about Kedro) stackoverflow Stack Overflow, S tool-specific Slack, chatgpt ChatGPT (or other AI tools), ➕ Other (please leave comment!)

Hi all! In the interest of building the best end-to-end Python ELT experience, a couple of us from the Kedro team (myself and @Juan Luis) chatted with two people from the dlt team about how we can best integrate the two. We explored a few options and have some initial ideas, enumerated in https://github.com/kedro-org/kedro/issues/4057, but would love to get the community's input (especially if you use both libraries, or have previously explored dlt!).

@Alexey Gravanov has left the channel

🚀 Help Us Improve Kedro! 🚀 We’re conducting a short survey to learn more about how you deploy Kedro pipelines and the challenges you face. Your feedback will directly contribute to enhancing Kedro’s documentation and deployment tools. 👉 Take the Survey Thank you for your valuable insights! Best regards, The Kedro Team

Hi all! I'm planning to spend some time on improving the integration between Kedro and Ibis this winter, and would love your thoughts on priorities and if I've missed anything in this list (especially if you've used or have considered using Kedro and Ibis together): https://github.com/kedro-org/kedro/issues/4190

Hi everyone! I'd love to get your thoughts on a new feature we're experimenting with. We're adding the option for users to change the flowchart orientation from top-to-bottom to left-to-right. Do you think this flexibility would be useful in your workflow? Your feedback will be super valuable as we work on creating this feature!

Hi Kedro community! Is anyone here a user of the LambdaDataset and if so, what do you use it for? We're thinking of removing the dataset, but would like to know its usage before doing it.

what if you could see the progress of the pipeline visually, and also immediately spot failing nodes? any early thoughts?

Hi Team, Would it be helpful if you could quickly visualize a Kedro pipeline without creating a full blown Kedro project during your project exploration journey in jupyter notebook ? Appreciate any ideas or thoughts on this feature we would like to ship in the future KedroViz release. Thank you

thoughts? https://github.com/kedro-org/kedro/discussions/4453

Hi Team! I have a PR(https://github.com/kedro-org/kedro-plugins/pull/1008/files) to add

s3://

support to spark dataset along with

s3a://

on a user request. I know s3a is recommended for spark but would love to take your viewpoint on would you really want that and for what use case? is it for EMRFS?

When your Kedro projects scale, have you faced challenges such as: • 🪢 High dependency between development and production? • 🚧 Difficulty in making changes without impacting the entire pipeline? • ⏳ Large dependency footprints, with long installation times? • 🧶 Monolithic testing, requiring full pipeline tests in CI/CD? • ✍️ Multiple teams using the same pipeline code but with different configurations? We've developed a tool called Alloy that helps data science teams scale their applications by building consumable Kedro pipelines from reusable Python packages, enforcing best development practices. We are currently seeking users interested in trialing Alloy to support our research on potentially open-sourcing it as a sister product to Kedro. Please react with 👍 if you'd be interested in learning more

Alloy-overview.png

Hi Team, Would anyone be interested in helping us QA and provide feedback on the Support filtering of pipelines on Kedro-Viz flowchart in VSCode feature before we ship it. I’d be happy to do a quick demo on my setup and would really appreciate your thoughts. Your feedback will help us refine the feature before release. GIF added in 🧵. Let me know if you're interested—thank you!

Hey, just noticed a small grammar issue in the "Visualise the project" section. The sentence "To install it your virtual environment" seems to be missing a word. Maybe it should be "To install it in your virtual environment"?

Hey kedro-vscode users ! We're adding two exciting features to the VS Code Kedro extension: ✅ Expand/Collapse Pipelines on kedro-viz ✅ Filter Pipelines on kedro-viz If you're a VS Code Kedro user and have 10 minutes for a quick call, we'd love to get your feedback! Just reply to this message, and we'll reach out. Alternatively, if the GIFs help you understand the features and you don’t have time for a call, feel free to leave your feedback in a message. Thanks! 🚀

Hi everyone! Happy Monday! K We’ve recently added documentation for using Delta tables and Iceberg tables and their versioning capabilities with Kedro. They’ll be out with the next release but you can already see them in the

latest

version of the docs here and here. While

kedro-datasets

offers various connectors to interact with Delta tables, it doesn’t have support for Iceberg tables currently. We’d like to hear from the Kedro community about what they’d like to see! If you’ve worked with Iceberg tables and Kedro before or would like to in the future, leave your comments under this issue!

📢 Calling all Kedro users! Help us improve the Kedro VS Code Extension Kvscode We're looking for Kedro users to participate in a 20-30 minute user testing session for our latest Kedro VS Code extension walkthrough. You can install the Kedro VS Code Extension here: https://marketplace.visualstudio.com/items?itemName=kedro.Kedro. Why join? • Share feedback on the setup experience of Kedro in VS Code. • Help us identify and fix any pain points in the extension. • Get an early look at improvements and shape the future of Kedro tooling! How to participate? • No prior experience with the extension required! • Simply walk through the setup steps and tell us what works (or doesn't). • Sign up here: Please react with 👍 if you'd be interested in participating Your feedback is invaluable—thank you for helping us build a better Kedro experience!

👋 Hey everyone! We’d love your help validating a new guide on node grouping methods in Kedro: pipelines, tags, and namespaces. 🔍 What we’re asking: Please read through the node grouping guide, then share your feedback on: • What’s clear vs. confusing? • Any pain points or inconsistencies? • How easy is it to understand the differences between pipelines, tags, and namespaces? 🧪 Think of it like a quick "bash test" — try out each method and let us know where things don’t make sense. 💬 This is a self-serve task — no supervision needed. Just reply directly in this thread with your thoughts and findings! If you're interested in a follow-up session to test out namespace-based grouping, give this message a 👍 and we’ll reach out soon. Thanks in advance for your insights — they’re super valuable in shaping the Kedro experience!

In case you haven't seen the news! 📣 K Quick reminder to fill in our SHORT survey, it'll only take 2 mins 🙂🙏 Survey closes in a few days! thankyou bufo pat https://kedro-org.slack.com/archives/C03RKAQ0MGQ/p1742992371097649