This message was deleted.

@Michal Siedlaczek helped me to resolve this issue, by using

SimplePythonGraphAdapter

🎉

Thank you @Michal Siedlaczek! Btw we didn’t advertise it but we now have

base.DefaultAdapter

. It’s a shortcuts for that one :)

hm, so would it actually be the default adapter now or it's just named this way? I think the pandas one was the default one or do I remember it wrong?

It’s named that way but we didn’t want to switch the default over for backwards compatibility

either way, good to know, much shorter to type this one 🙂

Great! Yeah I kept getting cranky when I typed it out so I made it shorter 😂

do people use this one often? do you have some data on that? I'm curious how many users have simpler use cases where you mostly operate on columns

Can look at what data we have (it’s limited), but there are two camps — dfs/series and general python objects (people obviously mix the two). My guess is that, when we started, it was entirely dfs/series, but these days it’s closer to 50/50.

interesting

Yeah! So that’s exactly it. Our broader vision is to be the layer that fits between you and your infrastructure. Represented as DAGs of functions, you can run your stuff anywhere, save it anywhere, etc… And easily run it in different contextx. Some more ineresting use-cases that don’t fit the series -> series bill: • More general ML training — pass around dataframes and models, functions that output metrics — you can use this to easily copy/paste from jupyter notebooks and have more “production-ready” code that you can cache pieces of, etc… • LLM/RAG workflows — handle prompts, API calls, etc… as a DAG • web-services — logic can get a little complicated and a DAG is a nice way to do it (can also run with pandas series/dfs but have modular pieces that form inputs/outputs) • parallelization to fan out, load, and process multiple files — see this example I just added • And quite a few more…

@Michal Siedlaczek if you construct the driver using our new

Builder

it defaults to .execute() returning a dictionary by using the “default adapter”.

dr = (
    driver.Builder()
    .with_config({})  # replace with configuration as appropriate
    .with_modules(*MODULES)
    .build()
)

Also, independent of how you construct the driver, you could substitute

.execute()

with

_, dict_result = dr.materialize(additional_vars=OUTPUTS, inputs=...)

and the second part of the tuple returned is always a dictionary corresponding to the outputs requested to

additional_vars

.

Ahh, yep, thanks @Stefan Krawczyk — I realized that we did make it the default (not just calling it that) for the new way to instantiate the driver