Amirali Sharifzad
09/24/2024, 12:14 AMto_parquet is not working, and somehow my df is a dict, and not a df despite using the pandas result_builder.
a. it created a df before using Builder pattern for my driver.Thierry Jean
09/24/2024, 12:18 AM.with_adapters() expects multiple adapters, not a list of adapters. Rewriting the code to:
dr = (
driver.Builder()
.with_modules(*modules)
.with_adapters(tracker, rga)
.with_materializers(saver)
.build()
)
Should properly setup the pandas result builder and the Hamilton tracker to populate the UI.
Then, to materialize the result, you will have to call .execute(["features__parquet"]), in other words, the id you set in the to.parquet object. When requesting .execute(["output_data"]) you are producing the dataframe, but not the parquet file, which is the next node in the DAG. This gives you some control over when you want to produce the file or notThierry Jean
09/24/2024, 12:18 AMAmirali Sharifzad
09/24/2024, 12:22 AMElijah Ben Izzy
09/24/2024, 12:47 AM