hi - quick question - i have `@save_to.parquet(.....
# hamilton-help
z
hi - quick question - i have
@save_to.parquet(...)
sprinkled around my pipelines, and in order to trigger the saving routine, i need to explicitly call out
save.<name>
in
final_vars
in
driver.execute
question - is there a way i can modify
my driver.execute
call, so that all vars that are decorated with
save_to
get persisted? thank you very much in advance!
t
AFAIK, it's not directly possible. Though, you could write a helper function to generate your list of requested nodes from all the available nodes
Copy code
def _include_saver_nodes(dr: Driver, requested_nodes: list[str]) -> list[str]
  selected_saved_nodes = []
  for node in dr.list_available_nodes():
    # save nodes will have the prefix `save.`
    if "save." not in node.name:
      continue
    
    _, _, base_node_name = node.name.partition(".")
    if base_node_name in requested_nodes:
       selected_saved_nodes.append(node.name)
  
  return requested_nodes + selected_saved_nodes
    

dr = Builder().with_modules(...).build()
requested_nodes = _include_saver_nodes(dr, ["customers", "sales_per_month"])

dr.execute(requested_nodes)
z
i got you, much appreciated!
e
Have you seen dr.materialize? This might help you
👍 1