Seth Stokes
10/24/2024, 7:05 PMoverrides with @subdag or would config work the same here or?Thierry Jean
10/24/2024, 7:28 PM# feature_modules.py
def feature_a(...):
def feature_b(...)
# main_dag.py
@subdag(
feature_modules,
inputs={"path": source("source_path")},
config={}
)
def feature_engineering(feature_df: pd.DataFrame) -> pd.DataFrame:
return feature_df
Then, the nodes of the subdag will be namespaced with `feature_engineering.NODE_NAME`which allows you to use overrides for feature_a via
dr.execute(
["feature_engineering],
overrides={
"feature_engineering.feature_a": ...
}
)
Does this help?Seth Stokes
10/31/2024, 3:05 AM