Seth Stokes
08/08/2024, 4:03 PM@subdag.Seth Stokes
08/08/2024, 4:03 PM@subdag(
execution_types,
inputs={"trade_numbers": source("trade_numbers")}
)
def raw_trades(
execution_type_flags: pd.DataFrame,
raw_loaded_report_data: pd.DataFrame,
) -> pd.DataFrame:
trades = pd.concat([
raw_loaded_report_data.assign(etrm=""),
])
return pd.merge(
trades,
execution_type_flags,
on="TRADE_NUM", how="left"
)
raw_loaded_report_data and trade_numbers are defined in the same file as above snippet.
I can get raw_loaded_report_data directly from the driver but when I ask for raw_trades via the driver, it says that i need to pass it in.
Any idea as to what I am missing here?Stefan Krawczyk
08/08/2024, 4:29 PM@subdag(
execution_types,
inputs={"trade_numbers": source("trade_numbers")},
external_inputs=["raw_loaded_report_data"]
)
2. Split into two functions:
@subdag(
execution_types,
inputs={"trade_numbers": source("trade_numbers")}
)
def raw_trades_partial(
execution_type_flags: pd.DataFrame,
) -> pd.DataFrame:
return execution_type_flags
def raw_trades(raw_trades_partial: pd.DataFrame, raw_loaded_report_data: pd.DataFrame) -> pd.DataFrame:
trades = pd.concat([
raw_loaded_report_data.assign(etrm=""),
])
return pd.merge(
trades,
raw_trades_partial,
on="TRADE_NUM", how="left"
)Seth Stokes
08/08/2024, 5:33 PM