Seth Stokes
07/02/2024, 3:36 PMdr.visualize_execution()
reveal what possible @config.when
arguments exist with their paths?Stefan Krawczyk
07/02/2024, 3:39 PMSeth Stokes
07/02/2024, 3:41 PMSeth Stokes
07/02/2024, 3:43 PM@config.when(env="test")
@pipe(
step(
_filter_data,
source("filter_src1"),
)
)
def trades__test(raw_trades: pd.DataFrame) -> pd.DataFrame:
return raw_trades
I was hoping to see in the DAG that filter_src1
is a node that can be used to filter trades
Elijah Ben Izzy
07/02/2024, 3:45 PMenv
is not “test” (I think your logic might be reversed), in which case it won’t show upSeth Stokes
07/02/2024, 3:55 PM@config.when(env="test")
@pipe(
step(
_filter_data,
source("filter_src1"),
)
)
def trades__test(raw_trades: pd.DataFrame) -> pd.DataFrame:
return raw_trades
Example (2)
Shows the optional dependency filter_src1
in DAG
@config.when(env="test")
def trades__test(
raw_trades: pd.DataFrame,
filter_src1: Optional[str] = None, # pandas.query(filter_src1)
) -> pd.DataFrame:
return raw_trades.pipe(_filter_data)
Stefan Krawczyk
07/02/2024, 4:06 PM