Slackbot
02/17/2023, 12:10 AMStefan Krawczyk
02/17/2023, 12:12 AMfrom non_hamilton_script import historical_dataframe
import transforms # your hamilton functions
dr = driver.Driver({... your config ...}, transforms)
df = dr.execute([ .... outputs ...], inputs={"historical_dataframe": historical_dataframe})
# Option 2: wrap import in a function with your transforms
# transforms.py
def historical_dataframe() -> pd.DataFrame:
from non_hamilton_script import historical_dataframe
return historical_dataframe
def load_data_feed(path: str) -> pd.DataFrame:
...
return df
def complete_dataset(load_data_feed: pd.DataFrame,
historical_dataframe: pd.DataFrame) -> pd.DataFrame:
return pd.concat([historical_dataframe, load_data_feed])
Seth Stokes
02/17/2023, 12:22 AMSeth Stokes
02/17/2023, 12:22 AMStefan Krawczyk
02/17/2023, 12:23 AMSeth Stokes
02/17/2023, 9:23 PMfrom non_hamilton_script import historical_dataframe
import transforms # your hamilton functions
dr = driver.Driver({... your config ...}, transforms)
df = dr.execute([ .... outputs ...], inputs={"historical_dataframe": historical_dataframe()})
I believe the historical_dataframe
needs to be instantiated in the inputs.Seth Stokes
02/17/2023, 9:35 PMStefan Krawczyk
02/17/2023, 9:43 PMStefan Krawczyk
02/17/2023, 9:43 PMSeth Stokes
02/17/2023, 9:44 PMStefan Krawczyk
02/17/2023, 9:46 PMI believe theyeah if there is a function to exercise, it needs to be run to create the dataframe so it’s passed in.needs to be instantiated in the inputs.historical_dataframe
Seth Stokes
02/17/2023, 9:58 PM# transforms.py
def historical_dataframe(historical_dataframe: pd.DataFrame) -> pd.DataFrame:
return historical_dataframe.rename(columns=columns_mapping)
Seth Stokes
02/17/2023, 10:20 PMdr.execute([], inputs={})
method escalate pandas warnings to errors by chance?
Option 1 logic breaks my kernel somehow. Option 2 is still good however.Stefan Krawczyk
02/17/2023, 10:35 PMStefan Krawczyk
02/17/2023, 10:35 PMis there any issue with reusing arg name as the new function/column name as below?
```# transforms.py
def historical_dataframe(historical_dataframe: pd.DataFrame) -> pd.DataFrame:
return historical_dataframe.rename(columns=columns_mapping)```Yes, this isn’t allowed.
Seth Stokes
02/17/2023, 10:40 PMStefan Krawczyk
02/17/2023, 10:41 PM```def historical_dataframe(historical_dataframe_old: pd.DataFrame) -> pd.DataFrame:
return historical_dataframe.rename(columns=columns_mapping)```just change the parameter name and then the input value accordingly.
Seth Stokes
02/17/2023, 11:51 PM