This message was deleted.

Hey — a few approaches. AFK but I can show you code samples later. 1. You can do so in the functions, but you have to maintain the join index with the series. Then, you may need to write a custom results builder to join if there’s funky logic (e.g. you might want to do a chained left merge). I’d recommend testing this out. 2. You can always set the values you don’t want as nan (or some sentinel value) in functions so the index stays the same and add to the results builder or some post-processing step to remove them. 3. You can write a function that accepts upstream series and joins them in the way you want — effectively (1) but you make it part of the DAG rather than outside.

Thanks for the approaches :) Codesamples would be really helpful :))

So yeah! Here’s what it lokos like (pseudocode): 1. Results builder — see https://hamilton.dagworks.io/en/latest/reference/api-extensions/custom-result-builders/ — should be easy enough to adapt. 2. nans:

res = dr.execute(["ID", "customer_id", "first_name"], ...)
res = res.dropna() # You should figure out the best way to do this -- maybe across just some columns?

3. Joining in functions:

def final_result(ID: pd.Series, customer_id: pd.Series, first_name: pd.Series, filter_name: str) -> pd.DataFrame:
    df = pd.concat([ID, customer_id, first_name], axis=0)
    return df[df.first_name == filter_name]

res = dr.execute(["final_results"], ...)

OTOH, it might just work! Worth a try. All it does is a concat (I think), and pandas is smart about indices:

>>> a = pd.Series(index=[1,2,3], data=['a','b','c'])
>>> b = pd.Series(index=[2,3,4], data=['e', 'f', 'g'])
>>> pd.concat([a,b], axis=1)
     0    1
1    a  NaN
2    b    e
3    c    f
4  NaN    g