Jean-Paul Abbuehl
08/07/2024, 1:01 PMThierry Jean
08/07/2024, 2:47 PMjson_normalize() quite a lot, and I use it in this tutorial! (you will have to follow the instructions to download the data). The Hamilton pattern I suggest is
a. load the raw data
b. clean and format the raw data (no complex transforms)
c. do analysis, ML, etc.
In the notebook, I use the following code
def pdl_data(pdl_file: str, data_dir: str = "data/") -> pd.DataFrame:
"""Load raw People Data Labs data stored locally"""
return pd.read_json(Path(data_dir, pdl_file))
def employee_count_by_month_df(pdl_data: pd.DataFrame) -> pd.DataFrame:
"""Normalized employee count data"""
return (
pd.json_normalize(pdl_data["employee_count_by_month"])
.assign(ticker=pdl_data["ticker"])
.melt(
id_vars="ticker",
var_name="year_month",
value_name="employee_count",
)
.astype({"year_month": "datetime64[ns]"})
)Seth Stokes
08/07/2024, 3:32 PMmiek
08/07/2024, 9:16 PMThierry Jean
08/07/2024, 10:01 PMmiek
08/08/2024, 5:02 AMJean-Paul Abbuehl
08/21/2024, 8:26 AM