set the channel description: This is an channel backed by a bot. Ask it questions about Hamilton!

What is the Hamilton UI?

What's your favorite Hamilton feature?

What are the benefits of using Hamilton or Dagster? When should I use either?

Give me a brief Python code example of how to use the

@subdag

function modifier. It should help me write more expressive data transformations using Python functions.

Provide me the documentation references for Parallelizable and Collect

How do I get started?

How do I get started?

What are the best resources on

@subdag

?

What are the best resources on

@subdag

?

How do I use

@subdag

?

This message was deleted.

How do I use

@subdag

?

where do I find documentation an examples for integrating hamilton with dagster or flyte?

How can Hamilton be categorized in comparison with Kedro? Which purposes should each of them preferably be used for?

If I want to store the contents of an ipywidgets Textarea widget in a text or arrow file, are the load_from and save_to decorators a good choice for use within a Jupyter notebook? How would that look in code with the widget separated into its own cell, for example?

I'm thinking of combining Hamilton with Dagster. Particularly for scheduled contiguous data ingress from a web API, that provides up-to-date data. Every week or month, the newest data would be downloaded and stored for further processing with Hamilton. Question 1: I wonder, if Dagster is actually necessary for this, however. I've written out the chunked downloading, including debouncing mechanism (back-off) in Python already. Currently I'm storing with sqlAlchemy to MariaDB, but considering parquet for raw data storage, too. Eventually,

How can I retrieve cached artifacts after a run? I have already discovered

hamilton.caching.cache_key.create_cache_key

and

hamilton.caching.stores.sqlite.SQLiteMetadataStore

but I'm having a hard time building the right cache_key for various nodes from the last run id.

Hi. I'm using wrenai and I'm getting the following warning:

db_schemas token count is greater than 100,000, so we will prune columns

And this error:

wren-ai-service-1  | Traceback (most recent call last):
wren-ai-service-1  |   File "/app/.venv/lib/python3.12/site-packages/hamilton/async_driver.py", line 122, in new_fn
wren-ai-service-1  |     await fn(**fn_kwargs) if asyncio.iscoroutinefunction(fn) else fn(**fn_kwargs)
wren-ai-service-1  |                                                                   ^^^^^^^^^^^^^^^
wren-ai-service-1  |   File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 256, in sync_wrapper
wren-ai-service-1  |     self._handle_exception(observation, e)
wren-ai-service-1  |   File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 517, in _handle_exception
wren-ai-service-1  |     raise e
wren-ai-service-1  |   File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 254, in sync_wrapper
wren-ai-service-1  |     result = func(*args, **kwargs)
wren-ai-service-1  |              ^^^^^^^^^^^^^^^^^^^^^
wren-ai-service-1  |   File "/src/pipelines/retrieval/retrieval.py", line 301, in construct_retrieval_results
wren-ai-service-1  |     columns_and_tables_needed = orjson.loads(
wren-ai-service-1  |                                 ^^^^^^^^^^^^^
wren-ai-service-1  | KeyError: 'results'

I'm slowly porting an existing codebase to Hamilton and I'm trying to access the result of one of the workflow step. The result is a DataFrame. I made a function to return it but when I access it from the execute function, each column is a key in the result dictionary. Is there a way to simply grab the whole dataframe?

archived the channel