https://github.com/stitchfix/hamilton logo
Join Slack
Powered by
Hi Team, Is there a way to use class methods inste...
# hamilton-help
p
Hi Team, Is there a way to use class methods instead of modules with method while executing driver?
s
@Prasanta Midya do you mind sketching out your use case? In python, functions in a module behave similarly to a class with static/class methods.
Or if you’re looking to reuse those functions — you’d need to “wrap” them in a hamilton function.
p
Hi @Stefan Krawczyk I have an existing RAG application. All existing codes are class and instance method oriented(for ex. index creator, tokenizer, embedder, doc_id generator)I am trying to migrate to hamilton framework. So any help will be appreciated
Also I have multiple implementations of same component. For ex. for embedder, I have local embedder that uses huggingface's local sentence transformer and openai's async embedder. This I achieved through factory design pattern, abstract class and two implementation classes. Based on env I can change my embedding model. How can I achieve these in hamilton framework?
s
so I would first draw out your process, where each “step” would correspond to some part of your logic. These can then become “functions” that you connect together the Hamilton way: E.g.
Copy code
def raw_data(...) -> ...:
   # your logic
   return data

def embedder(env_value: str) -> ...:
   # your logic
   return embedder

def embedded_data(raw_data: ..., embedder: ...) -> ...:
   # your logic 
   etc
In Hamilton function names typically are “assets”, i.e. nouns/variable names, that way when you read a function you know (a) what it creates, and via the function arguments (b) what it depends on. You could simply wrap existing logic with Hamilton, or move some of the logic out of classes into functions like the above. This is up to you. Without seeing your code hard for me to judge 🙂
If you haven’t done https://www.tryhamilton.dev/category/hamilton---basics-5-mins and https://www.tryhamilton.dev/category/hamilton---constructs that will help provide some constructs that could be helpful.
oh and I assume this code is running in some batch/script process, instead of in a web-service?
p
I am running via webservice. documents arrive for ingestion and then I am indexing those in vector db with tokenization and embedding. I have a doubt. If I use hamilton framework to obtain connection to vector db how can I ensure connection is closed with/without exception?
s
@Prasanta Midya oh yep — are you using async?
p
Yes. methods on fastAPI controller layers are async coroutines
s
I have a doubt. If I use hamilton framework to obtain connection to vector db how can I ensure connection is closed with/without exception?
How do you do that now? Hamilton doesn’t preclude you from writing the inside of a function however you want. You can catch exceptions, have finally blocks, etc.
In any case, you might like to look at two things: 1. https://blog.dagworks.io/p/ner-powered-semantic-search-using (for some ideas on code flow/structure) 2. https://github.com/DAGWorks-Inc/hamilton/tree/main/examples/async (for running in fastapi)
p
Currently I am doing like this
class OpensearchClientFactory(DatabaseClientFactory): """ Opensearch client factory to obtain a new OpenSearch DB connection """ def __enter__(self): return self._client def __exit__(self, , _, _): self._client.transport.close() def init_client(self): opensearch_host = os.getenv("OPENSEARCH_HOST", "localhost") opensearch_port = os.getenv("OPENSEARCH_PORT", "9200") opensearch_username = os.getenv("OPENSEARCH_USERNAME", "admin") opensearch_password = os.getenv("OPENSEARCH_PASSWORD", "admin") ......... return OpenSearch( connection_class=RequestsHttpConnection, hosts=[{'host': opensearch_host, 'port': opensearch_port}], http_compress = True, http_auth=(opensearch_username, opensearch_password), use_ssl=True, .... ) Then I use with OpensearchClientFactory() as db_client: .......
s
how big is the block under 
db_client
?
p
its quite big. I am now using dfs to invoke a chain of functions under this
I am little bit doubtful about try with resources to close db connection if I use hamilton. Other than than I can use hamilton async driver
s
ok so what you can do is somehting like this:
Copy code
my_driver = # instantiate the driver

...


with OpensearchClientFactory() as db_client:
   outputs = await my_driver.execute(OUTPUT_REQUESTED, inputs={"db_client": db_client, ...})
you just pass in any resources to your Hamilton code
p
Yes something like this. hamilton framework looks very interesting. Let me try this. Thank you
👍 1
2 Views
Open in Slack
PreviousNext
Powered by