https://docs.all-hands.dev/ logo
Join Slack
Powered by
<@U090ER7JPHA> Read OpenHands/software-agent-sdk a...
# questions
l
@nutritious-balloon-65098 Read OpenHands/software-agent-sdk and explain how to attach Local image file as ImageContent in a conversation. An example implementation with image url(github url) is available at software-agent-sdk/examples/01_standalone_sdk/17_image_input.py. But it doesn't explain how to attach local images into the conversation.
n
I'm on it! Vivek Vijayan can track my progress here.
I'm on it! Vivek Vijayan can track my progress here.
Summary I successfully analyzed the OpenHands software-agent-sdk repository and provided a comprehensive explanation of how to attach local image files as 
ImageContent
in conversations. Key Answer: The critical insight is that 
ImageContent
expects URLs, not file paths. To use local images, you must convert them to data URLs using base64 encoding. What I provided: 1. Core Solution: A utility function that converts local image files to data URLs:
Copy code
def local_image_to_data_url(image_path: str) -> str:
       # Read file as binary, encode as base64, create data URL
       return f"data:{mime_type};base64,{image_base64}"
2. Complete Examples: ◦ Single local image attachment ◦ Multiple local images in one message ◦ Mixing local and remote images ◦ Integration with existing SDK patterns 3. Comprehensive Documentation: Detailed README explaining the concept, implementation, error handling, and best practices 4. Reusable Utilities: Helper functions for easy integration into projects Comparison with existing example: • 
17_image_input.py
uses: 
ImageContent(image_urls=["<https://github.com/...>"])
• Local images require: 
ImageContent(image_urls=[local_image_to_data_url("local_file.png")])
The solution maintains the same 
ImageContent
interface while adding the necessary base64 conversion step for local files. This allows developers to seamlessly work with both local and remote images in their OpenHands SDK conversations. <sub>View full conversation</sub>
Summary: How to Attach Local Images as ImageContent in OpenHands Agent SDK After exploring the OpenHands software-agent-sdk repository and analyzing the existing 
17_image_input.py
example, I can explain how to attach local image files as 
ImageContent
in conversations. The Solution The 
ImageContent
class accepts local images by converting them to base64-encoded data URLs. While the existing example uses remote URLs like 
<https://github.com/>...
, local images work by encoding them in this format:
Copy code
data:{mime_type};base64,{base64_encoded_data}
Key Implementation
Copy code
import base64
from pathlib import Path
from openhands.sdk import ImageContent, Message, TextContent

def local_image_to_data_url(image_path: str) -> str:
    """Convert local image to data URL for ImageContent."""
    path = Path(image_path)
    
    # Determine MIME type from extension
    extension = path.suffix.lower()
    mime_types = {
        '.png': 'image/png', '.jpg': 'image/jpeg', '.jpeg': 'image/jpeg',
        '.gif': 'image/gif', '.webp': 'image/webp'
    }
    mime_type = mime_types.get(extension, 'image/png')
    
    # Read and encode image
    with open(path, 'rb') as f:
        image_data = base64.b64encode(f.read()).decode('utf-8')
    
    return f"data:{mime_type};base64,{image_data}"

# Usage - replace the remote URL approach:
# OLD: ImageContent(image_urls=["<https://github.com/...>"])
# NEW: ImageContent(image_urls=[local_image_to_data_url("my_image.png")])

message = Message(
    role="user",
    content=[
        TextContent(text="What do you see in this image?"),
        ImageContent(image_urls=[local_image_to_data_url("my_image.png")]),
    ]
)
Evidence from SDK This approach is already used within the SDK itself. The browser_use tool in 
openhands-tools/openhands/tools/browser_use/definition.py
converts screenshots to data URLs:
Copy code
data_url = f"data:{mime_type};base64,{self.screenshot_data}"
llm_content.append(ImageContent(image_urls=[data_url]))
Complete Examples Created I've created several files demonstrating this approach: 1. 
simple_local_image_example.py
 - Minimal working example based on 
17_image_input.py
2. 
image_utils.py
 - Reusable utility functions with error handling 3. 
README_LOCAL_IMAGES.md
 - Comprehensive documentation with best practices 4. 
test_image_encoding.py
 - Verified the encoding functionality works correctly Answer Summary To attach local images as ImageContent: Convert your local image files to base64-encoded data URLs using the format 
data:image/{type};base64,{encoded_data}
and pass these data URLs to the 
ImageContent(image_urls=[...])
parameter, exactly like you would with remote URLs. No changes to the SDK are needed - this works with the existing architecture. <sub>View full conversation</sub>
Open in Slack
PreviousNext
Powered by