Hi! Is there a possibility to show unstructured data like images, pngs and so on in Datahub?

Hi, you can add images to descriptions- could you clarify how you’d like to show this? You could also ingest from S3 for generic storage

Hi Paul, how would I do that? with which source? some background info from the other thread: some of our data sources are unstructured, and it would be great to show them along the structured sources. specifically images for example. it would be enough to manually ingest information if that is possible as well. i know it won't show column or schema information, but it would be great to show that it exists, which tags it has and so on...

@nutritious-megabyte-12020 so you want to have metadata for image by image? Not sure how you fit an individual image to concept of a dataset. If you can fit the unstructure source in concept of dataset, you should be able to manually/automatically emit the metadata to datahub.

@modern-artist-55754 no, more like as a whole. i understand that i won't get any schema/detail information, but how would i manually add information like tags, descriptions and stuff? do you have any documentation links? something cool on top would be some source type which would show the filetype, mabye average fileisze, dimensions and stuff, but that would be optional for me

https://datahubproject.io/docs/metadata-ingestion/as-a-library/ You can use emitter to send metadata to datahub. I am just thinking what kind of dataplatform it should be called. Because yours is none standard.

i first thought it would be the file source, so in my opinion a file source would be amazing which just looks into a dir, and lists filetypes, and if possible further information about those files like size, in case of images dimensions, ...

I think you can just make up name for a platform. Not sure how it will show up on the UI though