https://www.boundaryml.com logo
Powered by
🟡 Seeking Advice on Creating Synthetic Data with ...
# questions
b
Not sure if this is the right channel, but here we go. I need to create synthetic data, or copies of real data, including the actual formatting, strucutre and copying the choice of words etc. the content part, is something that I get fairly okay now, but the visual layout, structure etc. is something that I have no idea on how to do best. Sure I could use inpainting, and do it step by step, but this is just stupidly slow... So has anyone done this before? Input data is diverse set of text documents, with tabular data mixed in. Mostly german legal language (hence I can not use the original docs)
v
sadly its going to be quite slow here 🙁
there's no good way except for: 1. create the synthetic content as text only first 2. write a different ai pipeine to then take text -> structured layout in some well establishing program. (this pipeline can produce python code that produces the pdf etc)
but since you are trying to check for layouts, you likely will need N different layouts, so you'll have to pay for each of those
what i would do is: use my original german content to produce a python / etc file taht can produce that exact layout for that exact text
then feel the synthetic text and say now use this text in this python layout file
not sure if that makes sense?
b
It does, I think for repetable layout it fully makes sense, but for other layouts, I think I will just go and try an inpaint version. So Vllm --> BBox --> inpaint that area --> vllm validate . Not fast, token expensive but for like layouts that will not be there often, or not worth to replicate might be the best case. essentially I take an original piece, replace everything that could identify a person, or an event and replace it with soemthing AI generated that is coherent. Eg. court case, similar events, but slights different. But keep the orginal layout etc. and rough story arc, and laws touched. to keep the quality the same,. Process this air gapped at a firm, so I am not violating any gdpr or §203 StgB clauses, that make germany such a Pain to work with AI 😉
thanks for the input @Vaibhav (BAML 🐑)
If / when it works happy to showcase it at some point in the sheep council if you like
v
would love to check it out when its ready!
we can consider opening up demo slots in each sheep council meetin
b
also it is sunday, get some rest 😉
2 Views
PreviousNext
Powered by