https://linen.dev logo
#feedback-and-requests
Title
# feedback-and-requests
j

Jeff Crooks

01/04/2022, 3:08 PM
Is there any way to scan more than 10k documents in the Mongo connector?
the schema is different across the documents over time (dont even get me started) , and scanning 10k isnt returning the "full" schema
Hi @Jeff Crooks this is an "optimization" we made to make schema discovery faster, and this limit is hardcoded. The change was made in version
0.1.9
(PR). The ideal solution would be to allow this limit to be customized. I opened an issue for this. Feel free to upvote it. A workaround could be to downgrade your Mongo connector to
0.1.8
thanks - root cause is the insanity of the people who are writing to mongo, but upvoted!
Changing your source database system could be a workaround indeed 😛 😄
been in progress for multiple years 🙂
so i have high hopes....
Good luck then!
the "actual" schema is in a separate table
would it be useful to have an “infer schema” flag which when turned off just outputs untyped JSONs?
instead of doing the whole inference thing
j

Jason Ofua

01/04/2022, 8:00 PM
its less about the types and more just missing the columns themselves
3 Views