Oh, **, misread. Rink's dataset is here: https://w...
# feedback
c
Oh, **, misread. Rink's dataset is here: https://www.goedel.io/p/tft-performance-methodology
s
I have Dendronized a 10,000 note example of his here: - https://gitlab.com/docsgarden/dg-tft-perf-test-10000 Though, looking at his performance testing examples, he tests importing/exporting itself, not just how navigating the contents of the vault work. @kevins8 I mention in the README of this Dendronized repo some hurdles I have run into. For example, exporting with the export pod v2 at the moment doesn't seem to work for me. I have the export pod config saved to the repo at the moment, so you should be able to replicate. I just get locking up when trying to select in the menu (so it could be working, just taking time?). Haven't tested the CLI to see if that works any better. I have also tried importing natively in Dendron, the Markdown and the JSON examples, but I couldn't get them to work (I used custom Python code to work some magic and import the content instead). Source repo I converted the notes from: https://github.com/rcvd/interconnected-markdown - Example: Import Markdown contents from
Markdown/10000s/10000
(this is what I ran my code against and moved into a Dendron vault) - Example: Import JSON contents from
JSON/10000s/10000.json
I haven't had time to delve deeper, but I could make repo variants of his 2000 and 5000 note examples that he uses in the repo, but would like to have the hurdles figured out first so that I am not duplicating anything like incorrect configs to the other repos.
c
Nice!
More benchmarking 🤩
Also a good sales pitch for Dendron, given how badly Obsidian murders most other tools
If it can compete
k
thanks for doing a first pass - a little behind messages because of coming back from a conference but will be looking at this this week!
took a look at the export - there's currently an issue with bulk exporting lots of notes. I have a pr in to fix the bulk export issue: https://github.com/dendronhq/dendron/pull/3493
s
I realize that doing a perf test on such a giant repo first probably was the wrong place to start. I'm going to Dendronize vaults of the other smaller sizes for testing, since these same features I know have been working fine as expected on smaller vaults I've worked with previously. It would make it easier to see when certain vault sizes run into problems. This will help with testing things like: - Export - Import - Dendron Doctor commands - etc. I think, what happens for me, is that it's hard to tell when certain commands are working in the background, and will eventually complete, or when a command may be hanging (due to lack of progress bar, log output, etc.). Maybe doing the tests via CLI would provide better output, if there is something like a verbose and/or debug flag? Do you have recommendations for how I can check things like this and potentially provide better information to the Dendron team if I open Dendron issues related to performance/scaling?
k
the change i made to export will now show progress on export. we need to do something similar on import. if you do it with CLI, you'll see verbose logging To enable debug logging, set
LOG_LEVEL
to
debug
.
Copy code
sh
LOG_LEVEL=debug dendron {your-command}
s
That's awesome to hear, looking forward to more testing 🙂