curious: why are you interested in moving off of N...
# getting-started
l
curious: why are you interested in moving off of Neo4j?
h
Main reason is probably the lack of (reasonably priced) managed solutions among our trusted providers (AWS, Aiven).
Also simplifying the deployment by using ES is a plus
1
b
cost is the primary reason we've heard
but yes separately reducing operational complexity
m
@lively-judge-30357 @orange-night-91387: DGraph would be a great contribution. The two of you should talk. I'm happy to facilitate and help with design input.
o
I have an initial POC that I've written up in our fork, but the performance metrics I've been getting aren't as optimistic as I had hoped. Been still fiddling with the implementation to see if I can get it faster, but have been focused on other higher priority things for us currently. I can convert it over to mesh with the OSS version and put it up in a branch when I get a chance so I can get more eyes on it. Haven't tried reaching out directly to the DGraph team for implementation suggestions yet either. There aren't a lot of open implementations for references esp. in Java.
Our primary concern is ingestion speed, Graph DBs are notoriously slow for streamed data sources. Also have found this whitepaper, not sure if anyone has put out any actual implementations of it since it was relatively recent: https://arxiv.org/pdf/1905.08337.pdf , it still uses a more batch-style approach though for speed-ups.
l
Enrico has been doing some performance work on his Spark-DGraph connector. DGraph seems to scale pretty decently
at least, for what he was doing.
o
Yeah that was a big draw, that it is designed distributed first as opposed to Neo4J which is limited by its master-slave architecture. I haven't hit any problems with scaling the size of the data, just didn't get the several orders of magnitude ingestion speed boost I was hoping for with my connector to the current DAO logic 🙂 when I last checked the upserts still got really bogged down (only performing slightly better than Neo) when I hammered it with a ton of events. Most of my tests have been local though and it totally could just be me poorly implementing the connector or just not properly configuring the DGraph servers. In our deployments Neo4J is a big bottleneck that prevents us from ingesting lots of events in a timely fashion. Big bursts can slow us down to hour long updates which makes the graph not as usable.
m
@orange-night-91387 does it speed up if you are doing appends (basically adding new edges + nodes to the graph) or removing edges / nodes. Without doing read-modify-write?
o
IIRC DGraph was performing much better on modifications of existing nodes than Neo, but they both performed less than ideal on the initial load, but it's been awhile since I ran the tests.