http://coldfusion.com logo
Join Slack
Powered by
Anyone done a snowflake id generator? (bit harder ...
# cfml-general
a
Anyone done a snowflake id generator? (bit harder than I thought from scratch!) Happy to just use a java solution if people can recommend - thanks.
d
Do you mean the UUID to prevent the query from running twice, or something to do with OAuth?
a
d
Assuming we're talking about the data warehouse stuff, I'd probably just use the official java API
Oh, lol, I thought you meant something else
a
No problem - I've stumbled across the data warehouse stuff as well via googling 🙂
Basically I need a unique id across databases. so could use a uuid but snowflake ids seem like a nicer approach as they have the timestamp built in and you can backwards engineer them to get more info about how they were generated
d
yeah that is cool, and a great project for some type of extension or plugin (I don't know of any existing ones offhand)
there are lots of examples on github of doing it in java, fwiw
a
Thanks - I did take a look at them and then figured using an "proven" jar would probably make sense rather than reinvent the wheel or picking a random one of the internet (what could go wrong!)
d
Yeah, the callicoder one seems the most popular, and it's in maven, so that's nice if somehow there are updates (tho also a downside if the updates are nefarious! 😉)
👍 1
a
I was looking on Maven 'Yitter' is the most popular on there. Found via https://mvnrepository.com/search?q=snowflake&sort=popular
The 
idworker
one has vulnerabilities in it
👍 1
d
I don't understand how snowflake4j can do this: 
protects from non-monotonic clocks, i.e. clocks that run backwards
. If the system time jumped backwards, what does it do to prevent duplicating IDs?
a
it throws an error (or that's what others do that I've looked at)
so it protects against it happening but doesn't automatically correct it
👍 1
d
That makes sense I guess. So if you're using them as IDs in, say an error log, you just don't record anything until the time discontinuity has passed?
a
Looking at the yitter one it looks like it attempts to handle clock rollback
(via google translate) How to handle time rollback 🔶 When the system time is dialed back, the algorithm uses the reserved ordinal number of the past time sequence to generate a new ID. 🔶 The ID number generated by the callback is at the front by default, and can be adjusted to the back. 🔶 Allow time to be dialed back to the preset base of this algorithm (the parameter is adjustable).
It also can handle horizontal scaling of generator ids using redis
p
screwed with creating this based on some java ones and yea CF would take some work; prob best to just do a wrapper for running the java version
Everything puking me back a negative integer
a
Thanks for looking - I had a go but bitOr pukes on big integers, even when I explicitly cast them as 
java.math.BigInteger
p
yea that and problems with bitshln
Oh shoot, I think I have it
🎉 1
@aliaspooryorik so does it iterate every refresh or only based on time?
I get a value that when I use the Discord Snowflake timestamp converter returns the epoch I gave
a
it uses milliseconds for the time part and then a simple sequence where collisions in the same millisecond. The other parts are the generator id.
p
1268237743675047936
is the time right now
a
So the first part of that is the time in ms
The last part is the generator id and sequnce
p
So basically I have this working, the node (machine) is hard coded and sequence # not sure how that works like is it supposed to just be a sequencing #?
a
Cool!
This is as far as I got
Untitled
Untitled
p
What is the sequenceMask do?
a
Just makes sure it fits in the number of bytes
If it overflows then it makes it wait for the next millisecond
I largely ported this from java code so can't claim any credit
p
yea same some of this is just a kicker to try and plus you got me interested in what a snowflakeId was
yea I have it working now so that it doesnt generate the same ID on the same node and sequence
😯 1
image.png
So is yours functioning?
a
no - I didn't get it working. Stopped when I hit the "cannot fit inside an integer." issue. Nice work getting it working!
d
Are you guys publishing what you came up with for future generations?
p
I am going to make a public repo for this
👍 1
❤️ 1
d
Awesome possum!
p
Here is my first open source project, though basic it is very useful and something I have not seen in the CF realm. https://github.com/patrickfusion/cfSnowflakeID
🎉 4
🚀 1
a
Seems a bit odd to call it a ColdFusion project when it's only tested on Lucee. Maybe CFML project?
😁 1
😂 1
a
opened a PR with tests and ACF support https://github.com/patrickfusion/cfSnowflakeID/pull/1
❤️ 1
passing on Lucee and ACF 2023 - failing on BoxLang though!
🤘 1
d
Hmm, only ACF 2023? I don't know if that's enough versions of ACF to truly qualify as a ColdFusion project…😆
p
@Adam Cameron yea yea, well John has already fixed it to fit ACF and Lucee
❤️ 1
a
He's a star.
☝️ 1
👍 1
Might be worth looking at using final static variables for your constants rather than co-opt cfproperty's square peg for that round hole.
👍 1
Also I think you probably wanna throw an exception if that lock timeout is hit?
👍 1
Default behaviour is to skip over the locked code, I think?
a
Good idea about using final static. The tests fail on ACF2018 anyway so no need to worry about maintaining support for 2018
p
Yea agreed @Adam Cameron I wont have time today but if a pull comes then np
a
looks like 
throwOnTimeout
for lock defaults to true
👍 1
3 Views
Open in Slack
PreviousNext
Powered by