CACHEBOX question?? Im not sure if this is the wro...
# box-products
s
CACHEBOX question?? Im not sure if this is the wrong way to think about caching. But here we go. I have API requests that I am using throughout client based SAS product, some of which are intensive database requests that take longer than i would like (1-2 seconds), I have optimized the queries to the best of my ability and now I would like to entertain caching. My problem is it appears in caching most cached items are invalidated based on usage or time. However I need them to be invalidated by triggering an update when the clients settings are changed. Is this an anti-pattern or just not a common workflow. I imagine most caching workflows are for front facing sites that don’t change often. Either way a good workflow to use for invalidating on a user changing their data would be super helpful to me?
p
well I would assume you would want it invalidated upon them changing the data. I typically do this upon settings changes then the next call for that data gets cached again.
b
@Scott Steinbeck There are two ways something can leave the cache • the eviction policy removes it (based on whatever logic you want) • You remove it manually because your app code has determined it's stale
You appear to be asking about the second bullet, and perhaps conflating it with the first
The short answer is you do this however the heck you want, lol
What I recommend doing however is • name your cached items with a predictable key scheme • Register and announce custom interception points any time you update data • Create interceptors that listen to these announcements and clear the related keys from the cache. Depending on your cache provider, you may be able to use the clear by snippet methods if you need to wipe more than one key at a time based on a pattern
This is what @lmajano implemented in ForeBox and it works pretty well. We're able to cache for a long amount of time because we know the cached items will be cleared right away when the data changes
We announce our custom interception points from inside our model so it doesn't matter if the update happened as part of the website UI or an API call.
p
Yep, thats my approach, using an interceptor and having good naming conventions for cached items. Brad is just better at giving thorough explanation of course lol
👍 1
l
This is also implemented in contentbox
👍 1
You can see all the layers of caching with predictable keys
And interceptors for cleanup
s
In a meeting, thank you for all the feedback, ill read through it in 20 minutes
e
Just a note on the points above, when caching something forever (time-wise) and evicting it when another action happens in your application is perfectly fine. Make sure the cache provider you are using does not evict items out if they are not accessed within a certain amount of time (unless you want that).
s
Thanks all, i will dig into content box and try implementing a similar pattern. @bdw429s on your point about clear by snippet, is that something that is available in the
ConcurrentStore
or something I would need to implement myself?
b
@Scott Steinbeck
expireByKeySnippet()
is part of the
CacheBoxProvider
so it should work with the concurrentstore.
It would not work, for example, with an external provider like couchbase, redis, etc unless someone implemented it.
The cacheboxprovider is the provider that encompasses all our Ortus-built "stores" such as disk, JDBC, concurrent, concurrentsoftreference, and blackhole
s
oh gotcha, ill have to check out the source, I missed that one in the docs
does concurrentstore use lucee cache by default if running a lucee sever?
b
Well, it's not part of the
ICacheProvider
interface, so you can't asssume it will be there unless your provider is a subclass of
AbstractCacheBoxProvider
does concurrentstore use lucee cache by default if running a lucee sever?
No. It uses... a conccurent store 😉
Use the
LuceeProvider
if you know you're on Lucee and want to use a Lucee cache
If you need to be engine agnostic, use the
CFProvider
s
let me ask the question a different way, for my use case, since i am not going the way of couchdb or redis, is there a preferred/more performant cache to use?
you beat me to my question,
b
Both of those use the underlying CFML
cacheGet()
cachePut()
BIFs, etc. The Lucee one just supports Lucee better
I would recommend an out of process cache if you actually want this to perform well in production
👍 1
out of process caches (redis, couchbase, etc) are non-volatile across restarts and can be shared by an number of servers
If you have tons of RAM and just want a quick and dirty in memory cache, then the concurrent soft reference is fine and the fastest
The JDBC/disk stores of the cachebox provider are external, but they are not optimized very well for indexing lots of items
s
ok good to know!
b
An in memory cache also eliminates issues serializing complex objects which can be a blessing and a curse
p
Just spin up Redis, its badass, fast and efficient
s
if most of the things i will be caching are json api responses, does using the
concurrent soft reference
vs.
concurrent store
have any functional difference?
b
If you don't care if a given item drops from the cache, no
The only difference between the two is that the soft reference store will allow the JVM to garbage collect items from it if the JVM is plumb out of memory
Which means an item COULD disappear from the cache on its own (even if the expiration was set to 0)
s
oh ok, i see the advantage
b
The concurrent store will just fill up your heap until it dies!
That said, the default num of items like VERY small-- like 300 or something
You'll want to tune that to be much much higher based on the average size of the cached items
This is where the hits/misses graph from the cachebox tab in the debugger panel are super handy to see how your cache is actually doing
s
oh good to know
b
Luis set those default sizes many years ago
You could probably have 5000 items in your cache with no issue with a few Gigs in the heap
Assuming each item is only a few KB
s
beautiful
is there a limit on the cache key name? It seems like based on the method the a cache key could be composed like
mysite-user-145-api-fieldslist
and then i could call
expireByKeySnippet('mysite-user-145')
to expire all keys associated with the user, is that correct?
b
Pretty much
The key name limit depends on the backend cache in use. For the concurrent, it's just a java.lang.String so there is no limit
s
is there a benefit to using
expireByKeySnippet
over
clearByKeySnippet
b
expire just marks the cache key as expired so it will be reaped the next time reaping runs. Clear actually removes from the cache right then
There may not be an effective difference TBH since I'm fairly sure an expired item will present as just not existing at all when you try to get it, but you'd have to test.
Expiring the item would obviously cause it to still be in memory and show in the debugger cache panel until it was reaped
s
I would imagine the work might be less since its lazily deleting the cached item unless its reaped
i guess its a balance of filled up memory vs work at the current momemnt
b
Correct, actually removing an item from the cache involves update the indexer pool metadata
s
cool, that makes sense
given this idea with an api, is it possible to implement something in my baseHandler that i could run before a show/index route via a preHandler or Around Handler to implement caching in one shot?
im sure i would need a way to opt out of course.
I feel like since im sending an ID to the create/update/delete methods i should be able to similarly invalidate the cache in a similarly standardized way
but maybe im underthinking it
b
That's called ColdBox's event caching 🙂
It caches the entire event based on the incoming route/rc and caches the full response!
The coldboxcacheboxproviders already have methods to clear events by snippet
s
perfect, ive recreated the wheel 😛
so really i just need to add a consistent way to expire it
quick question on the Life-Cycle Caveats, if preProcess interceptor does not run on a cached item, will cbSecurity still block logged out/timed out users?
b
No
you can include the userID/etc as part of the rc so each user gets a different cached item (which reduces the effectiveness of it) but it's assumed the route is deterministic-- meaning any subsequent requests with the same details will expect the same result
If you still want to run custom auth code on every request and then have fine control over what data is cached at the service layer, then you need to roll your own.
s
oh ok
yeah looking at the workflow I see that i would have to actually hit the handler before checking to respond with a cached object. unlike event caching which fires way earlier
oh wait im dumb, i read this wrong
The ONLY interception points that will execute are preProcess and postProcess
so im good