https://solidus.io logo
Join Slack
Powered by
Hey everyone. I'm building a shop with 100k produc...
# general
d
Hey everyone. I'm building a shop with 100k products and 200k variants. Started to customize the frontend but having issues with performance (requests taking 1000ms+) for a simple products#index page with 12 records. Miniprofiler shows a lot of db queries so it's probably an N+1 issue and I was wondering how to modify controller code. Is there a preferred way to do this?
e
if that's not enough, I'd say build your own products controller from scratch, inheriting from Spree::StoreController and modify the routes
j
Alternatively, you can use decorators (what the Rails docs call overrides) to modify the existing controller.
d
Thanks for the quick replies! I'll try overwriting the searcher and see if that solves the issue.
Ok, found the following two things that are causing an issue: 1. Active Storage Records are not eager loaded. I tried to fix this by including 
images: [attachment_attachment: [blob: :variant_records]]
but it's not helping. Any ideas? 2. Removing pagination removes 500ms from page rendering.
j
Yeah, it's common to need to eager load various associations.
e
I'd install scoutapm locally and run in development mode and see the backtrace, it might be referring to 
master_including_variangs
association rater than just 
variants
then it blows the eager loading
d
@Edwin Cruz thanks for the PM. It's a Rails issue. There is a merged PR which has not been released yet. I solved it by monkey patching ActiveStorage and including 
images: { attachment_attachment: { blob: { variant_records: { image_attachment: :blob } } } }
. Reduced page rendering by another 200ms 👍 I'm now at 350ms without pagination in dev env which should be fine in production. Now trying to figure out how to speed up pagination.
k
If you have time to submit some PRs to Solidus with those improvements, it would be awesome! 😇
d
Will do 🙂 Currently fighting with the query speed of a simple 
Spree::Product.all
that takes ~130ms with 100k records: 
Spree::Product Load (127.0ms)  SELECT "spree_products".* FROM "spree_products" WHERE "spree_products"."deleted_at" IS NULL
Although that's not really the issue when using 
limit
Main culprit seems to be the 
@products.maximum(:updated_at)
query in the 
cache_key_for_products
method. Takes 150ms to finish.
I also want to include the total products count on the products pages. Using a count query in the searcher adds an additional 450ms.
Not sure how to speed things up yet but will update when I find a solution.
Note: Enabling 
dev:cache
and ignoring the 
@products.maximum(:updated_at)
and 
count
query would reduce page rendering speed to 50ms.
k
I'm afraid that as is, 
maximum
is required when cache is active because it's what tells Rails to invalidate the cache (when you update any product). You can probably find other ways to invalidate that cache though
d
The only solution I can currently think of is creating a custom 
Spree::Preference
to act as a cache for 
maximum
and 
count
. Would this be considered as a misuse of 
Spree::Preference
or should I go with something else like https://github.com/huacnlee/rails-settings-cached?
Ok, I solved the 
maximum
query issue by using 
@products.map(&:updated_at).max
instead of the additional 
maximum
query.
Only issue left is the 
count
query.
k
@products.map(&:updated_at).max
uses Ruby, and with a lot of records it could be much slower than a query, isn't it?
d
Sure, but the results are already paginated. So iterating over 12-25 records is much faster.
Using this approach I'm now at 
Completed 200 OK in 106ms (Views: 51.4ms | ActiveRecord: 47.5ms | Allocations: 44892)
for cached pages.
Uncached is 
Completed 200 OK in 601ms (Views: 121.9ms | ActiveRecord: 471.4ms | Allocations: 91574)
due to the count query.
k
very impressive, congrats!
d
Managed to "fix" the count issue as well by just not using it. Kaminari has a 
without_count
option which disables it (https://stackoverflow.com/a/43267340/587320). This brings down uncached page rendering time to ~230ms.
Using pagy instead of kaminari brings the whole thing down to ~170ms uncached.
👏 1
3 Views
Open in Slack
PreviousNext
Powered by