https://tfos.co logo
Join Discord
Powered by
My custom product analytics dashboard
# forum
j
In a rare fit of productivity, I have already written the newsletter for next week. Sneak peek: https://tfos.co/p/product-analytics-dashboard/ An aside/more publishing habit navel-gazing: maybe I should continue to write more directly about my work, as much as possible (hard to do that on weeks when I do consulting, on the somewhat bright side at least I only have one client), and write think-pieces somewhat less often relatively. And then structure the writing as part of my "regular" work, which is what I've done today, rather than e.g. having monday be "writing day", or trying to schedule 30 minutes of writing per day as I've thought about doing recently. And then the thinkpieces can pop out naturally whenever they're ready.
l
Love to see it!! 🙌 Building an analytics board is coming up fast for me (and if Indie Hackers is to be believed, it’s nearly as important as the actual product). I’m somewhat curious about the code, but very curious how to structure the internal database such that you can track and query stuff like view sources / onboarding / dau without Google Analytics or Plausible.
j
I'll put the dashboard code on a gist, definitely try messing around with that! for storing the data--some of it is just what's already in the database for the sake of the application. I have a 
query-events
function which takes stuff like that from the database and then returns a collection of "event" documents in the format that the dashboard code expects. e.g. I have a :user/joined-at key on user documents, so in part of the function I load all the users and return something like 
{:user #uuid "...", :type :signup, :timestamp #inst "..."}
for each one. The app also already records your reading history, which is what I use to count someone as active or not, so I didn't need to add anything to the db for that. For other apps you might need to add stuff though--basically you need to decide what would make a user count as "active" on a particular day, and then figure out if you need to start recording more data to be able to query for that. For page views (both on the landing page and inside the app) I have a bit of javascript that takes any query params from the url (so it'll include stuff like 
ref
, 
utm_source
etc), inserts a few extra things like 
document.referrer
, and passes it to an endpoint:
Copy code
{
  let params = new URLSearchParams(window.location.search);
  if (document.referrer) {
    params.set('referrer_url', document.referrer);
  }
  let landing = document.querySelector('input[name="landing"]');
  if (landing) {
    params.set('landing', landing.value);
  }
  params.set('path', window.location.pathname);
  params.set('inner-width', window.innerWidth);
  params.set('inner-height', window.innerHeight);
  fetch('/page-view?' + params.toString());
}
(
landing
is for A/B tests, which I've actually stopped doing since I don't get enough traffic for them to be significant)
on the backend: handler:
Copy code
(defn page-view [{:keys [headers
                         session
                         query-params
                         com.yakread/tracking-cookie]
                  :as sys}]
  (biff/submit-tx sys
    [(merge (biff/assoc-some
             {:db/doc-type :event
              :event/type :page-view
              :event/timestamp :db/now
              :event/cookie-uid tracking-cookie}
             :event/ip (headers "x-real-ip")
             :event/user (session :uid)
             :event/user-agent (headers "user-agent"))
            (update-keys query-params #(keyword "event.params" %)))])
  {:status 204})
middleware:
Copy code
(defn wrap-tracking-cookie [handler]
  (fn [{:keys [cookies] :as req}]
    (let [new-cookie (random-uuid)
          old-cookie (biff/catchall
                      (java.util.UUID/fromString
                       (get-in cookies ["uid" :value])))]
      (cond-> (handler (assoc req :com.yakread/tracking-cookie (or old-cookie new-cookie)))
        (not old-cookie) (assoc-in [:cookies "uid"]
                                   {:path "/"
                                    :max-age (* 60 60 24 365 10)
                                    :same-site :lax
                                    :value (str new-cookie)})))))
schema:
Copy code
:event/id :uuid
   :event (doc {:required [[:xt/id                      :event/id]
                           [:event/type                 [:enum
                                                         :page-view
                                                         :signup
                                                         :subscribe
                                                         :navigate]]
                           [:event/timestamp            inst?]
                           [:event/cookie-uid           :uuid]]
                :optional [[:event/ip                   :string]
                           [:event/user                 :user/id]
                           [:event/variant              :string]
                           [:event/user-agent           :string]
                           [:event.page-view/path       :string]
                           [:event.page-view/referrer   :string]
                           [:event.subscribe/opml       :string]
                           [:event.subscribe/feeds      [:vector :string]]
                           [:event.navigate/page        :keyword]
                           [:event.navigate/inner-width number?]
                           [:event.navigate/display-mode :keyword]]
                :wildcards {'event.params               any?}})
(doc is an alias for 
com.biffweb/doc-schema
) I use cookies to be able to count unique visits (i.e. in the dashboard code, the cookie id is what gets used as the user ID--unless the user signed up, in which case I match the cookie ID for any page views to any cookie IDs for signup events, and use the user document's ID). that also was helpful for fraud detection in The Sample when I was doing affiliate advertising (pay people $x per subscriber they send, via https://swapstack.co), though I've since solved that problem by just not doing affiliate advertising anymore, so probably could get by without cookies
e.g. plausible uses some sort of hashed combo instead of cookies, like date+ip+user agent, I think. It won't catch repeat visitors across separate days, but eh
l
Selfishly speaking, it would be nice to get by without cookies just so I never have write a cookie consent popup 😩
j
yeah I am currently operating under the assumption that I don't need one because they're first party cookies... please do not tell me if that's not how it works I probably will stop using the cookies at some point anyway
alright, no documentation, but here's the dashboard code, with example usage at least: https://gist.github.com/jacobobryant/99f2db7f1cba7e60d7258c87cf1ebfaf I'd love to clean that up and include it as part of Biff actually. it's just a pure function that takes a list of events and returns a Rum data structure
I guess it wouldn't hurt to include the actual 
query-events
implementation--just added it
l
Ooh, also my first time learning about Tippy. Neat!
j
it is a lot faster than plain pr-str/clojure.edn/read-string
l
Definitely added to my list of "ways to get React-like behavior that I want without using React" 😎
j
oh lol when you said tippy I thought you mispelled nippy
yes tippy is also cool! I discovered that recently
(I was very confused about your React comment for about four seconds)
l
Ohh, "nippy". Welp, also learning about that now, too 😂
j
Oh also one little detail I was going to mention: reason I submit the page view events via a JS snippet--besides the fact that you can add extra stuff to the request--is that it helps a bit to filter out crawlers and other bots. though I have a sneaking suspicion/paranoia that it's getting more common for those to execute JS as well
(I mean, why else would my landing page conversion rate be only 10%)
l
Smart. Yeah, given how many pages are JS soup, I'm sure they're executing at least some subset
I know I wouldn't be able to resist the "Stick it" button 😉
PreviousNext
Powered by