Hi! In `DatasetUsageStatistics` , what is the time...
# advice-metadata-modeling
w
Hi! In
DatasetUsageStatistics
, what is the time range for eg the
uniqueUserCount
field: daily, weekly, monthly, ...? https://github.com/datahub-project/datahub/blob/6901f31777314b6fcf704484983a9bebbf[…]rc/main/pegasus/com/linkedin/dataset/DatasetUsageStatistics.pdl
Copy code
/**
 * Stats corresponding to dataset's usage.
 */
@Aspect = {
  "name": "datasetUsageStatistics",
  "type": "timeseries",
}
record DatasetUsageStatistics includes TimeseriesAspectBase {
  /**
   * Unique user count
   */
  @TimeseriesField = {}
  uniqueUserCount: optional int
Since it is a timeseries, the metric can be reported frequently and so we can track how it evolves along time. This makes the reporting frequency totally custom. However, the aspect does not include any reference about the time range that eg the counter considers: forever, weekly, daily, ... For example: I could be reporting this aspect on weekly basis and count monthly unique users. So, I miss in the aspect definition the ability to specify this time-range for the counts. WDYT? How are ingestors currently handling this? Do they count since the dataset was created, since the last time it was reported or with some fixed time range (eg monthly)? Thanks!
Just had a look at
DashboardUsageStatistics
which is very similar. There the documentation mentions the
eventGranularity
field:
If this aspect represents a bucketed window of usage statistics (e.g. over a day), then the eventGranularity field should be set accordingly
https://github.com/datahub-project/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/dashboard/DashboardUsageStatistics.pdl This field is inherited from
TimeseriesAspectBase
https://github.com/datahub-project/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/timeseries/TimeseriesAspectBase.pdl and so it would apply for
DatasetUsageStatistics
too, and any time series aspect. So I understand this
eventGranularity
is the field I was asking for 😅 Self-solved