I have a requirement as below. Can you give me hin...
# general
g
I have a requirement as below. Can you give me hints how to achieve that? Segments Timestamp in DataSources ingested by Person1 are given below: Person1_DataSource1 = January 1 Person1_DataSource2 = January 2 Person1_DataSource3 = January 3 Person1_DataSource4 = January 4 Person1_DataSource5 = January 5 Person1_DataSource6 = January 6 Person1_DataSource7 = January 7 Person1_DataSource8 = January 8 Person1_DataSource9 = January 9 Person1_DataSource10 = January 10 Segment Timestamp in DataSources ingested by Person2 are given below: Person2_DataSource1 = January 21 Person2_DataSource2 = January 22 Person2_DataSource3 = January 23 Person2_DataSource4 = January 24 Person2_DataSource5 = January 25 Person2_DataSource6 = January 26 Person2_DataSource7 = January 27 Person2_DataSource8 = January 28 Person2_DataSource9 = January 29 Person2_DataSource10 = January 30 So, we have the above 20 Datasources, 10 ingested by Person1 and 10 ingested by Person2 Now, i need to apply the Drop and Load Rule so that latest 5 Days data is available in Historical for Person1. output => January 6 to 10 (5 Days) should be available latest 8 Days data is available in Historical for Person2. output => January 23 to 30 (8 Days) should be available How can i acheive that?
s
Each datasource has its own retention rules. Could person 1 just ingest data into Person1_datasource for multiple days? You then set the retention rules like:
Same would be true for Person2_datasource. You would change the loadByPeriod property to P5D or P8D days as each person requires.