Hey guys! Created PR https://github.com/datahub-project/datahub/pull/7410 for enhancing the S3 ingestion wherein data exists in the form of partitions. Currently, if there is any file that is present in the latest partition (day,month or year), the first file in the oldest partition gets picked. Due to this limitation, if there are any schema updates in the latest files, they are not visible on DataHub. We are trying to enhance this capability via this PR. Not sure whom to add in the reviewers section.

Thanks for the PR. You don’t need to worry about assigning reviewers, we pick it up on our end and find the right person.

Cool. Thanks @mammoth-bear-12532