Trying to understand something - If I ingest all users from Azure AD using this recipe - https://datahubproject.io/docs/metadata-ingestion/source_docs/azure-ad without enabling SSO, will those users be able to login into Datahub as well?

No, if you ingest them with the source they are only there as a resource. To be able for them to log in you would need to setup SSO or create users for them manually

+1!