I posted this a few weeks ago on discuss.airbyte.io but didn't get any responses: https://discuss.airbyte.io/t/how-to-manage-internal-id-for-dynamic-streams/4153 I have a connector where I'd like to refer to a stream by name ("foo"), but the underlying API is all based on IDs ("1234"). During discovery I can iterate over all objects and display the stream name. At that point I have the ID in hand ("1234") which would be used for all future API calls. But it looks like there is no way to store the ID, so all I have when someone selects the stream is the name ("foo"). This means that every time I have to iterate over all objects just to find the ID to then make the API calls I want. It's not the end of the world, but am I missing something? Is there any way take information found during discovery and store it alongside the name for later use? There is a "namespace" section but the documentation states that all streams returned from a source should share a namespace so I'm not sure that would be for general use.

You’re not missing anything, the capability you’re looking for isn’t available unfortunately. Does it make sense to embed the ID in the stream name?

Thanks for the response! That's too bad, APIs often use IDs in URL paths, not names, so I would expect this to be somewhat common. Either having a "displayName" distinct from "name" or a JSON object for arbitrary parameters would be nice. Embedding the ID in the stream name could definitely work, though it does create a suboptimal user experience. For what I'm currently doing there are only a small number of objects, so getting all the name -> ID mappings each sync is not terrible. I can resort to the name hack if I ever have to deal with a very large number of objects. Thanks for letting me know I wasn't missing something obvious!