I think if you want to make it feel natural, it could help to think about what types of sensory details you tend to rely on the most. For example, if you were to describe your happiest memory would you use details about things you saw, tasted, felt, smelled, or heard? However, I do think poems could also be made stronger sometimes by using the senses that you don't think about as often.