Does anyone know if when relationships are created Prisma automatically indexes foreign keys?

Yes they do, they are backed by foreign keys (and foreign keys are used to detect relations)

So, we are getting database notices that full table scans are being run on some of our relationship tables. Why would that be happening?

No, but you have access to the schema and the messages, so feel free to create a bug report issue in the prisma repo so we can look at that.