my hunch says something is off with your Redis instance.. that it may have been restarted and lost its data in memory

listing rooms would simply fetch a hash key in Redis: https://github.com/livekit/livekit/blob/master/pkg/service/redisstore.go#L176 The only reason why it would return no rooms is if Redis isn't giving us those keys back

Found something which seems interesting, not an exact log entry, but hey: 1)

```064126Z livekit-cli list-rooms […]

RM_tBcoTBazxmbn sg6NuVW6J1hsq1bUd8qQ:xMD7C```

☝️ Found a room 2)

```2022-10-19T064130.842Z DEBUG livekit rtc/dynacastmanager.go:281 subscribedMaxQualityChange {

"room": "sg6NuVW6J1hsq1bUd8qQ:xMD7C",

"roomID": "RM_tBcoTBazxmbn",

"participant": "lvsuser:{\"uid\":\"uEJI1rzct5gyqk1rQBVDDVjm9Fk1\",\"showId\":\"sg6NuVW6J1hsq1bUd8qQ\",\"ci\":0}",

"pID": "PA_cYWvpsQwjkZd",

"remote": false,

"trackID": "TR_VCxMnJfSHPv5gJ",

"subscribedCodecs": [

{

"codec": "video/H264",

"qualities": [

…

]

}

],

"maxSubscribedQualities": [

…

]

}```

☝️ Only log entry available between the first livekit-cli call and the next one. Note the

roomID

3)

```064142Z livekit-cli list-rooms […]

RM_tBcoTBazxmbn sg6NuVW6J1hsq1bUd8qQ:xMD7C```

☝️ still finds a room … no new logs in the livekit-server 4)

```064158Z livekit-cli list-rooms […]

there are no active rooms ```

☝️ room reference gone (still no new logs in livekit-server) 5)

```2022-10-19T064344.586Z DEBUG livekit service/roomallocator.go:111 selected node for room {

"room": "sg6NuVW6J1hsq1bUd8qQ:xMD7C",

"roomID": "RM_8notgC7Ppmtt",

"nodeID": "ND_9GUI2nes"

}```

☝️ First log entry after a new participant is joining - with a new roomID. Perhaps this points towards your theory that the redis storage have been reset or lost for whatever reason, and because we found no previous reference we create a new one? 6)

```2022-10-19T064344.587Z INFO livekit service/roommanager.go:257 starting RTC session {

"room": "sg6NuVW6J1hsq1bUd8qQ:xMD7C",

"nodeID": "ND_9GUI2nes",

"participant": "lvsuser:{\"uid\":\"ruTn0sBSjGQVhfcQ1G7K8v8w54l1\",\"showId\":\"sg6NuVW6J1hsq1bUd8qQ\",\"ci\":1}",

"sdk": "SWIFT",

"sdkVersion": "1.0.5",

"protocol": 8

}

2022-10-19T064344.588Z INFO livekit rtc/room.go:297 new participant joined {

"room": "sg6NuVW6J1hsq1bUd8qQ:xMD7C",

"roomID": "RM_tBcoTBazxmbn",

"pID": "PA_W4wJraGihJ5n",

"participant": "lvsuser:{\"uid\":\"ruTn0sBSjGQVhfcQ1G7K8v8w54l1\",\"showId\":\"sg6NuVW6J1hsq1bUd8qQ\",\"ci\":1}",

"protocol": 8,

"options": {

"AutoSubscribe": true

}

}```

☝️ However… the “new participant joined” log message seems to still use the old roomID. At this point did we manage to get our redis state back perhaps?

Timestamps, logs(livekit-server + redis) from around the times we observe this kind of behaviour: https://livekit-users.slack.com/files/T01KVSV78P7/F0474T0MBDG

some further digging from my end it looks like redis is flushed after this point:

Copy code

1:M 19 Oct 2022 06:41:57.499 * DB saved on disk

We’ll see if disabling the save on disk functionality in redis makes this any better

The funky part is indeed that the number of keys for whatever reason is changed during the saving process

✅ Update here: I believe we’ve sorted out this issue - it was indeed some other process which took liberty of manipulating the redis db, hence messing with what the livekit-server returned. Many thanks for the help and mental support!