There are issues with refreshing Dynamic Zones. Dynamic Zones don't seem to get removed when the device running the Zone is shut down, for example. However, in this case, there may be another issue.
Conversely, when I start 'A', Dining Room automatically re-appears on the library server zones list, but on 'B' it appears just as 'A' (i.e. the name of the PC) rather than as 'There: Dining Room'. Again, the only way to bring it back in sync is to close MC on 'B' and re-open it.
If a Client connects to a Server and loads the Server Library, then Zones on the Server will be shown in the format "There: Dining Room" or just "There" if only the default Player Zone exists.
However, if a Client is started but does not load the Server Library, and just loads a local Library, then the Zone displayed in the Client for the Server is just the Server PC name, and it will represent the currently selected Zone on the Server PC. I haven't tested with two MC Clients exactly as you have, but I expect MC is doing the same thing for the other connected Client, "B" in this case. Are you sure "B" has loaded the Server Library when you see this happen?
Note also that it does take a little while for Dynamic Zone discovery to complete, so you need to give Client "B" some time to refresh the Zones. Perhaps refreshing or changing Views would force it to update. But it could be that the full Zone name is only updated when the Client is restarted, as you found.