The Inaccessible Exchange Admin Centre and Missing Arbitration Mailboxes
Recently I came across a situation where, after migrating their mailboxes to Office 365, a client began receiving an error trying to access the on-premises Exchange Admin Centre (EAC). We were also trying to finish moving public folders to Office 365 – one of the final steps of the project.
When trying to set up the migration endpoint for the public folders migration in Exchange Online I kept receiving an error saying the “call to the https://server.contoso.com/ews/mrsproxy.svc failed” (the local server name has been removed to protect the client’s information).
Usually this would indicate an issue with Exchange Online communicating back to the Exchange server, however, I could validate access by manually navigating to the Exchange server’s public address from a browser. There were one of several causes for the failure; either the ‘mrsproxy’ service (my colleague’s article on cross-forest Exchange migrations has a great section describing the MRS proxy service) was disabled (a requirement to migrate public folders), the site was offline or the firewall was blocking the required 443 TCP port.
I was able to rule out those common problems through testing and validation.
I found one article where pending Windows Updates were causing a similar problem for others (really?). I reviewed the Exchange 2016 server and noted they did have pending updates but installing them did not resolve the issue.
NOTE: You should always keep your servers patched and schedule reboots of the servers to complete the installation.
It became evident the Exchange Admin Centre on the local Exchange server was not accessible to any administrator. I could enter the site URL and get prompted for credentials, however, I would get an ‘error 500’ once my credentials were provided.
The issue with the EAC was occurring across all three servers too. Two Exchange 2016 servers (including the one hosting the public folders) and one Exchange 2019 server.
This was a clue! No Exchange admins could access the EAC, however, they could successfully use the Exchange Management Shell (EMS). The issue was also occurring on all of their Exchange servers. Additionally, I kept getting the mrsproxy.svc failure when trying to create the public folder migration batch. Something org-wide rather than local to the server was the issue.
Let me be honest with you. I don’t exactly remember what caused me to search for the arbitration mailboxes, but I did find they were missing. I know they are used for a host of Exchange functions (including many features of Exchange management).
When running the EMS command; “Get-Mailbox-arbitration”, Exchange came back with no results. When searching Active Directory Users and Computers (ADUC) it also came back with no results. By default, the arbitration mailbox user accounts are stored in the default Users container in ADUC.
I thought this was odd. We couldn’t find why those mailboxes were missing but I knew they needed to be recreated (Yes! You can simply recreate them if they go missing). In my experience, I have seen some Active Directory administrators delete the accounts from ADUC due to a lack of understanding their purpose. This is your friendly reminder to protect your accounts from accidental deletion in Active Directory!
Start by enabling the AD Recycle Bin and running the following command to protect OUs from accidental deletion;
Get-ADOrganizationalUnit -filter * -Properties ProtectedFromAccidentalDeletion | where {$_.ProtectedFromAccidentalDeletion -eq $false} | Set-ADOrganizationalUnit -ProtectedFromAccidentalDeletion $true To recreate the arbitration mailboxes, you can mount the last used Exchange install setup file, open an elevated command prompt, navigate to the mounted drive of the Exchange install media and run the following command;
“E:\Setup.exe /IAcceptExchangeServerLicenseTerms_DiagnosticDataON /PrepareAD” This will recreate the arbitration user accounts but not their mailboxes. You will have to enable their mailboxes via EMS.
Once I completed this I was able to access the Exchange Admin Centre and I was able to get my public folders migration started (even better)!
So there you have it. Those arbitration mailboxes serve many purposes including your ability to access the Exchange Admin Centre as well as perform public folder migrations.
Insentra would love to hear from you. Perhaps we can be of service to evaluate both your Exchange on-premises and Exchange Online environments, helping to ensure everything is healthy across the board.
