I recently upgraded my home network to Forefront Threat Management Gateway (TMG), retiring my old ISA Server 2006 server. I exported my ISA configuration and imported it into TMG, but to be on the safe side I decided to keep a copy of the server as a virtual machine. I used the P2V wizard in System Center Virtual Machine Manager (SCVMM) R2, but after I hit the Scan System button this error appeared:
VMM is unable to complete the request. The connection to the agent <ISA Server FQDN> was lost.
Ensure that the computer <ISA Server FQDN> exists on the network, WMI service and the agent are installed and running and that a firewall is not blocking HTTP and WMI traffic.
ID: 3157
Details: The RPC server is unavailable (0x800706BA)
Preparing to run the P2V wizard I had already created a new access rule allowing all traffic between the SCVMM and ISA Server computers, and disabled the Strict RPC compliance on the rule:
Apparently something was interfering with the RPC communication. Since my P2V rule was at the top of the Firewall Policy whatever was causing the problem had to be in the System Policy. I displayed the System Policy in the Firewall Editor and looked through each rule that specified RPC communication to the ISA Server. Turns out there was only one; the Microsoft Management Console (MMC) rule. To successfully perform a P2V of an ISA Server you need to disable the Strict RPC compliance setting. There was no way to do that in the MMC rule in the System Policy. Since all System Policy rules are processed before the Firewall Rules the MMC rule matched the RPC traffic and blocked parts of it necessary for P2V. I disabled the MMC rule, leaving only my P2V rule. Now the wizard worked correctly.