How to connect a hosted, Windows 2003 Virtual Private Server to a network behind a Sonicwall TZ170. This setup allows access to the VPS securely and transparently from an office network.
References
IPSEC Modes
Without IPSEC, a fictional ‘example’ packet would contain some data (eg. a plain text message “Hello World”) and control information directing the packet toward a server called TESTSERVER on port 12345.
With IPSEC in transport mode, the packet data would contain encrypted data (“XKDFS1234″) and control information directing the packet toward the same server TESTSERVER and the same port 12345. This is a similar system to SSL.
With IPSEC in tunnel mode, the packet data would contain the same encrypted data (“XKDFS1234″), but the control information would also be encrypted and encapsulated into a new packet. This new packet is directed toward TESTSERVER but on a different port – in this case, the Internet Security Association and Key Management Protocol (isakmp) port 500. This is similar to a VPN, PPTP, or L2TP, but will run on Win2K3 without RRAS.
Firewall
Without IPSEC, certain ports would have to be open to accomodate certain applications.  Mail and Web traffic could use SSL via IMAPS and HTTPS, and non-SSL traffic could be encrypted by IPSEC in transport mode, but authentication would still be a problem.   Instead, with an IPSEC tunnel, all ports on the firewall except port 500 (TCP & UDP) can be closed, and only connections originating from the other end of the tunnel can connect.
Windows Security Policy Structure
The nested nature of an IP Security Policy is confusing. Here is an overview of a policy for creating an IPSEC tunnel.  Since only one policy may be active at one time, add additional inbound/outbound rules within the same policy to create multiple tunnels.
Policy (Tunnel Name)
Rule (Inbound)
Filter (Inbound, configure one filter per subnet at destination)
Action (Tunnel Name, Negotiate security, 3DES/SHA1, uncheck all)
Tunnel (IP address of this machine)
Authentication (Remove Kerbos, Preshared Key)
Rule (Outbound)
Filter (Outbound, configure one filter per subnet at destination)
Action (Same as Inbound rule)
Tunnel (IP address of remote end)
Authentication (Remove Kerbos, Preshared Key)
Test Setup
Since the only way to connect to a VPS is via network, a bad setting can make the VPS unreachable. Two possible practices are:
- Implement and test first on another server with physical access.
- Backup before doing anything. This is easy through most virtualization control panels.
If things go wrong:
- Disable IPSEC services through the virtualization control panel, if possible
- VPS’es often have multiple IP addresses. Try connecting to a different one.
- Call your provider and ask them to stop IPSEC services
- Call your provider and ask them to re-provision your machine and start from scratch! I’ve already had to do this once (backups were not working), which is why I note it here.
I installed the free VMWare server on a desktop computer and setup two virtual instances of Windows Server 2003 (Server1 and Server2) with NAT networking. This isolates the test servers from the rest of the network and lets the host (Host1) computer run tcpdump or Wireshark to monitor traffic. When the tunnel is working, packets will show up as ESP.
When complete, this setup allows Server1 to ping Server2 with IP Security (and vice-versa) and Host1 to ping Server1 and Server2 without IP Security. To prevent pings from Host1, enable the Windows firewall on Server1 and Server2 but add exceptions for isakmp port 500, UDP and TCP.
Sonicwall Setup
General:
- IKE using Preshared Secret
- IPSec Primary Gateway = VPS hostname
- IPSec Secondary = blank
- Shared Secret = common phrase between endpoints
- Destination Network: VPS IP address
Proposals:
- Exchange = Main Mode
- DH Group = Group 2
- Encryption = 3DES
- Authentication = SHA1
- Protocol = ESP
- Encryption = 3DES
- Authentication = SHA1
Advanced:
- everything unchecked
- VPN terminated at LAN/WAN
DNS
Setting up a DNS entry on the LAN for the VPS makes access more convenient and transparent.
