Skip navigation

Setting Up a VPN that Uses Certificates

In last month’s article I discussed some guidelines for setting up Microsoft Certificate Services for your company. This month, I focus on using certificates instead of pre-shared keys to establish a VPN. Make sure to use at least Triple Data Encryption Standard (3DES) or the stronger Advanced Encryption Standard (AES) to encrypt data on the VPN. DES is cryptographically weak and can be cracked. Using certificates instead of a pre-shared key during the authentication process significantly increases the VPN's security, especially when one side of a site-to-site VPN doesn’t have a fixed IP address. If you’re establishing a VPN from a fixed physical location with broadband access to the Internet, I strongly suggest that you use a standalone firewall to establish the VPN and protect the remote users. Mobile VPN clients work OK, but malicious hackers can review the traffic coming out of a firewall and hack the remote computer that has the remote VPN client on it. If you must use a mobile VPN client, make sure you have a firewall installed and enabled on the remote client workstation.

According to Cisco, the only Microsoft Certificate Authority (CA) it supports is a standalone root CA. The certificates issued to the Cisco PIX firewall must be issued from this standalone root CA and not a subordinate CA. This requirement presents a problem. Best practices dictate that you should take the root CA offline and issue certificates from a subordinate CA. I asked Cisco Technical Support if it supports a certificate revocation list (CRL) from a non-root CA; the official answer is that they support only a CRL from a standalone root CA. If you keep the root CA up to allow access to the CRL, you risk the entire CA infrastructure by keeping the root CA online. If you take down the root CA, you lose the ability to check the CRL. In the end, I decided to take the standalone root CA offline because this particular client has only a few VPN tunnels that don't change. If you plan to use Certificate Services for other authentication functions, such as Extensible Authentication Protocol-Transport Layer Security (EAP-TLS), consider setting up two standalone root CAs to keep the certificates segregated between those used for EAP/TLS and those that are issued to hardware such as firewalls and routers. In theory, the firewall should be able to receive a certificate from a non-root issuing CA, but I haven’t had a chance to test it.

After you install the standalone root CA, you should install Simple Certificate Enrollment Protocol (SCEP) on the root CA. SCEP is the protocol that Cisco hardware uses to obtain certificates. Alternatively, you can perform a manual certificate request by exporting the request in base-64 encoded PKCS#10 format to the CA. You can download SCEP at SCEP is handy if you need to install multiple certificates because it simplifies the certificate request process. You can also use the SCEP from the Windows Server 2003 Resource Kit, but the SCEP in the previous link is a later version. When you run cepsetup.exe, it will prompt whether you want to use a password for certificate requests. I strongly recommend enabling this option, or you'll be able to request a certificate from any piece of Cisco hardware that has access to the root CA. You must install a root certificate and identity certificate on each PIX firewall that will establish the VPN. Assuming that you chose to require a password when you installed SCEP, you must obtain the enrollment password before you install any certificates on the PIX. Start your Web browser and point it to http:///certsrv/mscep/mscep.dll. You'll be prompted for a user name and password. Log in with a valid account, and the Web page will display a password that's good for 60 minutes. You'll need this password when requesting the identity certificate from the PIX. This password is good for only one use, so you'll need to refresh the screen and obtain a different password for each certificate request.

You must configure the PIX firewall to obtain a root certificate from the standalone CA. The firewall I was using was running PIX OS 6.3 with version 3.0 of the PIX Device Manager (PDM). If you’re using the PDM go to Configuration, VPN Tab, and from the left menu select, IKE, Certificate Configuration and fill out the appropriate information. You can leave the LDAP field blank and enter ":/certsrv/mscep/mscep.dll" for the CA Script location. If you click the Advanced button and select the Peer Certificate Tab, you can enter additional validation parameters to ensure the certificate is valid. The equivalent PIX commands for this screen are:

ca identity <ca_nickname> <ca_root_ip_address> :/certsrv/mscep/mscep.dll 
ca configure <ca_nickname>  ra 1 0 crloptional
ca verifycertdn cn=<common_name_of_hardware_device> 

  1. The first command specifies the nickname, IP address, and CA script location.
  2. The second command specifies the nickname, a Registration Authority (RA), retry interval in minutes, retry count, and that the CRL is optional.
  3. The third command specifies a validation string, in this case the common name (CN) of the hardware device.

For this example, I'll take the root CA offline, so I disabled CRL checking. In general, you shouldn’t disable CRL checking, but I felt that taking the root CA offline was better than enabling CRL checking in this static environment. In the PDM, make sure to save your changes by clicking the Save current changes to flash or issuing the command:

ca save all

In the PDM, click the Authentication menu in the left window and click "Authenticate the CA." The equivalent command is ca authenticate <ca_nickname> Now you should verify that you obtained the root certificate. In the PDM, click the Enrollment menu item in the left window and verify that you obtained the certificate. The equivalent command is:

Show ca cert

At this point, you are ready to obtain the identity certificate for the PIX firewall. If you enabled an enrollment password when you installed SCEP, you'll need it to successfully issue the certificate request. In the PDM, enter the Challenge password and click "Enroll PIX with the CA". The equivalent command is: ca enroll <ca_nickname> <challenge_password> \[serial\] \[ipaddress\] You should get a confirmation that the request was submitted. Assuming you didn't configure Certificates Services to automatically issue certificates, start the Certification Authority on the root CA and view the pending requests. You should see a request from the PIX firewall. Right-click the request and select All Tasks, Issue. The certificate should disappear from the Pending Requests folder and should appear in the Issued Certificates folder. If you don't see the PIX request in the Pending Requests Folder, check the Event Viewer for any error messages. If the Pending Request doesn't appear, try refreshing the password and try the request again by starting Microsoft Internet Explorer (IE) and pointing it to http:///certsrv/mscep/mscep.dll. Enter a valid user name and password and refresh the page to obtain a new certificate request password. After you issue the certificate, it might take a few minutes before the PIX actually receives the certificate. You can refresh the certificate information by clicking the Refresh Certificate Information button or issuing the command

Show ca cert

After you've successfully obtained the certificate on the PIX, make sure to run the following command:

ca save all

This will save the certificate on the PIX. If you forgot to issue this command and you reboot the firewall, you'll lose the certificates, the VPN tunnel will break, and you must repeat the installation process over again. If you have difficulty getting the VPN tunnel up, make sure to check the time settings on each firewall. Of course, you must repeat this process on each firewall that will be used to establish a VPN tunnel.

One limitation I ran into with the PIX 6.3 firewall is the ability to specify the identity of the firewall when establishing a VPN. With PIX 6.3, identity is a global parameter that can’t be specified by VPN tunnel. One of the main reasons why I decided to use certificates on this firewall was to increase security with VPNs that didn’t have a fixed IP address. This particular PIX has other tunnels that do have fixed IP addresses. Usually with a site-to-site VPN tunnel that has a fixed IP address, you use the IP address for the identity of the remote firewall. However, with a floating IP address, you have to use another parameter, such as hostname, because the IP address isn't a fixed value. With version 6.3 of the PIX OS, identity is a global parameter that can't be specified by VPN tunnel. Version 7.x of the PIX OS addresses this limitation by allowing the user to specify the identity by VPN tunnel. To get all tunnels working, I’ll be upgrading this firewall to version 7.x next week.

Installing certificates on a SonicWALL firewall doesn't use SCEP, so you have to manually export the request and import the request into the CA. The process is similar to issuing a manual request on a PIX firewall except SonicWALL uses the DER Encoded Binary X509.cer format for its certificate requests. SonicWALL has some good documentation on setting up a VPN with certificates. Go to, select Support, Knowledge portal, and perform a search on VPN. You'll find a PDF document that does a great job of documenting the certificate installation on a SonicWALL TZ 150 security appliance and SW Pro 2040 firewall using Certificate Services on Windows 2003. Using certificates to establish VPNs isn't the easiest to set up, but it will significantly improve the security of your VPN tunnels.

I recently picked up a Palm Treo 700 and really like it. Because I’m on the road and on the phone a lot, I use a Bluetooth headset. One of the drawbacks to using the headset is it drains the battery on the Treo 700 rather quickly. I’m lucky if the phone lasts the entire day when I’m using the headset. I found an extended-life battery that has 33 percent more battery life than the standard battery. You can order it at With the wireless Exchange ActiveSync included with Exchange 2003, this unit lets me check mail during down times and respond with short messages. The keys on the Treo 700 are small, so I have to type with my thumbnails, but other than that it’s a great unit.

Hide comments


  • Allowed HTML tags: <em> <strong> <blockquote> <br> <p>

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.