HP NIC Teaming, Netapp CIFS, vRanger Pro Issue

I was seeing extremely slow CIFS performance on servers with HP NIC teams. This includes our vRanger backup server. I would map a drive and it would take minutes to come up, etc. I finally found a NOW article suggesting to run:

options ip.fastpath.enable off

This command, from my understanding, will not use cached MAC addresses. Because of the way our HP NIC team is setup, different MACs were being presented and this caused an issue. So far so good.

ESX NFS Snapshot Issues

I have been having snapshot issues with ESX on Netapp NFS. The server is hanging for a long time when deleting snapshots. We are also seeing ESX Ranger issues when doing backups.

The Netapp guide mentions to install patch ESX350-200808401-BG. I am on the version 3 of ESX 3.5 so it wasnt necessary. All I had to do was add this line to the esx config file.

prefvmx.consolidateDeleteNFSLocks = "TRUE"

ESX Ranger, ESX, Netapp, NFS

If you have ESX running on Netapp, you sure as hell probably installed the Netapp ESX tools. If you didnt then you should. A couple things that the tools do is change NFS settings within ESX.

I ran into an issue with ESX Ranger failing on backups from Netapp NFS to a Netapp CIFS share. The error was:

The Backup file /vmfs/volumes/NFS_TIER_BLAH/Server1//Server1-flat.vmdk that was transfered appears to be Invalid! Transfered Size: 112546265463, Expected Size: 94667312308.

The solution seems to be changing the NFS timeout values within ESX by:

Changing the value of NFS.HeartbeatFrequency to 12
Changing the value of NFS.HeartbeatMaxFailures to 10

This will work for most people if they havent already installed the Netapp tools and therefore already have these settings. This didnt fix my problem, but it may yours.

My solution was to Use NFS instead of CIFS for the target. It works thus far.

Sean

Backup and Recovery Setup

I recently designed and implemented a backup/recover solution for a small hedge fund. They were using Veritas Backup Exec, a Quantum P50 DLT (or something). Brick level backups were being done on the Exchange server and everything was being backed up over the LAN.

As everybody does, our backup and restore times were not meeting SLAs. I performed an analysis and ended up bringing in EMC Networker, Data Domain, EMC Replication Manager, Kroll On-track and ESX Ranger.

This solution allowed me to backup large data sets on the SAN using Replication Manager, it allowed for fast backups and restores with the ability to store around 50 TB of data on 4 TB of disk using Data Domain and the EMC Networker software, although harder to learn, is enterprise class.

Backing up virtual machines is cake. ESX Ranger snaps them off and puts them on Data Domain.

All Data Domain is replicated to our DR site.

No more slow brick level backups of Exchange. By using Replication Manager to clone to the backup server, we backup the Exchange database files in their interity. Replication Manager also truncates the logs when the backup completes.

What if we want to restore one mailbox? Using Kroll Ontrack we can look into the EDB files, choose the mailbox we want to restore and restore it to a PST.

SLAs are met, restores are quick and now I sit back on move on to the next project.