VDDK 5.0 U2 Release Notes
Release date: 20 DEC 2012 | Builds: ESXi 914586, VDDK 835872.
Last document update: 10 MAY 2013.
Check frequently for additions and updates to these release notes.
About the VDDK 5.0 U2
Virtual Disk Development Kit (VDDK) 5.0 U2 is a bug-fix update to VDDK 5.0,
and a follow-on to the 5.0 U1 release.
For information about the VDDK 5.0 release, see the
VDDK 5.0 Release Notes.
For information about the VDDK 5.0 U1 release, see the
VDDK 5.0 U1 Release Notes.
Compatibility Notices for Partners
The installable VDDK packages and redistributable libraries
are slightly modified in this release to fix issues with
suppressed write errors, large files, and package naming.
A snapshot consolidation fix in VDDK 5.0 U1 is also included.
Backup partners who encounter any of these issues should encourage customers
to run ESXi 5.0 U1 or U2, and to replace VDDK with the new build.
Recently Resolved Issues
VDDK 5.0 U2 resolves the following issues:
- No VDDK error for SAN reads and writes to offline LUN.
When attempting SAN transport reads or writes to virtual disk on an offline LUN,
VDDK 5.0 I/O operations appeared to succeed despite the LUN being offline.
The VixDiskLib_Open() function returned success but
VixDiskLib_Read() and VixDiskLib_Write()
failed to return an error even when they could not read or write the offline LUN.
For backup and restore software running through a proxy,
this issue caused backups and restores to return success,
when data was never actually read from or written to virtual disk.
This is fixed in the VDDK 5.1 and VDDK 5.0 U2 releases.
- Can now access large virtual disks using UNC pathnames.
In previous VDDK releases, when you attempted to access a virtual disk
larger than 2GB with VixDiskLib_Open() on Windows
giving its Uniform Naming Convention (UNC) pathname, the call would fail,
and the VDDK logs would contain errors like the following:
DISKLIB-LINK: "UNC-path.vmdk": failed to open (The file is too large)
This issue is resolved in this release.
- VDDK package and product version is 5.0.2 now.
In this release, the installable package names are correctly named
VDDK 5.0 U1 resolved the following issue:
- Snapshot consolidation failure when cleaning up after HotAdd backup.
When VMware Data Recovery (or VADP backup software) tried to clean up after HotAdd backup,
snapshot delete failed as the appliance was reconfigured, and an exception was reported.
The problem was caused by premature exit of the vpxa process,
so vpxa did not inform backup code that HotAdd disks should be removed.
Stale HotAdd disks remained, and were not recognized the next time backup software ran.
This caused failure of snapshot delete, resulting in a buildup of redo logs.
The solution is to wait longer for vpxa to report,
and apply heuristics to clean up stale HotAdd disks if they appear.
The VDDK 5.0 U1 fix also works with earlier ESX/ESXi versions
and for stale redo logs left by earlier VDDK versions.
Known Issues and Workarounds
The following issues pertain to this release,
and were reported since the VDDK 5.0 Release Notes were first published.
- Backing up the HotAdd proxy causes left-over virtual disk.
HotAdding virtual disk to back up the proxy itself results in unremovable disk.
While the proxy is running, ESXi hosts do not allow removal of HotAdd disk,
even using the vSphere Client.
The workaround is to avoid running HotAdd back up on the proxy virtual machine.
To recover from left-over HotAdd disk, first power off the proxy,
then use the vSphere Client to remove the disk from the proxy,
but do not erase this disk,
- Boot disk should be scsi0:0 for use with mount API.
When a virtual machine's boot disk is not on scsi0:0,
the VixMntApi_GetVolumeInfo function does not correctly return
the inGuestMountPoints (such as drive letter C:) or the numGuestMountPoints.
On a scsi0:0 disk the correct information is returned, but not otherwise.
For example with a scsi0:2 boot disk, the two properties are not populated.
This issue was recently reported against VDDK 5.0 and later.
- With application level quiescing, Changed Block Tracking overstates changes.
When calling QueryChangedDiskAreas to get changes between application level
quiesced snapshots, the areas returned can represent a much larger amount of changes
than the actual amount of changes. This issue occurs when taking quiesced snapshots
to back up powered-on VMs running Windows Server 2003, 2008, 2008 R2, and 2012.
The workaround is to set disk.enableUUID=false for any of these target VMs.
To do this in the vSphere Client, power off the VM, click Edit Settings >
Options > Advanced: General > Configuration Parameters, and change value
of the disk.enableUUID parameter. This workaround results in
file level quiesced backups instead of application level quiesced backups.
- Connect call crashes if non-existent snapshot is specified.
If a program calls VixDiskLib_ConnectEx() passing as third parameter
the managed object reference (moRef) of a deleted or non-existent snapshot,
the program crashes when dereferencing the snapshot moRef.
A fix has been identified, and will be available in a future release.
The workaround is to create a snapshot for each operation,
and never operate on virtual machines with different backup programs at the same time.
- SLES 11 GUI interferes with HotAdd proxy backup.
When using SUSE Linux Enterprise Server (SLES) 11 SP2 as a backup proxy,
the Gnome automount utility gvfs-mount tries to mount HotAdded virtual disks,
causing I/O errors and forcible unmount after backup tries to access the disk device.
The workarounds are to avoid running the GUI, or disable the gvfs-fuse daemon.
You do not want Linux to mount the HotAdd virtual disks.
- vixMntapi on Windows can access only the first 2TB of a volume.
The vixMntapi functions can be used to mount a Windows volume larger than 2TB.
However, attempts to access files on the resulting mount may result in this error
message: “The file system structure on the disk is corrupt and unusable.
Please run the chkdsk utility on the volume.”
Customers should not run chkdsk in this case.
Even if the message does not appear, there is a hazard of corrupting
the mounted volume if you attempt to change it, and a risk of getting
incorrect data if the accessed file exists beyond the 2TB limit.
There is no good workaround.
Splitting the volume is not always possible or convenient.
- Slower than expected read speed when using NBDSSL transport.
Apparently due to use of the OpenSSL library version 0.9.8q (also designated 0.9.8.17),
throughput of NBDSSL transport is only about a third of the speed experienced
in VDDK 1.2.1, or about one sixth the speed of unencrypted NBD transport.
In the next VDDK release, this will be fixed
by installing bin\ssleay32.dll version 0.9.8.20 on Windows,
and libssl.so.0.9.8 version 0.9.8t library files
under /usr/lib/vmware-vix-disklib on Linux.
- HotAdd restore by Windows 2008 proxy when opening multiple disks.
When a Windows Server 2008 virtual machine is deployed as a proxy,
HotAdd failures can occur when writing to the second and subsequent disks
of a virtual machine that is being restored. Due to SAN Policy changes,
W2k8 R2 and W2k8 R2 SP1 are more prone to this than earlier editions.
The cause: if COM function IVdsServiceLoader::LoadService is called soon after
its interface was released by a previous iteration, the call might fail.
There is no workaround. IVdsServiceLoader::LoadService can fail
the second time when called rapidly in succession, and VMware code
enumerates and mounts all disks the first time any disk is opened.
A fix has been identified for availability in a future release.
- To delay Storage vMotion, PrepareForAccess must run on vCenter not ESXi.
The Virtual Disk API Programming Guide on page 37 gives the impression that
programs can run VixDiskLib_PrepareForAccess() and VixDiskLib_EndAccess()
on ESXi hosts. Doing so actually throws an error saying
“VDDK: HostAgent is not a VirtualCenter, cannot disable sVmotion.”
These two functions must be run when connected to vCenter Server,
not when connected to an ESXi host.
- VDDK fails to returns error for SAN writes to offline or read-only LUN.
When attempting SAN transport writes to virtual disk on an offline or read-only LUN,
VDDK 5.0 writes appear to succeed despite the LUN being offline or read-only.
Specifically, VixDiskLib_Open() returns success
because of accessible LUN, but VixDiskLib_Write() does not return an error
even though it cannot write to an offline or read-only LUN.
For backup and restore software running through a proxy,
this issue causes restores to return success,
although nothing is actually written to virtual disk.
When a LUN is offline, write and write operations should return an error.
When a LUN is read-only, write operations should return an error.
See KB 2010428.
- EndAccess sometimes does not reenable migration after PrepareForAccess.
With vCenter Server managing vMotion on ESX/ESXi hosts,
the VDDK 5.0 library routine VixDiskLib_PrepareForAccess()
followed by VixDiskLib_EndAccess() occasionally does not
change virtual-machine state to reenable migration after backup.
This seems more likely to occur when VixDiskLib_PrepareForAccess() is called
twice in a row (the second call gets a “[migration] already disabled” response).
Additionally the message “Insufficient permission in the host operating system”
(error 3014, VIX_E_HOST_USER_PERMISSIONS) could appear after these calls.
The two problems might or might not be related. Neither is easily reproducible.
Processes running as Administrator or root should be able to change migration state.
- Cannot load, unload, and reload VixDiskLib.dll in the same process.
A VDDK 5.0 program cannot call VixDiskLib_Init() or VixDiskLib_InitEx(),
followed by VixDiskLib_Init() or VixDiskLib_InitEx() again.
Because the vixDiskLibPlugin must unload vmacore.dll on exit, the program
crashes when Init or InitEx requests VixDiskLib.dll reload.
Although the crash did not occur with VDDK 1.2.1 and before, programs should be revised
to eliminate multiple Init or InitEx calls in a process.
- Problem using UTF-16 characters in pathnames.
VDDK supports only UTF-8 locales.
When VixDiskLib_Open() is given a pathname containing UTF-16 characters,
the virtual disk library fails to find the file.
On Windows 2008 for example, the pathname Temp\vmware-système\*vm*
contains è as a UTF-16 character, whereas VixDiskLib expects UTF-8.
One workaround is to override the Temp pathname in the configuration file
by setting the tmpDirectory key, using a non-UTF-16 pathname.
For details, see documentation for VixDiskLib_InitEx().
Alternatively, software can be modified to use library routines
such as iconv() for transcoding UTF-16 to UTF-8 and vice versa.
- HotAdd code should ignore independent disks, which are unsuitable for backup.
The HotAdd transport should not be used to back up virtual machines with a VMDK
configured as “independent“ (not capable of snapshots).
Any attempt to use HotAdd in this circumstance causes an error that results
in none of the VMDKs being HotAdded, even ones capable of snapshots.