Copy fails after upgrading to vCloud Connector 2.6
After you upgrade a vCloud Connector node from version 2.5 to version 2.6, copy fails. You get the following error:
"The task has failed. Reason: Optimized Copy. Error creating staging folder /opt/vmware/hcagent/staging/CloudLibrary/UUID/VM-NAME.ova. Please check the space in the node staging
This problem occurs because permissions are changed during upgrade. A new vCloud Connector node has read, write, and execute permissions for user, group, and other (777) on the
staging directory. When you upgrade the node to version 2.6, these permissions are incorrectly set to 755, and the admin user no longer has write permission to the staging directory.
After you upgrade a node, update the permissions on the staging directory.
- Log into the node console as root.
- Use the following command to change the permissions:
chmod 777 /opt/vmware/hcagent/staging
Installation and Configuration
Server tab in the Server Admin Web console and Node tab in the Node Admin Web console display a blank page
If you are using Firefox 15 or a higher version, when you select the Server tab in the vCloud Connector Server Admin Web console, a blank page appears.
A blank page also appears when you select the Node tab in the vCloud Connector Node Admin Web console.
Use a different browser to log in to the vCloud Connector Server and Node Admin Web consoles.
Node or server screen loads for a long time or node cannot be registered with server
After you install and power on a vCloud Connector node or server, or restart it, when you log on to its Admin Web console and click on the Node tab (for the node)
or Server tab (for the server), the screen continues to load for a long time. You might also be unable to register the node with the server.
The following error appears in the log file:
Caused by: liquibase.exception.LockException: Could not acquire change log lock.
- Log in to the node or server console as admin. The default password is vmware.
- Log in to the node or server database.
For the node, type the following command: psql hcs postgres
For the server, type the following command: psql hcs1 postgres
- Type the following command:
select * from databasechangeloglock;
The status in the locked column displays t.
- Update the locked status to 'false':
update databasechangeloglock set locked='false';
- Verify the status:
select * from databasechangeloglock;
The status in the locked column now displays f.
- Exit the database:
- Restart the tomcat service.
For the node, type the following command: /etc/init.d/hcagent-tomcat restart
For the server, type the following command: /etc/init.d/hcserver-tomcat restart
When prompted, enter the password vmware.
- Log out of the node or server console.
To copy a very large workload over a slow network, configure certain settings in the nodes or copy might fail with a delete task id error
If you want to copy a very large workload and the network is very slow, configure certain settings in the source and destination vCloud Connector nodes, otherwise copy might fail with a delete task id error.
When you have a large copy workload and a low bandwidth network, change the following settings in both the source and destination vCloud Connector nodes. If you are copying to vCloud Hybrid
Service, change these settings in your on-premise node.
- Log in to the node as admin.
- Stop the tomcat service by using the following command: /usr/local/tcserver/vfabric-tc-server-standard/agent/bin/tcruntime-ctl.sh stop
- Change to the /usr/local/tcserver/vfabric-tc-server-standard/agent/webapps/agent/WEB-INF/spring/appServlet directory.
- Open the task.xml file for editing.
- Replace the following section (between lines 57-64):
<!-- Quartz job that physically deletes Task resources after a period of
<bean id="removeOldTasksJob" class="com.vmware.hc.agent.jobs.RemoveOldTasksJob"
<property name="jobExecutionIntervalInHours" value="4" />
<property name="taskCleanupIntervalInHours" value="48" />
with the following:
<!-- Quartz job that physically deletes Task resources after a period of time -->
<bean id="removeOldTasksJob" class="com.vmware.hc.agent.jobs.RemoveOldTasksJob"
<property name="jobExecutionIntervalInHours" value="24" />
<property name="taskCleanupIntervalInHours" value="720" />
- Start the tomcat service by using the following command: /usr/local/tcserver/vfabric-tc-server-standard/agent/bin/tcruntime-ctl.sh start
Copy and Content Sync fail for vApp templates that have been added to the catalog with the 'Make Identical Copy' option
If, in vCloud Director, you add a vApp connected to an Organization VDC network to a catalog using the Make Identical Copy option, the following issues occur when you
use the vApp template with vCloud Connector:
- If you copy the vApp template to another vCloud Director cloud, the copy task fails.
- If you subscribe to the catalog that contains the vApp template (and specify a subscription folder on a vCloud Director cloud), the subscription task fails.
The failure occurs when the vApp is being imported to the destination vCloud Director cloud. You get an error message such as the following:
WARN | backend-activity-pool-118 | VAppUploadManagerImpl | Transfer session: 7883d6f8-b81c-4351-9de3-319ad80e1f11. Error during content upload: Following errors occurred while updating
network connections: Invalid network parameter: Invalid IP address.
If you want to manage a vApp template in vCloud Connector, ensure that the Make Identical Copy option was not used to create the vApp template.
Isolated networks not listed in Copy wizard
When you use the Copy wizard to copy a virtual machine or vApp to a vCloud Director cloud, if you select the deploy option, the Select network configuration field lists the networks in the destination
cloud. However, if you have set Stretch Deploy settings on the destination cloud (from the vCloud Connector server Admin Web console), only routed organization networks
appear in the list; isolated networks do not appear in the list.
This problem also occurs when you use the Deploy command to deploy a template on a vCloud Director cloud.
Unregister the Stretch Deploy settings from the destination cloud before you copy or deploy.
1. Log in to the vCloud Connector server Admin Web console at https://vccServerIP:5480.
2. Click the Nodes tab.
3. Click the gears icon next to the vCloud Director cloud and select Unregister Stretch Deploy Settings.
4. Click OK to confirm.
Copy Wizard does not display a warning for attached ISO file while copying from vCloud Director clouds
vCloud Connector does not support copying objects that have an ISO file attached. When you try to copy an object with an attached ISO file from a vSphere cloud, the Copy wizard displays
a warning when you validate your copy selections. However, when you try to copy an object with an attached ISO file from a vCloud Director based cloud, the Copy wizard does not display a
warning. When you proceed with copy, the copy task fails.
Do not copy objects that have attached ISO files.
Temporary template created during copy validation is not deleted if copy fails
When you copy a template to a vCloud Director cloud, as part of copy validation, a temporary template is created to check permissions on the destination catalog. The temporary
template is deleted after the template is copied. However, in some cases when copy fails, the temporary template is not deleted.
Delete such templates manually. Temporary templates have a name such as $$$VCC1234Dummy1234Template123$$$.
Copy validation does not display error for vSphere Profile-Driven Storage issues
If you copy an object to a vCloud Director cloud whose underlying vSphere instance uses Profile-Driven Storage, and there are any errors related to the Profile-Driven Storage service, vCloud Connector
does not detect them during copy validation. Copy validation in the Copy wizard succeeds but the copy task fails. You get the following error when the task fails:
"Remote task failed. Reason : The task has failed. Reason : OVF import failed. 500: Unable to perform this action. Contact your cloud administrator"
Resolve the issues, then copy the object again.
Cannot copy from vCloud Director 5.5 to vCloud Director 1.5
You cannot copy a vApp or vApp template from a vCloud Director 5.5 cloud to a vCloud Director 1.5 cloud. The copy task fails with the following error:
"Validation failed for the OVF file you provided: Fatal: Line/char 26/34: cvc-complex-type.2.4.a: Invalid content was found starting with element 'vcloud:IpScopes'. "
None. Copying from a vCloud Director 5.5 cloud to a vCloud Director 1.5 cloud is not supported.
Cannot use UDT protocol across two firewalls
If your environment has two firewalls between the source and destination vCloud Connector nodes, you cannot use the UDT protocol to copy data. UDT copy occurs over
a dynamically-generated port on the source node and port 8190 on the destination node (or, when you copy between a private cloud and a public cloud, between a
dynamically-generated port on the private cloud node and port 8190 on the public cloud node). Any firewall rules must allow for this type of connection for data transfer. In an environment
with two firewalls, this type of connection is not possible.
UDT copy from a private cloud to a public cloud fails sometimes
In low bandwidth networks, when you copy a large virtual machine or vApp from a private cloud to a public cloud using the UDT protocol, copy might fail. The progress bar displays 30% for a
long time, then the copy task fails.
In the source and destination vCloud Connector nodes, set properties to configure a smaller UDT packet size.
- Log in to the node console as admin.
The default password is vmware.
- Open the /usr/local/tcserver/vfabric-tc-server-standard/agent/webapps/agent/WEB-INF/spring/appServlet/management.xml file in a text editor.
- Search for property name="udtProperties".
Edit the following:
udt_pkt_size: Packet size of UDT packets. The default value is 1048576 KB. For low bandwidth networks, set the value in the following range: 1024-1048576 (1KB to 1MB)
udt_snd_buf_size: Buffer size of UDT at the source. The default value is 10485760 KB. The value must be greater than udt_pkt_size. The recommended value is approximately 10 times the value of udt_pkt_size.
udt_rcv_buf_size: Buffer size of UDT at the destination. The default value is 10485760 KB. The value must be greater than udt_pkt_size. The recommended value is approximately 10 times the vaue of udt_pkt_size.
UDT copy fails with a "Could not connect" or "Bind failed" error
UDT copy might fail with a "Could not connect" or "Bind failed" error when the vCloud Connector nodes associated with the source and destination clouds are registered
to the vCloud Connector server using fully qualified domain names (FQDN).
Use FQDNs with proper entries in the DNS server so that the FQDNs get resolved to the correct address for UDT copy to work.
UDT copy fails with a "Server does not exist" error when FQDNs are used
UDT copy might fail with a "Server does not exist" error if you are using FQDNs for the vCloud Connector nodes and server. When the UDT service is enabled
with FQDN, the server address gets resolved to the loopback address (127.0.0.1) instead of the static IP address with which the node is associated. The
UDT service starts running on 127.0.0.1:8190 and cannot be reached on its FQDN or IP address.
You can use the following command in the vCloud Connector node console to check for this issue:
The command returns the following:
"udp 0 0 127.0.0.1:8190 0.0.0.0:*"
- Log in to the vCloud Connector node console as admin. The default password is vmware.
- In the /etc/nsswitch.conf file, change the hosts entry from
hosts: files dns
hosts: dns files
- Restart the node.
- Enable UDT.
- Use the following command again:
The command should return the following:
"udp 0 0 IPaddress:8190 0.0.0.0:*"
Cannot deploy from a template if cloud is added with vApp User role
If you add a vCloud Director cloud to vCloud Connector using a vApp User role, you cannot deploy vApps from templates. You get the following error:
"Access is denied."
Use a different user role to add the cloud.
Suspend command is unavailable in vCloud Connector 2.5 for virtual machines on vSphere 4.0 clouds
In the vCloud Connector 2.5 UI, the Suspend icon is disabled when you click on a powered-on virtual machine that is on
a vSphere 4.0 cloud.
Cannot see virtual datacenters when a node is registered with the server using system administrator credentials (vCloud Director only)
If you register a node associated with a vCloud Director cloud with the vCloud Connector server using system administrator credentials, you cannot see any of the virtual datacenters
or their contents in the UI. Only catalogs and templates are visible.
Do the following:
- In the Browser panel, expand the cloud.
- Reload each organization by right-clicking and selecting Reload.
- For vCloud Director 5.1.2 clouds, if all content is still not displayed after you have reloaded the organizations, collapse the tree and reload the cloud.
Cannot preserve all metadata in published vApp templates
When you subscribe to a published vCloud Director catalog, metadata in the templates is preserved depending upon the credentials with which the clouds containing the
published catalog and subscription catalog are registered with vCloud Connector. To preserve all metadata entries, register the clouds with system administrator credentials.
If you cannot register the cloud with system administrator credentials, use organization administrator credentials instead. This will not
preserve all metadata entries but will preserve the entries that have Read-Write access. For more information on how metadata is preserved during
Content Sync, see Using vCloud Connector.
All templates in subscription folders or catalogs deleted if "Remove entities if deleted at publisher" option selected
If you select the Remove entities if deleted at publisher option while subscribing to a published folder or catalog in the Content Library, a template should only
be deleted in your subscription folder or catalog when it is deleted in the published folder or catalog.
However, all templates in the subscription folder or catalog are deleted in the following cases if the Remove entities if deleted at publisher option was selected:
The templates are deleted at the next polling interval.
- If you unsubscribe from the published folder or catalog.
- If the published folder or catalog is unpublished from the Content Library.
- If the cloud containing the published folder or catalog is deleted from vCloud Connector.
If you do not want templates in your subscription folder or catalog to be deleted, do not select the Remove entities if deleted at publisher option while subscribing to
a published folder or catalog.
Delay before subscription tasks are updated
After you use the Subscribe command to subscribe to a folder or catalog in the vCloud Connector Content Library, there might be a delay before the task is updated.
Refresh to see the updated task status.
Content Sync does not pick up a vApp template with modified metadata
If you modify the metadata of a vApp template in vCloud Director, vCloud Connector Content Sync will not detect it as a modified vApp template
and synchronize it to subscription catalogs.
Copy the existing template to a new template in the published catalog and Content Sync will pick it up.
Virtual machine name contains timestamp after synchronization
When a template is synchronized to a subscription folder on vCloud Director, the virtual machine in the vApp has a timestamp appended to the name. The virtual machine
name should be the same as the synchronized vApp template name.
Datacenter Extension (Stretch Deploy and Reverse Stretch Deploy)
VM Network Adapter is changed from VMXNet3 to e1000 after virtual machine is stretch deployed to a vCloud Director 5.1.2 or 5.5 cloud
When a Windows virtual machine that has a VMXNet3 Network Adapter is stretch deployed to a vCloud Director 5.1.2 or 5.5 cloud, the network adapter is changed to e1000.
The virtual machine is then unable to retrieve traffic from the VPN tunnel. This issue also occurs when a virtual machine is reverse stretch deployed.
In the destination cloud, power off the virtual machine, modify its NIC type back to VMXNet3, then update the MAC address
to match the MAC address of the source virtual machine. If this issue has occurred during reverse stretch deploy, make these changes in the source cloud.
Stretch Deploy wizard displays Edge External Network Interface Name for stretched network field even if vApp network is connected to a Direct Organization network
When you stretch deploy a virtual machine from a vCloud Director cloud to a vCloud, in the Stretch Deploy wizard, the Edge External Network Interface Name for stretched network
field appears regardless of whether the vApp network is connected to a Routed Organization network or a Direct Organization network. This field is only required for Routed
You only need to specify this field if the vApp network is connected to a Routed Organization network.
Offline Data Transfer
UI does not display a failed task if the source node goes down or the server cannot reach it during export
If the node associated with the cloud from which you are exporting data goes down or if the server cannot reach it during an export task, the export task fails but the UI
does not display a failed task.