Category: Troubleshooting

Deployment of New VM from Template Fails with VMDK Locked Error

~ Anton Khitrenovich ~ 32 Comments

Some time ago I managed to isolate an ugly vSphere 5 bug that caused me some unpleasant moments. Today I finally reproduced this behavior in a clean lab environment, so I feel confident enough to describe it in public. Don’t worry, the way to recover is described also!

Take a vSphere 5 environment with a template (say, “MyTemplate”) that you want to deploy a virtual machine from. Step through a regular deployment wizard and select “Edit Virtual Hardware” on the last page. In the VM properties, choose the hard disk and change its size. The new VM (say, “MyVM”) will be created successfully, but the next time you’ll go through the deployment process it will fail with a strange message: “Unable to access file ‘MyTemplate/MyTemplate.vmdk’ since it is locked”…

A quick search in VMware KB (or just in Google) will tell you that the vmdk (the virtual disk of the VM) is usually locked when some other VM uses the same disk. Hey, this is a template! No other VMs are supposed to use the template disk, right? Well, the truth is that this disk is now attached to MyVM – yes, that one with different disk size… But let the lock alone. Your lovely template is now gone, since the disk that belonged to the template is now owned by the running virtual machine – with new name, settings and maybe even software!

Well, how do you recover from this situation? The good news are that the original disk from MyTemplate is not gone. In fact, vSphere really cloned a new vmdk file for MyVM and placed it where it should usually reside – in ‘MyVM/MyVM.vmdk’ – but for some reason set the MyVM to use a template file instead. At this moment you can take a deep breath, shut down MyVM, point it to the right virtual disk and physically swap the files in the datastore… Continue reading “Deployment of New VM from Template Fails with VMDK Locked Error” →

How to install Admin Pack for Windows 2003 on Virtual Machine

~ Anton Khitrenovich ~ Leave a comment

The internet is full of complains that Admin Pack for Windows 2003 cannot be installed due to protected Windows files. I guess that this can happen for various reasons, but I have found a solution for at least one of the cases – when the server where the installation of Windows 2003 Admin Pack fails is virtualized.

So, we have two servers – Windows 2003 Server with Active Directory installed as Domain Controller (server #1) and VM with Windows 2003 Server that is a member of this domain (server #2). The need – administer DNS and DHCP settings of AD remotely, but corresponding MMC snap-ins are not available by default on non-DC machines. The correct way is to install ADMINPACK.MSI, available from Windows 2003 installation CD-ROM or from Microsoft Download Center. (Of course, you can always open an RDP session to a DC – but this does not address the real need.)

However, the installation suddenly shows a message “The Windows Installer service cannot update one or more protected windows files” and rolls back everything that was already installed. The Event Viewer reveals error numbers (“Error 1933. … SPF Error: 1223.”) and the list of protected files – but this does not really help. So, what can we do about that?

In fact, the solution is really simple. Running the same installation via the console RDP session (or in the vSphere console) reveals the real cause for the failure – the Admin Pack install needs Windows 2003 installation CD-ROM, but for some reason does not tell that on non-console sessions. After the requested CD is provided (for VMs, this can be done by mounting the CD image in vSphere or via DaemonTools), the installation was able to complete successfully. Continue reading “How to install Admin Pack for Windows 2003 on Virtual Machine” →

How to fix disabled Preferences in Citrix Receiver for Windows

~ Anton Khitrenovich ~ Leave a comment

Imagine that you’ve just installed Citrix Receiver for Windows 3.0. The nice broadcast icon already appears in system tray, but Preferences item in the context menu is disabled for some reason. The system logs has no relevant errors, and Citrix is not mentioned at all under Programs in the Start Menu. This is the time to read the documentation and discover that Microsoft .NET 2.0 or higher is required for Citrix Receiver to operate. Unfortunately, Receiver installation process does not verify that .NET is actually installed on your machine…

How to turn on debug log in WebSEAL

~ Anton Khitrenovich ~ 4 Comments

You can turn on debug logging with “trace” utility from “pdadmin” shell via “server task” call. Here is the general format of the call:

server task <instance>-<host> trace set <component> <level> file path=<path>

The parts in the angle brackets should be replaced with appropriate values:

  • <instance> – the name of WebSEAL instance
  • <host> – hostname of the server that runs WebSEAL
  • <component> – name of the component to turn logging on for (see below)
  • <level> – log level to use (1 to 9, where 1 is the most detailed; use 0 to turn logging off)
  • <path> – filename of the log file to fill with the information

The list of the components can be obtained with “trace list” command. Among the useful WebSEAL-related components:

  • pdweb – trace all WebSEAL-related information
  • pdweb.debug – list of HTTP header going trough WebSEAL
  • pdweb.snoop – same, but including message body
  • pdweb.snoop.client – same, but only for data sent between WebSEAL and the web client
  • pdweb.snoop.jct – same, for data sent between WebSEAL and the actual web servers (aka “junctions”)
In the example below you can see how to turn on debugging of the web traffic that goes through WebSEAL. The log will go to “C:\pdweb.debug” file. [ Update (April 2013): How to open the ‘pdadmin’ shell. ]
C:\>pdadmin
pdadmin> login
Enter User ID: sec_master
Enter Password: ********
pdadmin sec_master> server task default-webseald-tameb trace set pdweb.debug 2 file path=C:\pdweb.debug
pdadmin sec_master>

You can read more about “server task trace” in the TAMeb documentation.

Fixing revert snapshot failures in vSphere

~ Anton Khitrenovich ~ Leave a comment

Did it ever happen to you that you want to revert some VM to another snapshot and get “The object has already been deleted or has not been completely created” error message? It is really painful, especially if you lab heavily relies on excellent snapshot functionality provided by vSphere… It appears that sometimes it can be fixed by deleting the VM from the inventory and adding it back. The process is really simple, but requires administrative credentials.

So, here it goes:

  1. Go to the Summary tab for your VM, find the relevant datastore and choose “Browse Datastore…” option.
  2. Find the folder that belongs to your VM and make sure you see the “.vmx” file (you’ll need it in step 5).
  3. Go back to your VM in the vSphere tree and choose “Remove from Inventory” in the context menu. Be careful – “Delete from Disk” is the next option, and it is not reversible!
  4. Wait while vSphere performs your request.
  5. Go to datastore window, open context menu of the “.vmx” file and choose “Add to Inventory”
  6. Go through the wizard, selecting relevant options for VM import.

That’s all – wait until the VM will be fully imported and try to revert to your favorite snapshot now.