Details
• Virtual machine cannot power on
• Error messages during power up include:
o Unable to open Swap File
o Unable to access a file since it is locked
o Unable to access Virtual machine configuration
• The following messages appear in /var/log/vmkernel:
o WARNING: World: VM xxxx: xxx: Failed to open swap file
o WARNING: World: VM xxxx: xxx: Failed to initialize swap file
• When opening a console within Lab Manager, you may receive the error:
o Error connecting to
• Powering on a virtual machine results in the power on task remaining at 95% indefinitely
• Cannot power on a virtual machine after deploying it from a templateA virtual machine reports conflicting power states between vCenter and the ESX host console
Solution
To prevent duplicate access to active virtual machine files, ESX hosts establish a lock on these files. In certain circumstances, these locks may not be released when the virtual machine is powered off. The files cannot be accessed if they are locked, and the virtual machine cannot power on.
These virtual machine files are commonly affected by lock issues:
•
• vmware.log
•
•
Identifying the locked file
To identify the locked file, try to power on the virtual machine. During power on, an error may display or be written to the virtual machine's log. The error and the log entry identify the virtual machine.
1. Connect VMware Infrastructure (VI) Client to either the ESX host or to the VirtualCenter Server.
2. Locate the affected virtual machine, and try to power it on.
3. Open a remote console window for the virtual machine.
4. If the virtual machine is unable to power on, an error on the remote console screen may display with the name of the affected file. If an error does not display, proceed to the next step to look in the vmware.log file for the virtual machine.
5. Log in as root to the ESX host using an SSH client.
6. To confirm that the virtual machine is registered on the server and to obtain the full path to the virtual machine, run the command:
[root@esxhostname]# vmware-cmd -l
The output returns a list of the virtual machines registered to the ESX host. Each line contains the full path of a virtual machine's .vmx file. For example:
/vmfs/volumes/
Record this information as it will be required in the remainder of this process. This is the
Verify that the affected virtual machine appears in this list. If it is not listed, the virtual machine is not registered on this ESX host. The host on which the virtual machine is registered typically holds the lock. Ensure that you are connected to the proper host before proceeding.
7. To move to the virtual machine's directory, run the command:
[root@esxhostname]# cd /vmfs/volumes/
8. Use a text viewer to read the contents of the vmware.log file. At the end of the file, look for error messages that identify the affected file.
Identifying the ESX host that is locking the file
Because a virtual machine can move between hosts, the host where the virtual machine is registered may not be the host with the file lock. The lock must be released by the ESX host that owns the lock. This host is identified by the MAC address of the primary service console.
To identify the host by the MAC address:
1. To report the MAC address of the lock holder, run the command:
[root@esxhostname]# vmkfstools -D /vmfs/volumes/
2. This command writes the MAC address of any host that is locking the .vmdk file to the vmkernel log file. To locate this information, run the command:
[root@esxhostname]# tail /var/log/vmkernel
Look for lines similar to the following:
Apr 5 09:45:26 Hostname vmkernel: 17:00:38:46.977 cpu1:1033)Lock [type 10c00001 offset 13058048 v 20, hb offset 3499520
Apr 5 09:45:26 Hostname vmkernel: gen 532, mode 1, owner 45feb537-9c52009b-e812-00137266e200 mtime 1174669462]
Apr 5 09:45:26 Hostname vmkernel: 17:00:38:46.977 cpu1:1033)Addr <4, 136, 2>, gen 19, links 1, type reg, flags 0x0, uid 0, gid 0, mode 600
Apr 5 09:45:26 Hostname vmkernel: 17:00:38:46.977 cpu1:1033)len 297795584, nb 142 tbz 0, zla 1, bs 2097152
Apr 5 09:45:26 Hostname vmkernel: 17:00:38:46.977 cpu1:1033)FS3: 132:
(END)
The second line displays the MAC address a fter the word owner. In this example, the MAC address is 00137266e200.
3. To determine if the MAC address corresponds to the host that you are logged into, see Identifying the ESX Service Console MAC address (1001167). If it does not correspond, you must establish an SSH connection to each host that can run this virtual machine. Once identified, use the host in the following procedures.
Note: If this process does not reveal the MAC address, it is possible that it is an NFS lock (for more information, see the section Removing the .lock file (NFS Only)), or that you have to migrate the virtual machine to each host to identify the lock holder.
4. If, due to power failure or other reasons, you are unable to obtain the MAC address in this method, you must then manually migrate the virtual machine to each host in the cluster. At each host, attempt to power on the virtual machine. When you reach the host that holds the lock, it should successfully be able to power on.
Removing the .lck file (NFS Only)
The virtual machine's files may be locked by the NFS storage. You can identify this by the .lck.#### (where #### refers to the World ID that has the file lock) at the end of the filename. This is an NFS filelock. These can be removed safely, as long as the virtual machine is not running on any other ESX host.
Determining if the file is being used by a running virtual machine
If the file is being accessed by a running virtual machine, the lock cannot be removed. It is possible that the lockholder host is running the virtual machine and has become unresponsive.
To determine if the virtual machine processes are running:
1. To determine if the virtual machine is registered on the server, run the command:
[root@esxhostname]# vmware-cmd -l
Note: If the virtual machine is registered on more than one ESX host, see Virtual machines appear to be running or registered on multiple ESX servers (1005051).
2. To assess the virtual machines current state, run the command:
[root@esxhostname]# vmware-cmd
If the output from this command is getstate() = on, the virtual machine has become unresponsive. To address this issue, see Troubleshooting a virtual machine that has stopped responding (1007819).
If the output from this command is getstate() = off, the ESX host may be unaware of the file lock.
3. To stop the virtual machine process, see Powering off a virtual machine hosted on ESX host from the command line (1004340).
Determining if the .vmdk file is in use by other virtual machines
A lock on the .vmdk file can prevent a virtual machine from starting. However, since virtual machine disk files can be configured for use with any virtual machine, the file may be locked by another virtual machine.
To determine if the virtual machine's disk file is configured for use on more than one virtual machine, run the command:
[root@esxhostname]# egrep -i
The output refers to the virtual machine you are currently working on. If any additional virtual machines are configured to use the disk, determine if they are currently running. Powering off the other virtual machine using the disk file releases the lock. You must determine which virtual machine should have ownership of the file, then reconfigure your virtual machines to prevent this error from occurring again.
If the .vmdk file is not used by other virtual machines, proceed to the next section to remove the lock.
Removing the lock
Note: Follow these sections in order. If removing the .vswp file does not unlock the file, try clearing the lock with the touch command. If that does not resolve the issue, try rebooting the ESX host. Do not skip a section.
Removing the .vswp file
The .vswp file is used by running virtual machines as memory swap space. It is typically deleted when the virtual machine is powered off. If this file and its reference in the .vmx file still exist when powering back on, it can prevent the virtual machine from starting. If the virtual machine is not running, this file can safely be deleted.
To remove the .vswp file:
1. To move to the virtual machine's directory, run the command:
[root@esxhostname]# cd /vmfs/volumes/
2. To delete the .vswp file, run the command:
[root@esxhostname]# rm /vmfs/volumes/
Note: Depending on the lock held on this file, it may or may not be successfully removed.
3. To backup the virtual machine's .vmx file, run the command:
[root@esxhostname]# cp
4. Use a text editor to open the .vmx file for the virtual machine. Locate and delete the line:
sched.swap.derivedName
5. Save and exit the file.
6. Try to power on the virtual machine. If the issue persists, proceed to the next section to remove the lock.
Clearing the file lock with the touch command
The file lock can often be resolved from the lockholder without impact to the operation of the host or the virtual machines running on the host.
To clear the file lock with the touch command:
1. Run the command:
[root@esxhostname]# touch
Note: Performing a touch * command will perform the operation on all files in your current directory.
2. Try to power on the virtual machine. If the issue persists, proceed to the next section to clear the lock.
Clearing the file lock by rebooting the ESX host
As a final troubleshooting step, try restarting the ESX host that holds the lock.
To restart the ESX host:
1. Migrate all virtual machines from the host to new hosts.
2. When the virtual machine are moved, place the host in maintenance mode and reboot.
Note: If you have only one ESX host or do not have the ability to migrate virtual machines, you must schedule a downtime for all affected virtual machines prior to rebooting. When the host has rebooted, start the affected virtual machine.
Ingen kommentarer:
Legg inn en kommentar