FatMin

Mapping nic hardware address to PCI address

When logged into an ESX host the following command will list out all your nics with their PCI address. As you can see I have quite a few active ports on this box. Specifically I have two onboard ports and two quad port cards.

esxcfg-nics -l

vmnic0  0d:00.00 e1000       Up   100Mbps  Full   1500   Intel Corporation Intel PRO/1000 PT Low Profile Quad Port PCIe Gigabit Ethernet Controller
vmnic5  0e:00.01 e1000       Up   1000Mbps Full   1500   Intel Corporation Intel PRO/1000 PT Low Profile Quad Port PCIe Gigabit Ethernet Controller
vmnic8  12:00.00 e1000       Up   1000Mbps Full   1500   Intel Corporation Intel PRO/1000 PT Low Profile Quad Port PCIe Gigabit Ethernet Controller
vmnic2  05:00.00 bnx2        Up   1000Mbps Full   1500   Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T
vmnic4  0e:00.00 e1000       Up   1000Mbps Full   1500   Intel Corporation Intel PRO/1000 PT Low Profile Quad Port PCIe Gigabit Ethernet Controller
vmnic1  03:00.00 bnx2        Up   1000Mbps Full   1500   Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T
vmnic3  0d:00.01 e1000       Up   1000Mbps Full   1500   Intel Corporation Intel PRO/1000 PT Low Profile Quad Port PCIe Gigabit Ethernet Controller
vmnic6  11:00.00 e1000       Up   1000Mbps Full   1500   Intel Corporation Intel PRO/1000 PT Low Profile Quad Port PCIe Gigabit Ethernet Controller
vmnic7  11:00.01 e1000       Up   1000Mbps Full   1500   Intel Corporation Intel PRO/1000 PT Low Profile Quad Port PCIe Gigabit Ethernet Controller
vmnic9  12:00.01 e1000       Up   1000Mbps Full   1500   Intel Corporation Intel PRO/1000 PT Low Profile Quad Port PCIe Gigabit Ethernet Controller

While this information is helpful, it does not tell me which ports are on the same card. However the following command gives me the mac addresses of each port, so I can figure out which ports are on the same quad port nic by comparing similar macs, I have highlighed the ports that are on the same card as vmnic9

grep "/net/pnic" /etc/vmware/esx.conf

root@rhost11 log]# grep "/net/pnic" /etc/vmware/esx.conf
/net/pnic/child[0000]/mac = "00:1c:c4:ec:65:58"
/net/pnic/child[0000]/name = "vmnic1"
/net/pnic/child[0000]/virtualMac = "00:50:56:5c:65:58"
/net/pnic/child[0001]/mac = "00:1c:c4:ec:65:56"
/net/pnic/child[0001]/name = "vmnic2"
/net/pnic/child[0001]/virtualMac = "00:50:56:5c:65:56"
/net/pnic/child[0002]/mac = "00:18:fe:2e:1b:21"
/net/pnic/child[0002]/name = "vmnic0"
/net/pnic/child[0002]/virtualMac = "00:50:56:5e:1b:21"
/net/pnic/child[0003]/mac = "00:18:fe:2e:1b:20"
/net/pnic/child[0003]/name = "vmnic3"
/net/pnic/child[0003]/virtualMac = "00:50:56:5e:1b:20"
/net/pnic/child[0004]/mac = "00:18:fe:2e:1b:23"
/net/pnic/child[0004]/name = "vmnic4"
/net/pnic/child[0004]/virtualMac = "00:50:56:5e:1b:23"
/net/pnic/child[0005]/mac = "00:18:fe:2e:1b:22"
/net/pnic/child[0005]/name = "vmnic5"
/net/pnic/child[0005]/virtualMac = "00:50:56:5e:1b:22"
/net/pnic/child[0006]/mac = "00:18:fe:2e:b3:91"
/net/pnic/child[0006]/name = "vmnic6"
/net/pnic/child[0006]/virtualMac = "00:50:56:5e:b3:91"
/net/pnic/child[0007]/mac = "00:18:fe:2e:b3:90"
/net/pnic/child[0007]/name = "vmnic7"
/net/pnic/child[0007]/virtualMac = "00:50:56:5e:b3:90"
/net/pnic/child[0008]/mac = "00:18:fe:2e:b3:93"
/net/pnic/child[0008]/name = "vmnic8"
/net/pnic/child[0008]/virtualMac = "00:50:56:5e:b3:93"
/net/pnic/child[0009]/mac = "00:18:fe:2e:b3:92"
/net/pnic/child[0009]/name = "vmnic9"
/net/pnic/child[0009]/virtualMac = "00:50:56:5e:b3:92"

Kill and VM from the Command Line

If you need to kill a guest VM via the command line, use the vmware-cmd command.

"vmware-cmd /<path to VM directory>/server.vmx stop"
"vmware-cmd -l" will get you your virtual machines names and config file locations

If that does not work you can take more drastic measures and kill the VM process itself.
ps -ef | grep <virtualmachinename>
kill -9 <pid>

Esxcfg-vswitch Commands

 
For us Unix guys, the idea of doing everything in ESX via a GUI is very hard to swallow. Especially when it comes to repetitive tasks like setting up vSwitches and Port Groups. Sure you can spend and arm and a leg on Enterprise Advanced licenses so that you can have distributed switches, but why when you can just script your commands and run that script on each box in your cluster. Below are a few examples of how to use the esxcfg-vswitch command.

Delete a Port Group:

esxcfg-vswitch -D "Service Console" vSwitch1

Add a nic (vmnic2) to a vswitch (vswitch1):

esxcfg-vswitch -L vmnic2 vswitch1 

Remove a pnic (vmnic3) from a vswitch (vswitch0):

esxcfg-vswitch -U vmnic3 vswitch0

Create a portgroup (VM Network3) on a vswitch (vswitch1):

esxcfg-vswitch -A "VM Network 3" vSwitch1

Assign a VLAN ID (3) to a portgroup (VM Network 3) on a vswitch (vswitch1):

esxcfg-vswitch -v 3 -p "VM Network 3" vSwitch1

Change your Service Console (vswif0) IP and Subnet Mask:

esxcfg-vswif -i 172.20.20.5 -n 255.255.255.0 vswif0

Add a Service Console (vswif0):
esxcfg-vswif -a vswif0 -p "Service Console" -i 172.20.20.40 -n 255.255.255.0

Add Vlan to a Port Group:

esxcfg-vswitch "vSwitch name" -p "portgroup name" -v "vlan ID" 

List the existing config so you can get your vswitch and portgroup name:

esxcfg-vswitch -l

Enabling Web Access in ESX 4

For some reason, Vmware has decided to disable the Web Access service in ESSX 4 by default. You will find this out if you attempt to click on "Log into Web Access" on your Vmware ESX 4 Welcome Page on a newly build ESX 4.0 server. You will get the following error "503 Service Unavaliable". Do not panic

To resolve this you should SSH into the ESX box and type the following

service vmware-webAccess start, to start the service

chkconfig –level 345 vmware-webAccess on, to set the service to start at boot.

vSphere 4 Configuration Maximums Document

When you select and configure your virtual and physical equipment, you must stay at or below the maximums

supported by vSphere 4.0. The limits presented in the attached document represent tested, recommended limits,

and they are fully supported by VMware.

 

http://www.vmware.com/pdf/vsphere4/r40/vsp_40_config_max.pdf

VMware vSphere Editions Comparison

The Vmware Marketing team is at it again, as they have introduced even more ESX and ESXi version in VSphere 4. This link is from Vmware.com and helps clarify the difference between versions.  

VMware vSphere Editions Comparison:

http://www.vmware.com/products/vsphere/buy/editions_comparison.html

VMK load_Mod panic failed to load module VMFS2

This past weekend I was doing a few memory upgrades when one of the boxes that I was working on decided not to boot up properly. This was a bummer considering that it was 2am.

Networking did not start properly so I was unable to ssh to the box, however I found the above error messages in the vmware kernel log (/var/log/vmkernel), the hostd log (/var/log/vmware/hostd.log) and on the console.

First I needed to verify the modules that started using the esxcfg-module command with the -l switch. Below is the output of this command, as it should appear, however in this case as we found that the vmklinux and the vmfs3 were loaded up fine but there was no vmfs2 module.

esxcfg-module -l
Device Driver Modules
Module         Enabled Loaded
vmklinux       true    true
bnx2           true    true
cciss          true    true
e1000          true    true
lpfc_740       true    true
lvmdriver      true    true
vmfs3          true    true
etherswitch    true    true
shaper         true    true
tcpip          true    true
cosShadow      true    true
migration      true    true
nfsclient      true    true
deltadisk      true    true

vmfs2          true    true

The we checked whether vmfs2 is not enabled by using the "-g" switch to the same command and found it was not set to load at boot. We then set the boot option to true by using the "-s" switch for the "vmfs2". The performed an esxcfg-boot -b and then rebooted the ESX and it booted up fine.

Determining build number on ESX 3.5.x hosts

In some cases, after patches have been installed on a ESX host, different build numbers are reported for different components of ESX Server. When a patch is applied, the build number of the updated components will change. As a result, the ESX Server installation can have different components at different build numbers, depending upon which patch bundles have been applied.

#esxupdate query

Installed

software bundles:

  ------ Name ------ --- Install Date --- --- Summary ---
         3.5.0-64607    17:53:02 10/15/07 Full bundle of ESX 3.5.0-64607
For a differential list of rpms, use the -l/--listrpms option.

Finding Large Files in Unix/Linux

Back in the day I used to have some really good notes on this topic, but alas they are long gone, so I have decided to put together a small list with some helpful examples. Note that a lot of folks will show you examples where they pipe their find out to a 'rm' command. This is bad mo-jo in my opinion and should be avoided. Mass deleting files based on size alone is not a good practice. There are exceptions, but as a rule I avoid it like hoof and mouth disease.

Use this simple command to find large directories. To find directories over 1GB
[root@localhost]# du -h / | grep ^[0-9\.]*G

G can be replaced with M if looking for MB instead.

To find directories over 10GB and sort the output with the largest directories on top
[root@localhost]# du -h / | grep ^[1-9][0-9][0-9\.]*G | sort -rn

To find files within a size limit.
find /etc -size +100k -size -150k

Find files under /, and do not cross filesystems. This will keep you from searching pesky nfs and san mounts.

find / -name -depth +100k

Show directory sizes using du
 du -sk /usr/*

Find the largest 10 files
find / -size +15M -printf "%s - %p\n" | sort -n | tail

Cool Linux Command: ifstat

Ifstat is a little tool to report interface activity, just like iostat/vmstat do for other system statistics.

Usage is simple...

>ifstat
       eth0
 KB/s in  KB/s out
    0.87      0.61
    0.07      0.13
    0.13      0.13
    0.07      0.13
    0.13      0.13
    0.07      0.13
    0.13      0.13
    0.07      0.13
    0.13      0.13
    0.07      0.13