FatMin

Cisco UCS Visio Stencils

Cisco UCS Visio Stencils -- Use the link below and scroll down to "Unified Computing and Servers"

http://www.cisco.com/en/US/products/hw/prod_cat_visios.html

Cisco UCS Firmware Best Practices Document

Not sure that I have ever run into a product with as many separate support documents as Cisco UCS - here is another that appears to be useful (the jury is still out on this one). Anyway the link is below

http://goo.gl/KOm8J

The link below will take you to the Firmware update and install guides -- note that Cisco has not upgraded the documentation to reflect an upgrade from an older 2.0 firmware to a newer 2.0 firmware.

http://www.cisco.com/en/US/products/ps10281/prod_installation_guides_list.html

 

How Not to Assign KVM IP Addresses Via Cisco UCS Manager

After a few hours poking around a newly deployed UCS cluster trying to get some basic profiles created and assigned. I realized that I had actually no idea how the KVM is actually supposed to work inside the UCS cluster. Which is funny as this was a subject that we touched on during my DCUDC class. Apparently we did not touch on it enough.

Anyway, before I get ahead of myself, lets review the gear in this cluster.

_{2 5108 chassis
7 B200 M2 blades with 2104 IOMs
2 6248s Fabric Interconnects}

Now in my network all lights out management ips (ilos, ipmi, etc) are all on one particular vlan, which for the purpose of this post we will call VLAN 100. Non application related infrastructure equipment (servers, virtual hosts) are on another vlan, which we will call VLAN 200. So when the Fabric Interconnets were deployed, I gave them each an ip address on VLAN 200. And once UCS Manager was up and running, I created a KVM ip address pool of unused ip addresses on VLAN 100. Well guess what, this is wrong.

Routing for the KVM addresses is done through the management interfaces on the Fabric Interconects, so unless you are using vlan tagging, your KVM pool must be on the same vlan as the ip addresses assigned to your Fabric Interconnects.

But wait, why is this?

I thought that I could even assign private 192.168.x.x ip addreses to the KVMs as they were only supposed to be managed via the UCS Manager, but this also incorrect.

Navigate to one of your working KVM ip addresses in a web browser and you can access the KVM of the blade outside of UCS Manager. Nice, which is how I actually expected this to work. 

Note that I find it rather dumb to have my KVM management ips and Fabric Interconnects on the same vlan as a rule, however since this is how its supposed to work I am going to have to let that one go.

Now, the fact that you can navigate to a specific KVM IP address via a web browser also makes the idea of using a pool of ip addresses silly. Would you not want to hard code the KVM ip address in the service profile so that you always know which server's console you are logging into? Dunno, I am still working on figuring that one out.

 

Cisco UCS Manager (2.0) Configuration Guide

This is the document that you will want to have access to in order to configure your USC Cluster Manager.

http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/gui/config/guide/2.0/b_UCSM_GUI_Configuration_Guide_2_0.pdf

Cisco 6248 Fabric Interconnect CLI Guide

Cisco calls this document the "Cisco UCS Manager CLI Configuration Guide", which is not exactly the best description this document. I call this document the "Cisco 6248 Fabric Interconnect CLI Guide", and you can too. Just print out a copy and stick a piece of tape over the title page, and boom, you have renamed the document.

http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/cli/config/guide/2.0/b_UCSM_CLI_Configuration_Guide_2_0.pdf

XenServer Switch Ports Configuration Best Practices

Finally I have found it! Citrix's XenServer switch configuration best practices document.

While everyone in the world has blog posts and documentation regarding how to set up and configure bridged networks in Xen, they hardly ever go into the physical switch configuration required.

This is the document that you will need to pass along to your friendly Network Administrator, as they will more than likely not be familar with networking for Xen as its much different from networking for Vmware ESX.

http://support.citrix.com/article/CTX123158

The contents of the document above are outlined below.

Change the following options on the switches for XenServer ports:

Enable PortFast on XenServer connected ports.
PortFast allows a switch port running Spanning Tree Protocol (STP) to go directly from blocking to forwarding mode by skipping the learning and listening modes. PortFast should only be enabled on ports connected to a single host. Port must be an 802.1q trunk port if you are using VLANS and the port must be in access mode.
Ports used for storage should have PortFast enabled.
Note: It is important that you enable PortFast with caution, and only on ports that do not connect to multi-homed devices such as hubs or switches.
Disable Port Security on XenServer connected ports.
Port security prevents multiple MAC addresses from being presented to the same port. In a virtual environment, you see multiple MAC addresses presented from Virtual Machines to the same port. If you have enabled Port Security, it shuts down the port.
Disable Spanning Tree Protocol on XenServer connected ports.
Spanning Tree Protocol must be disabled if you are using Bonded or teamed NICs in a virtual environment. Spanning Tree Protocol should be disabled because of the nature of Bonds and NIC teaming, to avoid failover delay issues when using bonding.
Disable BPDU guard on XenServer connected ports.
BPDU is a protection setting part of the STP that prevents you from attaching a network device to a switch port. When you attach a network device, the port shuts down and has to be enabled by an administrator.
A PortFast port should never receive configuration BPDUs.
Note: When BPDUs are received by a PortFast port, it indicates another bridge is connected to the port, and it indicates that there is a possibility of a bridging loop formation during the Listening and Learning phases. In a valid PortFast configuration, configuration BPDUs should never be received, so Cisco switches support a feature called PortFast BPDU Guard, which is a feature that shuts down a PortFast-enabled port in the event a BPDU is received. This feature ensures that a bridging loop is not formed, because the switch's shutting down the port removes the possibility of a loop forming.

Forcibly Remove Storage Devices From XenCenter via CLI

This is my second post in focused on forcibily something from XenCenter, as apparently XenCenter has a hard time knowing when to "forget" a device or host on its own. Now while this is annoying, it can be resolved rather easily from the command line.

Now, lets take a step back and reflect for a moment on the first post in this series... you can find it here. Anyway this first post showed us how to remove the orphaned device via the Windows CLI, however I find the process of using the CLI in Windows to be painful, so this post is going to outline a similar process but via the CLI on the XenServer host itself.

Anyway in the previous post I had to force remove a host from XenCenter via the CLI, and in this post I need to remove the "Local Storage", "Removable Storage" and "DVD Drive" that were associated with the orphaned host. These are the items in red below.

By clicking on each one of the items in XenCenter I can see that they are all in a Detached State, plus i can see the UUID of each item. You will need the UUID for the next step.

Now ssh into your master node and run the following

#xe sr-list

Look for the entry with the matching UUID

uuid ( RO)                : 64e69b7d-ae97-4d42-c4d8-c260553b31d1
name-label ( RW): Local storage
name-description ( RW):
host ( RO): <not in database>
type ( RO): lvm
content-type ( RO): user

Then run the following command to remove it from XenCenter Inventory

# xe sr-forget uuid=64e69b7d-ae97-4d42-c4d8-c260553b31d1

Now rinse and repeat for any remaining orphaned items.

 

Forcibly Remove a Host From XenCenter via CLI

Welcome back to another XenServer adventure. Today we are going to review the process of removing an orphaned XenServer host from XenCenter via the command line.

A few days ago one of my Xen hosts died due to an issue with the raid controller. In order to revive it I had to rebuild it, and add it back into the pool. However there was a problem; the original entry for this host in XenCenter appeared in red, which indicated that it was disconnected, and another entry for the same exact host with the same exact hostname appeared next to it in the host list for this particular cluster. I was unable to remove the orphaned host via XenCenter as there was no remove option.

I was in a pickle.

However I was able to find a solution.

First and formost it appears that there are actually two ways to do this; the first one being from the Windows command line where you are running Xencenter, or on the master node in the cluster.

This example is from the Windows CLI. Note that the UUID that is output to the command line is the UUID of the Host that is unreachable -- the orphaned host.


C:\Program Files\Citrix\XenCenter>xe -s <Hostname_of_master> -u root -pw <root_password> pool-sync-database
You attempted an operation which involves a host which could not be contacted.
host: 560e233b-0e8b-4c2f-a641-cf1876630a6b (virt04.atlc1)

C:\Program Files\Citrix\XenCenter>xe -s <Hostname_of_master> -u root -pw <root_password> host-foget uuid=560e233b-0e8b-4c2f-a641-cf1876630a6b
WARNING: A host should only be forgotten if it is physically unrecoverable;
WARNING: if possible, Hosts should be 'ejected' from the Pool instead.
WARNING: Once a host has been forgotten it will have to be re-installed.
WARNING: This operation is irreversible.
Type 'yes' to continue
yes

Returning to XenCenter I found that the orphaned host entry had been removed.