Infrastructure Adventures

07/30/2012

Creating Host Profile Answer Files with PowerCLI

Filed under: Virtualization — Tags: , , , , , , , — Joe Keegan @ 10:41 PM

Recently I was involved in a project with the goal of completely automating the creation and configuration of a VMware cluster. Basically we wanted to point a script at an empty vCenter server & a UCS manager and 30 minutes later have a fully configured cluster ready to add to our cloud.

One of the hurdles we had implementing this automation was getting a host profile to apply automatically to a newly AutoDeployed ESXi host. This was mostly related to automatically generating an answer file for a particular host. Here are the lessons learned.

Configuring MAC Addresses for VMKs

When working with host profiles in the past I wouldn’t pay too much attention to the process when I applied a host profile to a host.  I would get to the screen that asked for the MAC address for a VMK and would mindless hit next. ESX always generated the MAC just fine and I never gave it much thought. Unfortunately you can’t ignore this when trying to setup the answer file via PowerCLI.

The default for this settings is Prompt the user for the MAC Address if no default is available, but this seems to always need input. Luckily Paul Whitman had posted over at his blog that if you change this setting for each of your VMKs to User must explicily choose  the policy option then the system will automatically generate the MAC addresses.  This has to be a bug (at least in 5.0.0/441354), the wording of the option just doesn’t make any sense to me and they didn’t even spell explicitly correctly!

When editing your Host Profile the MAC address specification for your VMKs can be found in one of two places.

  • Networking configuration –> Host Port Group –> Port-Group Name –> Determine how MAC address for vmknic should be decided
  • Networking configuration –> Host virtual NIC –>  vDS Name : Port-Group Name -> Determine how MAC address for vmknic should be decided

Specifying the IP Addresses in the Answer File

It doesn’t seem like there is a way to create an answer file with any of the defined cmdlets. The Apply-VMHostProfile cmdlet has a way to “Specify a hash table object that provides values for the host profile required variables.” by using -Variable. As far as I can tell this allows you to specify the answers the host profile needs to apply the profile, but it doesn’t create an answer file. This may not be too big of a deal for a stateful boot-from-disk ESXi server, since the config changes made by the host profile stick around, but for a stateless AutoDeployed ESXi server which needs the answers everytime, its no good.

So get-view to the rescue. Again, as luck would have it, most of the leg work was done by someone else and I found this script in the Vmware PowerCLI community that pointed me in the right direction.

I’ve adapted that script for my own needs. The community script provided the info from a CSV file, but I already had all the info in Arrays in my script. Here is the meat as something stand alone.

$targetHost = Get-VMHost -Name "ESXiHost"
$hostProfileManagerView = Get-View "HostProfileManager"
$answerFileCreateSpec = New-Object VMware.Vim.AnswerFileOptionsCreateSpec

$propPath = New-Object VMware.Vim.ProfilePropertyPath
$propPath.ProfilePath = 'network.dvsHostNic["key-vim-profile-host-DvsHostVnicProfile-vDS-PG"].ipConfig'
$propPath.PolicyId = "IpAddressPolicy"

$addr = New-Object VMware.Vim.KeyAnyValue
$addr.key = "address"
$addr.value = "X.X.X.X"
$mask = New-Object VMware.Vim.KeyAnyValue
$mask.key = "subnetmask"
$mask.value = "255.255.255.0"

$param = New-Object VMware.Vim.ProfileDeferredPolicyOptionParameter
$param.InputPath = $propPath
$param.Parameter += $addr
$param.Parameter += $mask
$answerFileCreateSpec.UserInput += $param

$hostProfileManagerView.UpdateAnswerFile($targetHost.ExtensionData.MoRef, $answerFileCreateSpec)

Lines 1 -3: Get the host object and create some needed objects to create the answer file. Replace the name on line 1 with the name of your host.

Lines 5-7: Define the path to the info in the answer file that you are going to provide. You will need to replace the vDS and PG with the information from your vDS. See below for info on how to get this information.

Lines 9 -14: Provide the IP information, make sure to update lines 11 and 14 with the correct IP and subnet information.

Lines 16 – 20: finish constructing the details for the answer file, no changes are needed in this section.

Line 22: Creates the answer file based on the provided information.

Obviously you would need to do this for every VMK you need to provide IP information.

You can also use the HostProfileManager object  (stored in $hostProfileManagerView in the example above) to check the host profile.

PS>$hostProfileManagerView.CheckAnswerFileStatus($targetHost.ExtensionData.MoRef)

CheckedTime : 7/30/2012 1:05:37 AM
Host : HostSystem-host-5239
Status : valid
Error :
LinkedView :
DynamicType :
DynamicProperty :

You are looking for the Status of valid. If it’s invalid, then the answer file is not complete.

You can use the Get-VMHostProfileRequiredInput cmdlet to get the profile paths that you need to specify (i.e. what goes into line 6 of the above script) for the answer file configuration.

PS>Get-VMHostProfileRequiredInput -VMHost $VMHost -Profile $HostProfile | fl

Key : network.dvsHostNic["key-vim-profile-host-DvsHostVnicProfile-vDS-PG"].ipConfig.IpAddressPolicy.address
Type : Required
VMHost : esx-2.domain.com
VMHostProfile : HostProfile
Key : network.dvsHostNic["key-vim-profile-host-DvsHostVnicProfile-vDS-PG"].ipConfig.IpAddressPolicy.subnetmask
Type : Required
VMHost : esx-2.domain.com
VMHostProfile : HostProfile

Specifying the Root Password in the Answer File

The last part of the puzzle was setting the root password in the host profile. Since we were importing the host profile the root password is stripped from the host profile and it defaults to leaving the root password unchanged, which would be blank for the AutoDeployed host.

And for the third lucky find, it turns out LucD had already written a function called Set-VMHostProfileExtended that updates the root password stored in a host profile. Running this function against our imported host profile set the profile to update the root password and the password to the desired value.

Final Words

When I started this post I thought it would be nice to share a solution to a problem I was having. I didn’t realize it would mostly be just putting together solutions from three different people. But I guess that is what the Internet is all about and I’m glad they shared. Hopefully this post will save someone else the time it took for me to find and put these different components together.

03/19/2012

VMware Auto Deploy Rules & Rule Sets

Filed under: Compute, Virtualization — Tags: , , , — Joe Keegan @ 6:54 PM

I’ve been playing with Auto Deploy pretty much since vSphere 5 was released. I thought I would share some of the things I have learned around how Auto Deploy behaves when configured in certain ways. This is intended for folks who are somewhat familiar with Auto Deploy.

Deploy Rules

The heart of the Auto Deploy configuration is the deploy rules.  A deploy rule includes three major parts, the rule name, the rule items and the rule pattern. Deploy rule name can be pretty much anything reasonable. Info on items and patterns below.

Deploy Rule Items

There are three attributes for deploy rule items. These include the Image Profile, the VC Inventory Location and the Host Profile.

A deploy rule is created with the New-DeployRule cmdlet:


New-DeployRule -Name Example -Item ImageProile<,Location><,HostProfile> -Pattern Pattern

The Image Profile (Mandatory)

This is the ESXi image that will be used to boot the server. You can use a standard image profile from VMware or customize an image profile for your environment. See this post on Auto Deploy and HP servers for an idea on customizing your images.

You can add the VMware online depot to get the standard image profiles like so:

Add-EsxSoftwareDepot https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

leaving an image profile out of a deploy rule will just cause the server to sit at the gPXE screen for five minutes and then reboot. This is the same behavior as if there wasn’t a rule that matched the host.

VC Inventory Location (Optional)

This can be the name (or PowerCLI object) of the Data Center, Folder or Cluster in which the host should be added after the host has finished booting the image profile. If you do not specify the VM inventory location then the host is added as a stand alone host to a Data Center (not sure how it chooses which DC if you have more then one).

While this technically optional, I would say that it’s best practice to always specify a location.

Host Profile (Optional)

This is the name (or PowerCLI object) of the Host Profile that will be applied to the host after it has booted and been added to vCenter.

The host profile is attached to the host the very first time the host is added to vCenter. If any items within the host profile are set to prompt for input then the host will remain in maintenance mode and you will need to apply the host profile manually. A answer file is created using the information provided to the host profile. This answer file is stored by vCenter and used to provide the answers for each subsequent time the auto deploy is rebooted.

You do not need to specify the host profile in the rule, but you can still have a host profile automatically applied to an host provisioned via auto deploy. You can do this by either manually attaching a host profile to the host or attaching a host profile to a cluster that contains the auto deployed host. In either case you will need to manually apply the host profile (just like if it was attached via a deploy rule)  and create an answer file the first time it is added to vCenter. The auto deployed host will have the host profile automatically attached and applied after subsequent reboots.

Deploy Rule Patterns

Auto deploy uses a pattern to match the rule that should be applied to a host. What can be included in the pattern can be found in the VMware documentation, but the list includes:

Pattern Description
vendor Machine vendor name.
model Machine model name.
serial Machine serial number.
hostname Machine hostname. Retrieved from DNS.
domain Domain name.
ipv4 IPv4 address of the machine. Ranges can be specified like ipv4=10.1.1.0-10.1.1.255
mac Boot NIC MAC address.
asset Machine asset tag.
oemstring OEM-specific strings in the SMBIOS. How useful this is depends on the vendor. Cisco’s UCS provides service profile information which is quite useful.

Multiple matches can be specified for each pattern type. So for example you can specify multiple MAC addresses like so:


New-DeployRule -Name Example -Item Image,Location,Profile -Pattern mac="00:17:a4:77:04:00,00:17:a4:77:04:FF"

You can even specify multiple ranges of IP addresses:

New-DeployRule -Name Example -Item Image,Location,Profile -Pattern ipv4="172.17.112.194-172.17.112.254,10.10.112.194-10.10.112.254".

Just make sure to include the quotation marks.

The pattern matching you should use depends on your vSphere architecture. I would expect that in most designs each cluster  would be assigned an IP range using DHCP reservations and that ipv4 pattern matching would be the most prevalent. If your design does not include a predictable method of assigning a host to a cluster then check out this post on running Auto Deploy in Semi Autonomous Mode.

And there is the -AllHosts flag which will cause a rule to apply to all hosts.

Deploy Rule Sets

Once you create a deploy rule you need to add it the the active deploy rule set for it to be used to provision ESXi hosts. You can add a rule to the rule set via the Add-DeployRule cmdlet.

Add-DeployRule -DeployRule <DeployRule>

Deploy Rule Order

The order of the deploy rules that are include in a deploy rule matter. The first rule that matches a host will be applied. So for example say we have a rule set with the following three deploy rules.


PS>Get-DeployRuleSet

Name : Special
PatternList : {ipv4=10.1.1.2}
ItemList : {SpecialImage, Cluster, HostProfile}

Name : General
PatternList : {ipv4=10.1.1.0-10.1.1.254}
ItemList : {Image, Cluster, HostProfile}

Name : CatchAll
PatternList :
ItemList : {Image, Folder}

The first rule, Special, will apply to a host with IP address 10.1.1.2, while the rule General will apply to any other hosts in the range and then finnally the last rule is a catchall they will apply to any hosts (the rule was added with the -AllHosts flag).

While there is very little indication of this each rule is assigned a rule number starting at 0. So the Special rule is rule 0, General is rule 1 and CatchAll is rule 3. If you added another rule it would be added as rule 4 by default. This wouldn’t work so well since the CatchAll rule would be applied before the new rule and the new rule would never be used.

To get around this you will need to add the deploy rule to the rule set as a specific number. Say for example we wanted this new rule to go below the General rule and above the CatchAll rule. Then you would do something like this:


New-DeployRule -Name NewRule -Item Image,NewCluster,NewHostProfile -Pattern  ipv4=10.1.2.0-10.1.2.254
Add-DeployRule -DeployRule NewRule -At 2

Name : Special
PatternList : {ipv4=10.1.1.2}
ItemList : {SpecialImage, Cluster, HostProfile}

Name : General
PatternList : {ipv4=10.1.1.0-10.1.1.254}
ItemList : {Image, Cluster, HostProfile}

Name : NewRule
PatternList : {ipv4=10.1.2.0-10.1.2.254}
ItemList : {Image, NewCluster, NewHostProfile}

Name : CatchAll
PatternList :
ItemList : {Image, Folder}

If you wanted the NewRule to go in front of the General rule then you specify -At 1, and if you wanted it to be the first rule you would specify -At 0.

Active and Working Rule Sets

Deploy rules can be added to either the active rule set or the working rule set. The active rule set are the rules that Auto Deploy will use to evaluate requests from booting servers. The working rule set is an inactive rule set that you can use to make changes without impacting live operations. The idea is that if you want to make a lot of changes to a rule set that you make the working rule set look the way you want and then make it the active rule set.

By default deploy rules are added to the active rule set. You add rules to the working rule set by including the -NoActivate flag when using the Add-DeployRule cmdlet.

So as an example I can look at my current rule set and see that it just includes a single general rule.

PS>get-deployruleset
Name : General
PatternList : {ipv4=172.17.112.194-172.17.112.254}
ItemList : {Image, Cluster}

Looking at my existing deploy rules I can see that I have a specific and allhosts rule that i would like to add.

PS>get-deployrule
Name : Specific
PatternList : {ipv4=172.17.112.194-172.17.112.254}
ItemList : {Image, Cluster, HostProfile}

Name : General
PatternList : {ipv4=172.17.112.194-172.17.112.254}
ItemList : {Image, Cluster}

Name : AllHosts
PatternList :
ItemList : {Image}

I can use the Set-DeployRuleSet cmdlet to create the working rule set. I could also use the Add-DeployRule cmdlet to add the new rules, but since I want one rule before the General rule and one rule after the General Rule I would need to do Add-DeployRule twice and specify their rule numbers (see above). Instead I can use the Set-DeployRuleSet and do it all in one command. Using the -NoActivate makes the command apply the the working rule set.

PS>Set-DeployRuleSet -DeployRule Specific,General,AllHosts -NoActivate
Name : Specific
PatternList : {ipv4=172.17.112.194-172.17.112.254}
ItemList : {Image, Cluster, HostProfile}

Name : General
PatternList : {ipv4=172.17.112.194-172.17.112.254}
ItemList : {Image, Cluster, HostProfile}

Name : AllHosts
PatternList :
ItemList : {Image}

Now if I get the active rule set and the working rule set I can see that the active rule set still contains a single General rule, while the working rule set has all three rules. I provide the -Working flag to the Get-RuleSet cmdlet to see the working rule set.

</pre>
PS>Get-DeployRuleSet
Name : General
PatternList : {ipv4=172.17.112.194-172.17.112.254}
ItemList : {Image, Cluster}
<pre>PS>Get-DeployRuleSet -Working
Name : Specific
PatternList : {ipv4=172.17.112.194-172.17.112.254}
ItemList : {Image, Cluster, HostProfile}

Name : General
PatternList : {ipv4=172.17.112.194-172.17.112.254}
ItemList : {Image, Cluster}

Name : AllHosts
PatternList :
ItemList : {Image}

Now I can use the Switch-ActiveDeployRuleSet to make the working rule set the active rule set.

PS>Switch-ActiveDeployRuleSet
PS>Get-DeployRuleSet
Name : Specific
PatternList : {ipv4=172.17.112.194-172.17.112.254}
ItemList : {Image, Cluster, HostProfile}

Name : General
PatternList : {ipv4=172.17.112.194-172.17.112.254}
ItemList : {Image, Cluster}

Name : AllHosts
PatternList :
ItemList : {Image}

03/15/2012

Semi Automated Auto Deploy

Filed under: Compute, Virtualization — Tags: , , , — Joe Keegan @ 6:27 PM

I was asked the other day about using Auto Deploy in a semi automated fashion. Basically they wanted to know if they could use auto deploy to provision ESXi, but they didn’t want it to automatically add it to a cluster or apply a specific host profile. They wanted to make the cluster changes and apply the host profile manually and they wanted the cluster and host profile assignments to survive reboots.

I wasn’t sure if this would work, but testing in the lab shows that it works exactly like they wanted. Basically you just create a folder in vCenter for new hosts. Create a deploy rule that adds new hosts to that folder. New hosts are placed into that folder and you can manually drag them into the desired cluster. Attach and apply the desired host profile, or even better yet have the host profile attached to the cluster. Lastly answer the host profile questions to create an answer file.

Now next time you reboot the host the auto deploy server will provision ESXi and add it to vCenter. When the host is added to vCenter it will go back into the correct cluster and the host profile will automatically be applied. This of course requires for the ESXi vCenter object to still exist when the host comes back up, otherwise it will just be added again to the folder.

I’m not sure I would advocate this approach in a production environment. I think it’s probably best to have a rule for each cluster either based on IP ranges, MAC addresses, etc. But for a development or lab environment I think this could be useful.

Older Posts »

The Silver is the New Black Theme Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 32 other followers

%d bloggers like this: