Infrastructure Adventures

09/24/2012

Update vSphere Host Profile from Reference Host via PowerCLI

Filed under: Virtualization — Tags: , , , — Joe Keegan @ 7:18 PM

I’m dealing with many clusters of stateless hosts deployed via vSphere AutoDeploy and updating host profiles from reference host is something I badly needed. I was surprised this cmdlet didn’t exist so I wrote a function to do the job. Hope folks find this useful.

Function Update-VMHostProfile {
	<# 		
	.SYNOPSIS 			
		Updates the host profile from its reference host.		
	
	.DESCRIPTION 			
		The Update-VMHostProfile updates a host profile from its reference host.
		
	.PARAMETER  Profile
		Specify the host profile in which you wish to update from reference host.
		
	.EXAMPLE 			
		C:\PS> Update-VMHostProfile -Profile $Profile

		Updates the host profile $Profile from its reference host.

	.EXAMPLE 			
		C:\PS> Get-VMHost -Name $Host | Get-VMHostProfile | Update-VMHostProfile

		Get's the host profile attached to the VMHost $Host and updates it from the host profile's reference host.
	
	.EXAMPLE 			
		C:\PS> Get-VMHostProfile | Update-VMHostProfile

		Updates all host profiles from reference host.

	.NOTES
			Function has only been tested with vSphere 5.1, should work with vSphere 5.0.

	.LINK

http://infrastructureadventures.com/

	#>

	[CmdletBinding()]
	Param (
		[Parameter(Mandatory=$True,ValueFromPipeline=$True)]
		$Profile
	)

	BEGIN {}

	PROCESS {
	
		if ( ($Profile.GetType()).name -eq "String" ) {
			$Profile = Get-VMHostProfile -Name $Profile
		}

		$HostProfileReferenceHostId = ("host-" + (($Profile.ReferenceHost.Id).split("-"))[2])
	
		$HostProfileHostBasedConfigSpec = New-Object VMware.Vim.HostProfileHostBasedConfigSpec
		$HostProfileHostBasedConfigSpec.host = New-Object VMware.Vim.ManagedObjectReference
		$HostProfileHostBasedConfigSpec.host.type = "HostSystem"
		$HostProfileHostBasedConfigSpec.host.Value = $HostProfileReferenceHostId
		$HostProfileHostBasedConfigSpec.useHostProfileEngine = $true

		$HostProfileView = Get-View -Id $Profile.Id
		$HostProfileView.UpdateHostProfile($HostProfileHostBasedConfigSpec)
		
		$Profile
	}

	END {}
}

Managing SSD LUNs and vSphere Host Cache Configuration via PowerCLI

Filed under: Storage, Virtualization — Tags: , , , , , , — Joe Keegan @ 6:13 PM

I have been experimenting with SSD and Host Cache in my home lab. I am also trying to do as much with PowerCLI as possible. Coming from a Unix background I think it would be awesome to be able to do basically everything in vSphere (and vCloud) via a CLI. Turns out there are not any cmdlets to enable the SSD options on LUNs or manage Host Cache configurations, so I wrote some functions to do that.

I imagine this could be particularly useful for someone who plans on using Host Cache and has a clusters of hosts they need to configure to use that feature. My plan is to write more “missing” cmdlets and make a module available in the near future.

Set-SSDScsiLUN and Get-SSDScsiLUN

Host Cache requires data store backed by SCSI LUNs to be marked as SSDs. Presumably ESXi will detect and automatically mark actual SSD backed data stores as SSD. But for home labs or instances where ESXi doesn’t correct identify the SCSI LUN as an SSD, the LUN can be marked as SSD manually. This can be done via ESXCLI and it turns out you can call ESXCLI via PowerCLI’s Get-ESXCLI cmdlet.

The Set-SSDScsiLUN function is a PowerCLI wrapper for the Get-ESXCLI cmdlet that will mark the specified SCSI LUNs as SSD backed. This works with the pipeline so you can mark a LUN with a specific Canonical Name as SSD for an entire cluster or data center with a single line.

Since this is a wrapper for ESXCLI and involves calling a process to “re-claim” all the LUNs there is a slight lag between running this command and having the LUNs shows up as SSD.

Function Set-SSDScsiLUN {
	<#
		.SYNOPSIS
			Adds or Removes the "enable_ssd" option for a specified SCSI LUN.

		.DESCRIPTION
			The Set-SSDScsiLUN function uses the ESXCLI to update the SATP claim rule for the specified SCSI LUN. Please note there is a few second delay for the claim rules to run and for the data store to show up as SSD.
			
		.PARAMETER  ScsiLun
			Use this parameter to specify the SCSI LUN you wish to enable/disable the SSD option.

		.PARAMETER  Disabled
			Used to disable/unset the SSD option on the sprcified LUN.
		
		.PARAMETER  ReClaimOnly
			It's possible to end up in a situation where the claim rule has been added or removed, but for whatever reason the claim process does not complete successfully. This can be caused by running this function several rimes in a row. Using this switch will just rerun the claim process. If you run the Set-SSDScsiLUN function and the change does not take effect, you may want to try running with the ReClaimOnly switch.	
		
		.EXAMPLE
			C:\PS> Set-SSDScsiLUN -ScsiLun $SCSILUN
			
			Enables the SSD option for $SCSILUN
			
		.EXAMPLE
			C:\PS> Set-SSDScsiLUN -ScsiLun $SCSILUN -Disabled
			
			Disables the SSD option for $SCSILUN
		
		.Example
			C:\PS> Get-VMHost | Get-ScsiLun -CanonicalName $SCSILUN | Set-SSDScsiLUN
			
			Enables the SSH option for all hosts with a SCSI LUN matching the Canonical Name of $SCSILUN (e.g. mpx.vmhba1:C0:T1:L0)

		.EXAMPLE
			C:\PS> Get-Datastore | where { $_.name -like "*HostCache*" } | Get-ScsiLun | Set-SSDScsiLUN
			
			Enables the SSD option for all LUNs that belong to data stores with "HostCache" as part of thier name.			
	
		.NOTES
			Function has only been tested with vSphere 5.1, should work with vSphere 5.0. This fucntion has only been tested for local disks.		
	
		.LINK

http://infrastructureadventures.com/

	#>
	
	[CmdletBinding()]
	Param (
		[Parameter(Mandatory=$True,ValueFromPipeline=$True)]
		$ScsiLun,
		[Switch]
		$Disabled,
		[Switch]
		$ReClaimOnly
	)

	BEGIN {}

	PROCESS {

		if ( ($ScsiLun.GetType()).name -eq "String" ) {
			$ScsiLun = Get-ScsiLun -CanonicalName $ScsiLun
		}

		$LUN_CName = $ScsiLun.CanonicalName

		$VMHost = get-vmhost -name $ScsiLun.VMHost
		$ESXCLI = $VMHost | get-EsxCli
		$SATP = ($ESXCLI.storage.nmp.device.list($LUN_CName))[0].StorageArrayType
		$ClaimRules = $ESXCLI.storage.nmp.satp.rule.list()

		if ( $ReClaimOnly -eq $False ) {
			if ( $Disabled -eq $True ) {
				if ( $ESXCLI.storage.core.device.list($LUN_CName)[0].IsSSD -eq "true" ) {
					foreach ( $Rule in $ClaimRules ) {
						if ( $Rule.Device -eq $LUN_CName -and $Rule.Options -eq "enable_ssd" ) {
						$ESXCLI.storage.nmp.satp.rule.remove($FALSE,"","",$LUN_CName,"","","enable_ssd","","",$SATP,"","","") | Out-Null
						$ESXCLI.storage.core.claiming.reclaim($LUN_CName)
						}
					}
				}
			} elseif ( $Disabled -eq $False ) {
				if ( $ESXCLI.storage.core.device.list($LUN_CName)[0].IsSSD -eq "false" ) {
					foreach ( $Rule in $ClaimRules ) {
						if ( $Rule.Device -eq $LUN_CName -and $Rule.Options -eq "enable_ssd" ) {
							$RuleExists = $True
							break
						}
					}
					if ( $RuleExists ) {
						$ESXCLI.storage.core.claiming.reclaim($LUN_CName)
					} else {
						$ESXCLI.storage.nmp.satp.rule.add($FALSE,"","",$LUN_CName,"",$FALSE,"","enable_ssd","","",$SATP,"","","") | Out-Null
						$ESXCLI.storage.core.claiming.reclaim($LUN_CName)
					}
				}
			}
		} elseif ( $ReClaimOnly ) {
			$ESXCLI.storage.core.claiming.reclaim($LUN_CName)
		}
	$ScsiLun
	}

	END {}
}

This next function will output the LUNs in an existing data store that have been marked as SSD. I went back and forth on what should be the input and output for this function, and I admit I”m still not entirely satisfied. If anyone has any suggestions, i’d be up for a re-write.

Function Get-SSDScsiLUN {
	<#
		.SYNOPSIS
			Gets SCSI LUN objects that are marked as SSD.

		.DESCRIPTION
			The Get-SSDScsiLUN function outputs the SCSI LUNs that belong to a data store that are marked as SSD based. Please note that it takes a couple of seconds for the changes made by the Set-SSDScsiLUN function to complete.

		.PARAMETER  DataStore
			Use this parameter to specify the data store you wish to see if the SSD option has been set.

		.EXAMPLE
			C:\PS> Get-SSDScsiLUN -DataStore $DataStore
			
			Outputs to the SCSI LUN objects that are marked as SSD for the specified $DataStore.
			
		.NOTES	
			Function has only been tested with vSphere 5.1, should work with vSphere 5.0.	
	
		.LINK

http://infrastructureadventures.com/

	#>
	
	[CmdletBinding()]
	Param (
		[Parameter(ValueFromPipeline=$True)]
		$DataStore
	)

	BEGIN {}

	PROCESS {
		if ( $DataStore ) {
			if ( ($DataStore.GetType()).name -eq "String" ) {
				$DataStore = Get-Datastore -Name $DataStore
			}
		} else {
			$DataStore = Get-DataStore | where { $_.Type -eq "VMFS" }
		}
		$DataStore | where { $_.Type -eq "VMFS" } | Get-ScsiLUN | where { $_.ExtensionData.Ssd -eq "True" }
	}

	END {}
}

Set-VMHostCache and Get-VMHostCache

Now really this is all about setting the Host Cache Space on an SSD backed LUN and the Set-VMHostCache can be used to do just that. Simply provide it a SSD backed data store object and it will configure a Host Cache Space as large as the available free space on the data store, or optionally provide a size in GBs. Specifying zero (0) for the size will turn Host Cache off.

And there is a Get-VMHostCache function which can be used to see the Host Cache Space defined on each data store.

Function Set-VMHostCache {
	<#
		.SYNOPSIS
			Configures Host Cache Space on a specified SSD backed Data Store.

		.DESCRIPTION
			The Set-VMHostCache function configures Host Cache Space on a specified SSD backed Data Store.
			
		.PARAMETER  DataStore
			Use this parameter to specify the Data Store you would like to use for the Host Cache Space.
		
		.PARAMETER  Space
			This parameter specifies the space in GBs that should be used for the Host Cache Space. Omitting this parameter will cause the Host Cache Space to be set to the maximum amount of space available on the data store.

		.PARAMETER  Disabled
			Used to disable/remove the Host Cache Space from the specified Data Store.
		
		.EXAMPLE
			C:\PS> Set-VMHostCache -DataStore $datastore -Size 15
			
			Sets the Host Cache Space on Data Store $datestore to 15 GBs.
			
		.EXAMPLE
			C:\PS> Set-VMHostCache -DataStore $datastore -Disabled
			
			Removes the Host Cache Space from $datastore.

		.EXAMPLE
			C:\PS> Get-VMHost | New-Datastore -VMFS -Path $SSDSCSILUN -Name "HostCache" | Set-VMHostCache
			
			Creates a Data Store on each host with the SCSI LUN $SSDSCSILUN called HostCache and then sets the Host Cache Space to equal the maximum space available on the data store.		
	
		.NOTES
			Function has only been tested with vSphere 5.1, should work with vSphere 5.0.		
	
		.LINK

http://infrastructureadventures.com/

	#>
	
	[CmdletBinding()]
	Param (
		[Parameter(Mandatory=$True,ValueFromPipeline=$True)]
		$DataStore,
		[int]
		$Space,
		[Switch]
		$Disabled
	)

	BEGIN {}

	PROCESS {

		if ( ($DataStore.GetType()).name -eq "String" ) {
			$DataStore = Get-Datastore -Name $DataStore
		}

		$HostCacheConfigurationSpec = New-Object VMware.Vim.HostCacheConfigurationSpec
		$HostCacheConfigurationSpec.datastore = New-Object VMware.Vim.ManagedObjectReference
		$HostCacheConfigurationSpec.datastore.type = "Datastore"
		$HostCacheConfigurationSpec.datastore.Value = ($DataStore.id).substring(10,13)

		if ( $Disabled ) {
			$HostCacheConfigurationSpec.swapSize = 0
		} else {
			if ( $Space ) {
				$SpaceMB = $Space * 1024
				if ( $SpaceMB -gt $DataStore.FreeSpaceMB ) {
					Write-Warning ("Specified Host Cache Space size is larger then avialable space, setting Host Cache Space size to maximum avialable space.")
					$HostCacheConfigurationSpec.swapSize = $DataStore.FreeSpaceMB
				} else {
					$HostCacheConfigurationSpec.swapSize = $SpaceMB
				}
			} else {
				$HostCacheConfigurationSpec.swapSize = $DataStore.FreeSpaceMB
			}
		}

		$HostCacheConfigurationManager_ID = ("HostCacheConfigurationManager-cacheConfigManager-" + (($DataStore.ExtensionData.host[0].Key.value).split("-"))[1])
		$HostCacheConfigurationManager = Get-View -Id $HostCacheConfigurationManager_ID
		$HostCacheConfigurationManager.ConfigureHostCache_Task($HostCacheConfigurationSpec) | Out-Null

		$DataStore

	}

	END {}
}

 


Function Get-VMHostCache {
	<#
		.SYNOPSIS
			Gets the Host Cache configuration for the specified host.

		.DESCRIPTION
			The Get-VMHostCache function gets the Host Cache configuration for a specified host. It displays the amount of space on each data store that has been configured as Host Cache Space.
			
		.PARAMETER  VMHost
			Use this parameter to specify the Host that you wish to get the host cache configuration.
		
		.EXAMPLE
			C:\PS> Get-VMHostCache -VMHost $Host
			
			Gets the Host Cahce configuration for $Host
			
		.NOTES
			Function has only been tested with vSphere 5.1, should work with vSphere 5.0.		
	
		.LINK

http://infrastructureadventures.com/

	#>
	
	[CmdletBinding()]
	Param (
		[Parameter(Mandatory=$True,ValueFromPipeline=$True)]
		$VMHost
	)

	BEGIN {}

	PROCESS {

		if ( ($VMHost.GetType()).name -eq "String" ) {
			$VMHost = Get-VMHost -Name $VMHost
		}

		$HostCacheConfigurationManager_ID = ("HostCacheConfigurationManager-cacheConfigManager-" + (($VMHost.id).split("-"))[2])
		$HostCacheConfigurationManager = Get-View -Id $HostCacheConfigurationManager_ID

		foreach ( $HostCacheDataStore in $HostCacheConfigurationManager.CacheConfigurationInfo ) {
			$DataStore_ID = ("Datastore-" + $HostCacheDataStore.key.value)
			$Datastore = Get-DataStore -Id $DataStore_ID

			$DataStore | select name, @{Name="HostCacheSpaceGB";Expression={$HostCacheDataStore.SwapSize / 1024}}, FreeSpaceGB, CapacityGB
		}
	}

	END {}
}

07/30/2012

Creating Host Profile Answer Files with PowerCLI

Filed under: Virtualization — Tags: , , , , , , , — Joe Keegan @ 10:41 PM

Recently I was involved in a project with the goal of completely automating the creation and configuration of a VMware cluster. Basically we wanted to point a script at an empty vCenter server & a UCS manager and 30 minutes later have a fully configured cluster ready to add to our cloud.

One of the hurdles we had implementing this automation was getting a host profile to apply automatically to a newly AutoDeployed ESXi host. This was mostly related to automatically generating an answer file for a particular host. Here are the lessons learned.

Configuring MAC Addresses for VMKs

When working with host profiles in the past I wouldn’t pay too much attention to the process when I applied a host profile to a host.  I would get to the screen that asked for the MAC address for a VMK and would mindless hit next. ESX always generated the MAC just fine and I never gave it much thought. Unfortunately you can’t ignore this when trying to setup the answer file via PowerCLI.

The default for this settings is Prompt the user for the MAC Address if no default is available, but this seems to always need input. Luckily Paul Whitman had posted over at his blog that if you change this setting for each of your VMKs to User must explicily choose  the policy option then the system will automatically generate the MAC addresses.  This has to be a bug (at least in 5.0.0/441354), the wording of the option just doesn’t make any sense to me and they didn’t even spell explicitly correctly!

When editing your Host Profile the MAC address specification for your VMKs can be found in one of two places.

  • Networking configuration –> Host Port Group –> Port-Group Name –> Determine how MAC address for vmknic should be decided
  • Networking configuration –> Host virtual NIC –>  vDS Name : Port-Group Name -> Determine how MAC address for vmknic should be decided

Specifying the IP Addresses in the Answer File

It doesn’t seem like there is a way to create an answer file with any of the defined cmdlets. The Apply-VMHostProfile cmdlet has a way to “Specify a hash table object that provides values for the host profile required variables.” by using -Variable. As far as I can tell this allows you to specify the answers the host profile needs to apply the profile, but it doesn’t create an answer file. This may not be too big of a deal for a stateful boot-from-disk ESXi server, since the config changes made by the host profile stick around, but for a stateless AutoDeployed ESXi server which needs the answers everytime, its no good.

So get-view to the rescue. Again, as luck would have it, most of the leg work was done by someone else and I found this script in the Vmware PowerCLI community that pointed me in the right direction.

I’ve adapted that script for my own needs. The community script provided the info from a CSV file, but I already had all the info in Arrays in my script. Here is the meat as something stand alone.

$targetHost = Get-VMHost -Name "ESXiHost"
$hostProfileManagerView = Get-View "HostProfileManager"
$answerFileCreateSpec = New-Object VMware.Vim.AnswerFileOptionsCreateSpec

$propPath = New-Object VMware.Vim.ProfilePropertyPath
$propPath.ProfilePath = 'network.dvsHostNic["key-vim-profile-host-DvsHostVnicProfile-vDS-PG"].ipConfig'
$propPath.PolicyId = "IpAddressPolicy"

$addr = New-Object VMware.Vim.KeyAnyValue
$addr.key = "address"
$addr.value = "X.X.X.X"
$mask = New-Object VMware.Vim.KeyAnyValue
$mask.key = "subnetmask"
$mask.value = "255.255.255.0"

$param = New-Object VMware.Vim.ProfileDeferredPolicyOptionParameter
$param.InputPath = $propPath
$param.Parameter += $addr
$param.Parameter += $mask
$answerFileCreateSpec.UserInput += $param

$hostProfileManagerView.UpdateAnswerFile($targetHost.ExtensionData.MoRef, $answerFileCreateSpec)

Lines 1 -3: Get the host object and create some needed objects to create the answer file. Replace the name on line 1 with the name of your host.

Lines 5-7: Define the path to the info in the answer file that you are going to provide. You will need to replace the vDS and PG with the information from your vDS. See below for info on how to get this information.

Lines 9 -14: Provide the IP information, make sure to update lines 11 and 14 with the correct IP and subnet information.

Lines 16 – 20: finish constructing the details for the answer file, no changes are needed in this section.

Line 22: Creates the answer file based on the provided information.

Obviously you would need to do this for every VMK you need to provide IP information.

You can also use the HostProfileManager object  (stored in $hostProfileManagerView in the example above) to check the host profile.

PS>$hostProfileManagerView.CheckAnswerFileStatus($targetHost.ExtensionData.MoRef)

CheckedTime : 7/30/2012 1:05:37 AM
Host : HostSystem-host-5239
Status : valid
Error :
LinkedView :
DynamicType :
DynamicProperty :

You are looking for the Status of valid. If it’s invalid, then the answer file is not complete.

You can use the Get-VMHostProfileRequiredInput cmdlet to get the profile paths that you need to specify (i.e. what goes into line 6 of the above script) for the answer file configuration.

PS>Get-VMHostProfileRequiredInput -VMHost $VMHost -Profile $HostProfile | fl

Key : network.dvsHostNic["key-vim-profile-host-DvsHostVnicProfile-vDS-PG"].ipConfig.IpAddressPolicy.address
Type : Required
VMHost : esx-2.domain.com
VMHostProfile : HostProfile
Key : network.dvsHostNic["key-vim-profile-host-DvsHostVnicProfile-vDS-PG"].ipConfig.IpAddressPolicy.subnetmask
Type : Required
VMHost : esx-2.domain.com
VMHostProfile : HostProfile

Specifying the Root Password in the Answer File

The last part of the puzzle was setting the root password in the host profile. Since we were importing the host profile the root password is stripped from the host profile and it defaults to leaving the root password unchanged, which would be blank for the AutoDeployed host.

And for the third lucky find, it turns out LucD had already written a function called Set-VMHostProfileExtended that updates the root password stored in a host profile. Running this function against our imported host profile set the profile to update the root password and the password to the desired value.

Final Words

When I started this post I thought it would be nice to share a solution to a problem I was having. I didn’t realize it would mostly be just putting together solutions from three different people. But I guess that is what the Internet is all about and I’m glad they shared. Hopefully this post will save someone else the time it took for me to find and put these different components together.

Older Posts »

The Silver is the New Black Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 36 other followers

%d bloggers like this: