Infrastructure Adventures

09/24/2012

Update vSphere Host Profile from Reference Host via PowerCLI

Filed under: Virtualization — Tags: , , , — Joe Keegan @ 7:18 PM

I’m dealing with many clusters of stateless hosts deployed via vSphere AutoDeploy and updating host profiles from reference host is something I badly needed. I was surprised this cmdlet didn’t exist so I wrote a function to do the job. Hope folks find this useful.

Function Update-VMHostProfile {
	<# 		
	.SYNOPSIS 			
		Updates the host profile from its reference host.		
	
	.DESCRIPTION 			
		The Update-VMHostProfile updates a host profile from its reference host.
		
	.PARAMETER  Profile
		Specify the host profile in which you wish to update from reference host.
		
	.EXAMPLE 			
		C:\PS> Update-VMHostProfile -Profile $Profile

		Updates the host profile $Profile from its reference host.

	.EXAMPLE 			
		C:\PS> Get-VMHost -Name $Host | Get-VMHostProfile | Update-VMHostProfile

		Get's the host profile attached to the VMHost $Host and updates it from the host profile's reference host.
	
	.EXAMPLE 			
		C:\PS> Get-VMHostProfile | Update-VMHostProfile

		Updates all host profiles from reference host.

	.NOTES
			Function has only been tested with vSphere 5.1, should work with vSphere 5.0.

	.LINK

http://infrastructureadventures.com/

	#>

	[CmdletBinding()]
	Param (
		[Parameter(Mandatory=$True,ValueFromPipeline=$True)]
		$Profile
	)

	BEGIN {}

	PROCESS {
	
		if ( ($Profile.GetType()).name -eq "String" ) {
			$Profile = Get-VMHostProfile -Name $Profile
		}

		$HostProfileReferenceHostId = ("host-" + (($Profile.ReferenceHost.Id).split("-"))[2])
	
		$HostProfileHostBasedConfigSpec = New-Object VMware.Vim.HostProfileHostBasedConfigSpec
		$HostProfileHostBasedConfigSpec.host = New-Object VMware.Vim.ManagedObjectReference
		$HostProfileHostBasedConfigSpec.host.type = "HostSystem"
		$HostProfileHostBasedConfigSpec.host.Value = $HostProfileReferenceHostId
		$HostProfileHostBasedConfigSpec.useHostProfileEngine = $true

		$HostProfileView = Get-View -Id $Profile.Id
		$HostProfileView.UpdateHostProfile($HostProfileHostBasedConfigSpec)
		
		$Profile
	}

	END {}
}

Managing SSD LUNs and vSphere Host Cache Configuration via PowerCLI

Filed under: Storage, Virtualization — Tags: , , , , , , — Joe Keegan @ 6:13 PM

I have been experimenting with SSD and Host Cache in my home lab. I am also trying to do as much with PowerCLI as possible. Coming from a Unix background I think it would be awesome to be able to do basically everything in vSphere (and vCloud) via a CLI. Turns out there are not any cmdlets to enable the SSD options on LUNs or manage Host Cache configurations, so I wrote some functions to do that.

I imagine this could be particularly useful for someone who plans on using Host Cache and has a clusters of hosts they need to configure to use that feature. My plan is to write more “missing” cmdlets and make a module available in the near future.

Set-SSDScsiLUN and Get-SSDScsiLUN

Host Cache requires data store backed by SCSI LUNs to be marked as SSDs. Presumably ESXi will detect and automatically mark actual SSD backed data stores as SSD. But for home labs or instances where ESXi doesn’t correct identify the SCSI LUN as an SSD, the LUN can be marked as SSD manually. This can be done via ESXCLI and it turns out you can call ESXCLI via PowerCLI’s Get-ESXCLI cmdlet.

The Set-SSDScsiLUN function is a PowerCLI wrapper for the Get-ESXCLI cmdlet that will mark the specified SCSI LUNs as SSD backed. This works with the pipeline so you can mark a LUN with a specific Canonical Name as SSD for an entire cluster or data center with a single line.

Since this is a wrapper for ESXCLI and involves calling a process to “re-claim” all the LUNs there is a slight lag between running this command and having the LUNs shows up as SSD.

Function Set-SSDScsiLUN {
	<#
		.SYNOPSIS
			Adds or Removes the "enable_ssd" option for a specified SCSI LUN.

		.DESCRIPTION
			The Set-SSDScsiLUN function uses the ESXCLI to update the SATP claim rule for the specified SCSI LUN. Please note there is a few second delay for the claim rules to run and for the data store to show up as SSD.
			
		.PARAMETER  ScsiLun
			Use this parameter to specify the SCSI LUN you wish to enable/disable the SSD option.

		.PARAMETER  Disabled
			Used to disable/unset the SSD option on the sprcified LUN.
		
		.PARAMETER  ReClaimOnly
			It's possible to end up in a situation where the claim rule has been added or removed, but for whatever reason the claim process does not complete successfully. This can be caused by running this function several rimes in a row. Using this switch will just rerun the claim process. If you run the Set-SSDScsiLUN function and the change does not take effect, you may want to try running with the ReClaimOnly switch.	
		
		.EXAMPLE
			C:\PS> Set-SSDScsiLUN -ScsiLun $SCSILUN
			
			Enables the SSD option for $SCSILUN
			
		.EXAMPLE
			C:\PS> Set-SSDScsiLUN -ScsiLun $SCSILUN -Disabled
			
			Disables the SSD option for $SCSILUN
		
		.Example
			C:\PS> Get-VMHost | Get-ScsiLun -CanonicalName $SCSILUN | Set-SSDScsiLUN
			
			Enables the SSH option for all hosts with a SCSI LUN matching the Canonical Name of $SCSILUN (e.g. mpx.vmhba1:C0:T1:L0)

		.EXAMPLE
			C:\PS> Get-Datastore | where { $_.name -like "*HostCache*" } | Get-ScsiLun | Set-SSDScsiLUN
			
			Enables the SSD option for all LUNs that belong to data stores with "HostCache" as part of thier name.			
	
		.NOTES
			Function has only been tested with vSphere 5.1, should work with vSphere 5.0. This fucntion has only been tested for local disks.		
	
		.LINK

http://infrastructureadventures.com/

	#>
	
	[CmdletBinding()]
	Param (
		[Parameter(Mandatory=$True,ValueFromPipeline=$True)]
		$ScsiLun,
		[Switch]
		$Disabled,
		[Switch]
		$ReClaimOnly
	)

	BEGIN {}

	PROCESS {

		if ( ($ScsiLun.GetType()).name -eq "String" ) {
			$ScsiLun = Get-ScsiLun -CanonicalName $ScsiLun
		}

		$LUN_CName = $ScsiLun.CanonicalName

		$VMHost = get-vmhost -name $ScsiLun.VMHost
		$ESXCLI = $VMHost | get-EsxCli
		$SATP = ($ESXCLI.storage.nmp.device.list($LUN_CName))[0].StorageArrayType
		$ClaimRules = $ESXCLI.storage.nmp.satp.rule.list()

		if ( $ReClaimOnly -eq $False ) {
			if ( $Disabled -eq $True ) {
				if ( $ESXCLI.storage.core.device.list($LUN_CName)[0].IsSSD -eq "true" ) {
					foreach ( $Rule in $ClaimRules ) {
						if ( $Rule.Device -eq $LUN_CName -and $Rule.Options -eq "enable_ssd" ) {
						$ESXCLI.storage.nmp.satp.rule.remove($FALSE,"","",$LUN_CName,"","","enable_ssd","","",$SATP,"","","") | Out-Null
						$ESXCLI.storage.core.claiming.reclaim($LUN_CName)
						}
					}
				}
			} elseif ( $Disabled -eq $False ) {
				if ( $ESXCLI.storage.core.device.list($LUN_CName)[0].IsSSD -eq "false" ) {
					foreach ( $Rule in $ClaimRules ) {
						if ( $Rule.Device -eq $LUN_CName -and $Rule.Options -eq "enable_ssd" ) {
							$RuleExists = $True
							break
						}
					}
					if ( $RuleExists ) {
						$ESXCLI.storage.core.claiming.reclaim($LUN_CName)
					} else {
						$ESXCLI.storage.nmp.satp.rule.add($FALSE,"","",$LUN_CName,"",$FALSE,"","enable_ssd","","",$SATP,"","","") | Out-Null
						$ESXCLI.storage.core.claiming.reclaim($LUN_CName)
					}
				}
			}
		} elseif ( $ReClaimOnly ) {
			$ESXCLI.storage.core.claiming.reclaim($LUN_CName)
		}
	$ScsiLun
	}

	END {}
}

This next function will output the LUNs in an existing data store that have been marked as SSD. I went back and forth on what should be the input and output for this function, and I admit I”m still not entirely satisfied. If anyone has any suggestions, i’d be up for a re-write.

Function Get-SSDScsiLUN {
	<#
		.SYNOPSIS
			Gets SCSI LUN objects that are marked as SSD.

		.DESCRIPTION
			The Get-SSDScsiLUN function outputs the SCSI LUNs that belong to a data store that are marked as SSD based. Please note that it takes a couple of seconds for the changes made by the Set-SSDScsiLUN function to complete.

		.PARAMETER  DataStore
			Use this parameter to specify the data store you wish to see if the SSD option has been set.

		.EXAMPLE
			C:\PS> Get-SSDScsiLUN -DataStore $DataStore
			
			Outputs to the SCSI LUN objects that are marked as SSD for the specified $DataStore.
			
		.NOTES	
			Function has only been tested with vSphere 5.1, should work with vSphere 5.0.	
	
		.LINK

http://infrastructureadventures.com/

	#>
	
	[CmdletBinding()]
	Param (
		[Parameter(ValueFromPipeline=$True)]
		$DataStore
	)

	BEGIN {}

	PROCESS {
		if ( $DataStore ) {
			if ( ($DataStore.GetType()).name -eq "String" ) {
				$DataStore = Get-Datastore -Name $DataStore
			}
		} else {
			$DataStore = Get-DataStore | where { $_.Type -eq "VMFS" }
		}
		$DataStore | where { $_.Type -eq "VMFS" } | Get-ScsiLUN | where { $_.ExtensionData.Ssd -eq "True" }
	}

	END {}
}

Set-VMHostCache and Get-VMHostCache

Now really this is all about setting the Host Cache Space on an SSD backed LUN and the Set-VMHostCache can be used to do just that. Simply provide it a SSD backed data store object and it will configure a Host Cache Space as large as the available free space on the data store, or optionally provide a size in GBs. Specifying zero (0) for the size will turn Host Cache off.

And there is a Get-VMHostCache function which can be used to see the Host Cache Space defined on each data store.

Function Set-VMHostCache {
	<#
		.SYNOPSIS
			Configures Host Cache Space on a specified SSD backed Data Store.

		.DESCRIPTION
			The Set-VMHostCache function configures Host Cache Space on a specified SSD backed Data Store.
			
		.PARAMETER  DataStore
			Use this parameter to specify the Data Store you would like to use for the Host Cache Space.
		
		.PARAMETER  Space
			This parameter specifies the space in GBs that should be used for the Host Cache Space. Omitting this parameter will cause the Host Cache Space to be set to the maximum amount of space available on the data store.

		.PARAMETER  Disabled
			Used to disable/remove the Host Cache Space from the specified Data Store.
		
		.EXAMPLE
			C:\PS> Set-VMHostCache -DataStore $datastore -Size 15
			
			Sets the Host Cache Space on Data Store $datestore to 15 GBs.
			
		.EXAMPLE
			C:\PS> Set-VMHostCache -DataStore $datastore -Disabled
			
			Removes the Host Cache Space from $datastore.

		.EXAMPLE
			C:\PS> Get-VMHost | New-Datastore -VMFS -Path $SSDSCSILUN -Name "HostCache" | Set-VMHostCache
			
			Creates a Data Store on each host with the SCSI LUN $SSDSCSILUN called HostCache and then sets the Host Cache Space to equal the maximum space available on the data store.		
	
		.NOTES
			Function has only been tested with vSphere 5.1, should work with vSphere 5.0.		
	
		.LINK

http://infrastructureadventures.com/

	#>
	
	[CmdletBinding()]
	Param (
		[Parameter(Mandatory=$True,ValueFromPipeline=$True)]
		$DataStore,
		[int]
		$Space,
		[Switch]
		$Disabled
	)

	BEGIN {}

	PROCESS {

		if ( ($DataStore.GetType()).name -eq "String" ) {
			$DataStore = Get-Datastore -Name $DataStore
		}

		$HostCacheConfigurationSpec = New-Object VMware.Vim.HostCacheConfigurationSpec
		$HostCacheConfigurationSpec.datastore = New-Object VMware.Vim.ManagedObjectReference
		$HostCacheConfigurationSpec.datastore.type = "Datastore"
		$HostCacheConfigurationSpec.datastore.Value = ($DataStore.id).substring(10,13)

		if ( $Disabled ) {
			$HostCacheConfigurationSpec.swapSize = 0
		} else {
			if ( $Space ) {
				$SpaceMB = $Space * 1024
				if ( $SpaceMB -gt $DataStore.FreeSpaceMB ) {
					Write-Warning ("Specified Host Cache Space size is larger then avialable space, setting Host Cache Space size to maximum avialable space.")
					$HostCacheConfigurationSpec.swapSize = $DataStore.FreeSpaceMB
				} else {
					$HostCacheConfigurationSpec.swapSize = $SpaceMB
				}
			} else {
				$HostCacheConfigurationSpec.swapSize = $DataStore.FreeSpaceMB
			}
		}

		$HostCacheConfigurationManager_ID = ("HostCacheConfigurationManager-cacheConfigManager-" + (($DataStore.ExtensionData.host[0].Key.value).split("-"))[1])
		$HostCacheConfigurationManager = Get-View -Id $HostCacheConfigurationManager_ID
		$HostCacheConfigurationManager.ConfigureHostCache_Task($HostCacheConfigurationSpec) | Out-Null

		$DataStore

	}

	END {}
}

 


Function Get-VMHostCache {
	<#
		.SYNOPSIS
			Gets the Host Cache configuration for the specified host.

		.DESCRIPTION
			The Get-VMHostCache function gets the Host Cache configuration for a specified host. It displays the amount of space on each data store that has been configured as Host Cache Space.
			
		.PARAMETER  VMHost
			Use this parameter to specify the Host that you wish to get the host cache configuration.
		
		.EXAMPLE
			C:\PS> Get-VMHostCache -VMHost $Host
			
			Gets the Host Cahce configuration for $Host
			
		.NOTES
			Function has only been tested with vSphere 5.1, should work with vSphere 5.0.		
	
		.LINK

http://infrastructureadventures.com/

	#>
	
	[CmdletBinding()]
	Param (
		[Parameter(Mandatory=$True,ValueFromPipeline=$True)]
		$VMHost
	)

	BEGIN {}

	PROCESS {

		if ( ($VMHost.GetType()).name -eq "String" ) {
			$VMHost = Get-VMHost -Name $VMHost
		}

		$HostCacheConfigurationManager_ID = ("HostCacheConfigurationManager-cacheConfigManager-" + (($VMHost.id).split("-"))[2])
		$HostCacheConfigurationManager = Get-View -Id $HostCacheConfigurationManager_ID

		foreach ( $HostCacheDataStore in $HostCacheConfigurationManager.CacheConfigurationInfo ) {
			$DataStore_ID = ("Datastore-" + $HostCacheDataStore.key.value)
			$Datastore = Get-DataStore -Id $DataStore_ID

			$DataStore | select name, @{Name="HostCacheSpaceGB";Expression={$HostCacheDataStore.SwapSize / 1024}}, FreeSpaceGB, CapacityGB
		}
	}

	END {}
}

08/11/2012

vCloud Director Policies – Part 2: Quotas & Limits

Filed under: Cloud — Tags: , , , , , , — Joe Keegan @ 5:28 PM

I covered leases in my previous post on vCloud Director Policies and will be covering quotas and limits in this post.

Quotas

Quotas are used to limit the number of VMs a user can have in the org. Quotas come in two flavors, the stored VM quota, which limits the number of VMs a user can have in the org and the running VM quota which limits the number of VMs that a user can have running in the org. While it’s possible to set the running VM quota to higher then the stored VM quota (at least in 1.5.1), every VM in the org counts against the stored VM quota. Meaning the stored VM quota is the maximum number of VMs a user can have in the org. I.e. A user with a stored VM quota of 10 and a running VM quota of 15, would only be able to have 10 VMs in the org.

The default storage VM and running VM quotas are configured at the org level, but this just sets the the quotes for new users added to the org. So for example if you set the quotas for an org to 20 stored VMs and 10 running VMs, then each user added to that org will have a quota of 20 storage and 10 running. If you have 10 users in the org, then the org could have up to 200 stored VMs and 100 running VMs.

If you change the quota at the org level, say you lowered the number of running VMs to 5, then only new uses will have that quota, all existing users will still have a running VM quota of 10. To further make the point, a user’s quota does not have to match the org defaults and can be set to what ever the org admins feel is appropriate for that user.

This makes the application of quotas much different then leases. Leases are set at the org level and apply to all users, well leases really apply to vApps, but vApps are created by users. Where quotas are set on a user by user basis and two users could have very different quota settings.

Quotas in general don’t seem all that useful, except for maybe in a Pay As You Go model. But even then, since quotas are an org level configuration, instead of a org vDC level configuration, you could quickly complicate things by having an org with vDCs using both PAYGO and Allocation models, then what?

Limits

Limits are set at an org level to help prevent prevent a DoS attack and generally ensure a single user can’t tax the cloud too much. Three different limits can be set, with two of them limiting the number of resource intensive operations and one of them limiting the number of VMRC console connections to a VM.

The limits to the number of resource intensive operations are set as a per user number and a per org number. So you could set the limit for the number of resource intensive operations to 3 per user and 6 per org. In this configuration two users could run 3 resource intensive operations without issue, but if another user tried to start a resource intensive operation they would receive an error stating the maximum number of simultaneous operations has been reached.

The next logical questions is what counts as a resource intensive operation and I can’t find a definitive list. Examples such as copy, move, Add to My Cloud and Add to Catalog are given in the UI. So it sounds like mostly provisioning type activities.

The last limit is the number of simultaneous connections per VM. And by connections it means VMRC connections. Since VMRC connections are proxied by the vCD cell, limiting the number of connections can help ensure your cells don’t get overloaded. This is a somewhat poor protection from someone trying to DoS your cloud by opening a ton of VMRC connections. Even if the number of connections was limited to 1, an attacker could spawn a ton of VMs and open a VMRC connection to each one.  So only with reasonable quotas (what can be set/changed by an Org Admin) can the limit to the number of connections provide any type of protection.

One last point is that limits are set by the system administrator and can not be modified by an Org Admin.

Storage Limits

Storage Limits are a bit different then the Limits above since they apply to an Org VDC instead of an org as a whole. A storage limit does what it sounds like, it limits the amount of storage that can by used by an Org VDC.

The primary consumer of Org VDC storage space are vApps, but vApp templates & media also consume space and are counted towards the limit.

Storage limits, like the Org limits described above, are set by a system administrator and can not be modified by an Org Admin.

 

 

 

 

Older Posts »

The Silver is the New Black Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 37 other followers

%d bloggers like this: