Holodeck Toolkit Overview

Holodeck Toolkit 1.3 Overview

The VMware Cloud Foundation (VCF) Holodeck Toolkit is designed to provide a scalable, repeatable way to deploy nested Cloud Foundation hands-on environments directly on VMware ESXi hosts. These environments are ideal for multi-team hands on exercises exploring the capabilities of utilitizing VCF to deliver a Customer Managed VMware Cloud.

Graphical user interface, application

Description automatically generated

Delivering labs in a nested environment solves several challenges with delivering hands-on for a  product like VCF, including:  

  • Reduced hardware requirements: When operating in a physical environment, VCF requires four vSAN Ready Nodes for the management domain, and additional hosts for adding clusters or workload domains. In a nested environment, the same four to eight hosts are easily virtualized to run on a single ESXi host.   
  • Self-contained services: The Holodeck Toolkit configuration provides common infrastructure services, such as NTP, DNS, AD, Certificate Services and DHCP within the environment, removing the need to rely on datacenter provided services during testing.  Each environment needs a single external IP.
  • Isolated networking. The Holodeck Toolkit configuration removes the need for VLAN and BGP connections in the customer network early in the testing phase.  
  • Isolation between environments. Each Holodeck deployment is completely self-contained. This avoids conflicts with existing network configurations and allows for the deployment of multiple nested environments on same hardware or datacenter with no concerns for overlap. 
  • Multiple VCF deployments on a single VMware ESXi host with sufficient capacity. A typical VCF Standard Architecture deployment of four node management domain and four node VI workload domain, plus add on such as VMware vRealize Automation requires approximately 20 CPU cores, 512GB memory and 2.5TB disk.  
  • Automation and repeatability. The deployment of nested VCF environments is almost completely hands-off, and easily repeatable using configuration files.  A typical deployment takes less than 3 hours, with less than 15 min keyboard time.

Nested Environment Overview 

The “VLC Holodeck Standard Main 1.3” configuration is a nested VMware Cloud Foundation configuration used as the baseline for several Private Cloud operation and consumption lab exercises created by the Cloud Foundation Technical Marketing team. The Holodeck standard “VLC-Holo-Site-1” is the primary configuration deployed. The optional VLC-Holo-Site-2 can be deployed at any time later within a Pod.  VLC-Holo-Site-1 configuration matches the lab configuration in the VCF Hands-On Lab HOL-2246 and the nested configuration in the VCF Experience program run on the VMware Lab Platform. 

Each Pod on a Holodeck deployment runs an identical nested configuration. A pod can be deployed with a standalone VLC-Holo-Site-1 configuration, or with both VLC-Holo-Site-1 and VLC-Holo-Site-2 configurations active. Separation of the pods and between sites within a pod is handled at the VMware vSphere Standard Switch (VSS) level.  Each Holodeck pod connects to a unique VSS and Port Group per site.    A VMware vSphere Port Group is configured on each VSS and configured as a VLAN trunk.  

  • Components on the port group to use VLAN tagging to isolate communications between nested VLANs. This removes the need to have physical VLANs plumbed to the ESXi host to support nested labs.  
  • When the Holo-Site-2 configuration is deployed it uses a second VSS and Port Group for isolation from Holo-Site-1  

The VLC Holodeck configuration customizes the VCF Cloud Builder Virtual Machine to provide several support services within the pod to remove the requirement for specific customer side services. A Cloud Builder VM is deployed per Site to provide the following within the pod: 

  • DNS (local to Site1 and Site2 within the pod, acts as forwarder) 
  • NTP (local to Site1 and Site2 within the pod) 
  • DHCP (local to Site1 and Site2 within the pod) 
  • L3 TOR for vMotion, vSAN, Management, Host TEP and Edge TEP networks within each site 
  • BGP peer from VLC Tier 0 NSX Application Virtual Network (AVN) Edge (Provides connectivity into NSX overlay networks from the lab console)

The figure below shows a logical view of the VLC-Holo-Site-1 configuration within a Holodeck Pod. The Site-1 configuration uses DNS domain vcf.sddc.lab.

 Figure 1: Holodeck Nested Diagram

The Holodeck package also provides a preconfigured Photon OS VM, called “Holo-Router”, that functions as a virtualized router for the base environment. This VM allows for connecting the nested environment to the external world. The Holo-Router is configured to forward any Microsoft Remote Desktop (RDP) traffic to the nested jump host, known as the Holo-Console, which is deployed within the pod.

The user interface to the nested VCF environment is via a Windows Server 2019 “Holo-Console” virtual machine. Holo-Console provides a place to manage the internal nested environment like a system administrators desktop in a datacenter. Holo-Console is used to run the VLC package to deploy the nested VCF instance inside the pod. Holo-Console VM’s are deployed from a custom-built ISO that configures the following 

  • Microsoft Windows Server 2019 Desktop Experience with: 
  • Active directory domain “vcf.holo.lab” 
  • DNS Forwarder to Cloud Builder  
  • Certificate Server, Web Enrollment and VMware certificate template 
  • RDP enabled 
  • IP, Subnet, Gateway, DNS and VLAN configured for deployment as Holo-Console  
  • Firewall and IE Enhanced security disabled  
  • SDDC Commander custom desktop deployed 
  • Additional software packages deployed and configured 
  • Google Chrome with Holodeck bookmarks 
  • VMware Tools 
  • VMware PowerCLI 
  • VMware PowerVCF 
  • VMware Power Validated Solutions 
  • PuTTY SSH client 
  • VMware OVFtool 
  • Additional software packages copied to Holo-Console for later use 
  • VMware Cloud Foundation 4.5 Cloud Builder OVA to C:\CloudBuilder 
  • VCF Lab Constructor 4.5.1 with dual site Holodeck configuration
    • VLC-Holo-Site-1 
    • VLC-Holo-Site-2 
  • VMware vRealize Automation 8.10 Easy Installer

The figure below shows the virtual machines running on the physical ESXi host to deliver a Holodeck Pod called “Holo-A”. Notice an instance of Holo-Console, Holo-Router, Cloud Builder and four nested ESXi hosts.  They all communicate over the VLC-A-PG Port Group   

Figure 2: Holodeck Nested Hosts

Adding a second site adds an additional instance of Cloud Builder and additional nested ESXi hosts. VLC-Holo-Site-2 connects to the second internal leg of the Holo-Router on VLAN 20. Network access from the Holo-Console to VLC-Holo-Site-2 is via Holo-Router.

The figure below shows a logical view of the VLC-Holo-Site-2 configuration within a Holodeck Pod. The Site-2 configuration uses DNS domain vcf2.sddc.lab

 Figure 3: Holodeck Site-2 Diagram

Accessing the Holodeck Environment

User access to the Holodeck pod is via the Holo-Console.  Access to Holo-Console is available via two paths:

VLC Holodeck Deployment Prerequisites 

  • ESXi Host Sizing   
  • Good (One pod): Single ESXi host with 16 cores, 384gb memory and 2TB SSD/NVME 
  • Better (Two pod): Single ESXi host with 32 cores, 768gb memory and 4TB SSD/NVME 
  • Best (Four or more pods):  Single ESXi host with 64+ cores, 2.0TB memory and 10TB SSD/NVME 
  • ESXi Host Configuration: 
  • vSphere 7.0U3 
  • Virtual switch and port group configured with uplinks to customer network/internet  
  • Supports stand alone, non vCenter Server managed host and single host cluster managed by a vCenter server instance 
  • Multi host clusters are NOT supported
  • Holo-Build host 
  • Windows 2019 host or VM with local access to ESXI hosts used for Holodeck + internet access to download software. (This package has been tested on Microsoft Windows Server 2019 only) 
  • 200GB free disk space 
  • Valid login to https://customerconnect.vmware.com  
  • Entitlement to VCF 4.5 Enterprise for 8 hosts minimum (16 hosts if planning to test Cloud Foundation Multi region with NSX Federation) 
  • License keys for the following VCF 4.5 components
    • VMware Cloud Foundation
    • VMware NSX-T Data Center Enterprise
    • VMware vSAN Enterprise 
    • VMware vSphere Enterprise Plus 
    • VMware vCenter Server (one license)
    • VMware vRealize Suite Advanced or Enterprise
    • Note: This product has been renamed VMware Aria Suite
  • External/Customer networks required
    • ESXi host management IP (one per host) 
    • Holo-Router address per pod

NSX-T 3.1 Home lab – Part 2

With all the Fabric configuration done we can test our setup.

I’m creating two overlay segments in NSX connected to a Tier-1 gateway, and after that we’ll create a Tier-0 gateway and connect the T1 gateway to it to get North/South connectivity to the overlay resources

Two VMs will be deployed, one VM in each of the two overlay segments

Create a Tier-1 Gateway

The Tier-1 Gateway will initially not be connected to a Tier-0 Gateway (I haven’t configured a T0 gw yet) or an Edge Cluster.

Tier-1 Gateway

Tier-1 Gateway

Create Logical segments

We need two logical segments, both using the Overlay Transport Zone. I’m defining different subnets on them, 10.0.1.0/24 and 10.0.2.0/24.

Afbeelding met tekst, monitor, schermafbeelding, televisie

Automatisch gegenereerde beschrijving

Segments

Add VMs to Logical segments

We have two Photon VMs which should be added to the logical segments.

Two Photon VMs

Test connectivity

Afbeelding met tekst

Automatisch gegenereerde beschrijving Afbeelding met tekst

Automatisch gegenereerde beschrijving

Now let’s verify that the two VMs can ping each other

Afbeelding met tekst

Automatisch gegenereerde beschrijving

Don’t forget to enable the echo rule on the Windows Firewall….

Connectivity test

This shows that the overlay is working, and note again that the Edge VMs are not in use here.

External connectivity

Traffic is flowing between VMs running on Logical segments inside the NSX-T environment, but what if we want to reach something outside, or reach a VM inside a NSX-T overlay?

Then we need to bring a Tier-0 Gateway in to the mix.

The T-0 gateway can be configured with Uplinks that are connected to the physical network. This is done through a segment which can reach the physical network, normally through a VLAN.

To configure the uplink interfaces we need to have Edge VMs so finally we get to bring those into play as well.

Create segment for uplinks

First I’ll create a segment mapped to VLAN 99 in my lab. Note that I select the VLAN transport zone, and I do not connect the segment to a gateway

Create Uplink VLAN segment

Create Uplink VLAN segment

Create Tier-0 gateway

Now we’ll create a Tier-0 gateway, note that I now also select my Edge cluster.

Afbeelding met tekst, schermafbeelding, monitor

Automatisch gegenereerde beschrijving

Create T0 gateway

Static route

To be able to forward traffic out of the NSX-T environment the T0 gateway needs to know where to send queries for IPs it doesn’t control. Normally you would want to configure a routing protocol like BGP or OSPF so that the T0 gateway could exchange routes with the physical router(s) in your network.

I’ve not set up BGP or any other routing protocol on my physical router, so I’ve just configured a default static route that forwards to my physical router. The next hop is set to the gateway address for the Uplink VLAN 99, 192.168.99.1

Static route

Static route

Link T1 gateway to T0 gateway

We’ve done a lot of configuring now, but still we’ve not got connectivity in or out for our VMs. The final step is to connect the Tier-1 gateway to the Tier-0 gateway, and we’ll also activate Route Advertisement of Connected Segments and Service Ports

Afbeelding met tekst, schermafbeelding, monitor, zwart

Automatisch gegenereerde beschrijving

Tier-1 Gateway

Test connectivity

Afbeelding met tekst

Automatisch gegenereerde beschrijving

Verify North/South connectivity

Yes!

Test Distributed Firewall

Let’s also do a quick test of the Distributed Firewall feature in NSX-T.

First we’ll create a rule blocking ICMP (ping) from any to my test vm and publish the rule

Afbeelding met tekst, schermafbeelding, monitor, binnen

Automatisch gegenereerde beschrijving ICMP firewall rule

ICMP firewall rule

Now let’s test pinging from from my pc to nested Windows 2016 server. With the rule not enabled en enabled.

Afbeelding met tekst

Automatisch gegenereerde beschrijving

Ping blocked

Summary

Hopefully this post can help someone, if not it has at least helped me.

Now we have working environment so we can go testing some things.
Also scripting/automation against a nsx environment I will look in to!

VMware vSphere PowerCLI 11.0

VMware vSphere PowerCLI 11.0 New Features

New features available on  VMware vSphere PowerCLI 11.0 is to support the new all updates and release of VMware products , find the below following has been features,

  • New Security module
  • vSphere 6.7 Update 1
  • NSX-T 2.3
  • Horizon View 7.6
  • vCloud Director 9.5
  • Host Profiles – new cmdlets for interacting with
  • New Storage Module updates
  • NSX-T in VMware Cloud on AWS
  • Cloud module multiplatform support
  • Get-ErrorReport cmdlet has been updated
  • PCloud module has been removed
  • HA module has been removed

Now we will go through above mentioned new features to find what functionality it bring to PowerCLI 11.0

What is PowerCLI 11.0 New Security Module

The new security module brings more powerful automation features to PowerCLI 11.0 available  new cmdlets include the following

  • Get-SecurityInfo
  • Get-VTpm
  • Get-VTpmCertificate
  • Get-VTpmCSR
  • New-VTpm
  • Remove-VTpm
  • Set-VTpm
  • Unlock-VM

Also New-VM cmdlet has enhanced functionality with the security module functionality and it includes parameters like KmsCluster, StoragePolicy, SkipHardDisks etc which can be used while creating new virtual machines with PowerCLI .In addition to that  Set-VM, Set-VMHost, Set-HardDisk, and New-HardDisk cmdlets are added.

Host Profile Additions

There are few additions to the VMware.VimAutomation.Core module that will make managing host profiles from PowerCLI

  • Get-VMHostProfileUserConfiguration
  • Set-VMHostProfileUserConfiguration
  • Get-VMHostProfileStorageDeviceConfiguration
  • Set-VMHostProfileStorageDeviceConfiguration
  • Get-VMHostProfileImageCacheConfiguration
  • Set-VMHostProfileImageCacheConfiguration
  • Get-VMHostProfileVmPortGroupConfiguration
  • Set-VMHostProfileVmPortGroupConfiguration

Storage Module Updates

These new Storage Module updates specifically for VMware vSAN , the updates has predefined time ranges when using Get-VsanStat. In addition  Get-VsanDisk has additional new properites that are returned including capacity, used percentage, and reserved percentage. Following are the  cmdlets have been added to automate vSAN

  • Get-VsanObject
  • Get-VsanComponent
  • Get-VsanEvacuationPlan – provides information regarding bringing a host into maintenance mode and the impact of the operation on the data, movement, etc

Additionally  following modules have been removed

  • PCloud module
  • HA module

Download now and start using

Update-module VMware.Powercli

Useful Links

Deploy Multi VM’s based on Windows Template

I love powershell. I created a little script to deploy multi VM based on a Windows Template throug CSV file.

It’s create a computer account at the specfified ou. He greates also a Domain Local Group for management. (It used in the customization not specified here)

TempVMlist.csv

server,cpu,memory,DestinationCluster,OSCustomizationSpec,VMtemplate,adgroup

WARDTEST01,2,8,CLUSTER01,W2012R2_Demo,TPL_W2012R2_STD,ServerAdmin

MultiVM.ps1

#Filename: MultiVM.ps1

#Author: W. Vissers

#Source:

#Version: 1.1

#Date: 08-05-2018

#ChangeLog:

# V1.0 – Module Active Directory

#      – Module VMware PowerCli

#      – Active Directory Computer Account, Group

#      – Host Selected from Cluster with Least Memory

#      – Storage selection based on volume with most free space

# V1.1 – Added Harddisk 1&2

#      – Changed porte group other vlan

#

<#

.SYNOPSIS

Script to create a virtual machine from template

.DESCRIPTION

Script to create a virtual machine from template

.EXAMPLE

MultiVM.ps1

#>

################################## INIT #################################################

# LoadModule Active Directory

if (!(Get-Module “activedirectory”)) {Import-module activedirectory}

Else {Write-Host “Module Active Directory is al ready loaded”}

# LoadModule VMware PowerCLI

# if (!(Get-Module “VMware.PowerCLI”)) {

#    Find-Module VMware.PowerCLI

#    Install-Module -Name VMware.PowerCLI -Scope CurrentUser

#}

#Else

# {

# Write-Host “Module PowerCLI is al ready loaded”

# }

#Config

$ouservers=”OU=Servers,DC=wardvissers.nl,DC=nl”

$ougroup=”OU=GroepObjecten,DC=wardvissers,DC=nl”

$folder=”Applicatie Servers”

$DestinationVC =”vcenter01.wardvissers.nl

#Username

if (!$username ) { $username = Read-Host “Give vCenter username ‘wardvissers\admin'”}

#Password

if ( -NOT $Password ) {

$PasswordSec = Read-Host “Give vCenter password” -AsSecureString

$Password = [Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($PasswordSec))

}

#Connect vCenter

$ConnectVC = Connect-VIServer $DestinationVC -Username $Username -Password $Password -AllLinked

$AllVMs = @()

$AllVMs = Import-Csv “D:\TempVMlist.csv”

foreach ($vm in $AllVMs) {

#Haal De Gegevens op

$server=$($vm.server)

$memory=$($vm.memory)

$cpu=$($vm.cpu)

$DestinationCluster=$($vm.DestinationCluster)

$OSSpec=”$($vm.OSCustomizationSpec)”

$VMtemplate=$($vm.VMtemplate)

$group=$($vm.adgroup)

$harddisk1=$($vm.harddisk1)

$harddisk2=$($vm.harddisk2)

Write-Host “$server heeft $memory GB memory en $cpu cpu(‘s)”

if ($server.length -gt 15) {

Write-Output “Hostname cannot contain more than 15 characters.”

$server = Read-Host “Re-enter hostname for host $server”}

Else

{

Write-Host “Server is umc server”

#Maak AD Groep aan en Computer Account

New-ADComputer -Name $server -Path $ouservers -Enabled $true

New-ADGroup -Name “DLG.$server” -SamAccountName “DLG.$server” -GroupCategory Security -GroupScope DomainLocal -DisplayName “DLG.$server” -Path $ougroup

Add-ADGroupMember -Identity “DLG.$server” -Members $group

}

# Rol server uit van Template

# Select the host with the less used memory

$DestinationHost = Get-Cluster –Name $DestinationCluster –Server $DestinationVC | Get-VMhost -State Connected | Sort-Object -Property MemoryUsageGB | Select-Object -First1

# Select DataStore with the most free space and not in maintance

$destinationDatastore = Get-Cluster $DestinationCluster | Get-Datastore | Where {$_.State -ne “Maintenance”} | Sort-Object -Property FreeSpaceGB -Descending | Select-Object -First 1

# Finally, I deploy my VM with the New-VM cmdlet using my template and OS specs. I place the VM on the ESXi host and store the VM on the datastore.

New-VM -Name $server -Template $VMTemplate -OSCustomizationSpec $OSSpec -VMHost $DestinationHOST -Datastore $DestinationDatastore -Location $folder

Get-VM $server | Set-VM -NumCpu $cpu -MemoryGB $memory -Confirm:$false

if ($harddisk1 -gt 60){Get-HardDisk -vm $server | Where {$_.Name -eq “Hard disk 1”} | Set-HardDisk -CapacityGB $harddisk1 -Confirm:$false}

if ($harddisk2 -gt 20) {Get-HardDisk -vm $server | Where {$_.Name -eq “Hard disk 2”} | Set-HardDisk -CapacityGB $harddisk2 -Confirm:$false}

Get-VM $server | Start-VM -Confirm:$false

Get-VM $Server | Get-NetworkAdapter | Set-NetworkAdapter -Connected $true -Confirm:$false

}

Translate »