ERROR LCMVROPCONFIG20013 while VCF Operations update from 9.0.0 to 9.01
ERROR LCMVROPCONFIG20013
Installed Tree: apt install tree # tree
File Exsist on the Offline Depot
File Remove from the Binary Management 
Download the upgrade file again
Download the file again: sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VROPS
Pre-staged the VROPS update again
Update successfully
Upgrade VMware VCF 9.0.1: Essential Setting Up an Offline Depot on Ubuntu Instructions
If you are planning to upgrade to latest release of VMware Cloud Foundation (VCF) 9.0.1 and you what to install/upgrade you need to have a offline depot.
I had some struggle with the VCF Fleet upgrades. I thought the VCF installer Offline Depot was sufficient. I seems not.
William Liam did a nice explaining about the options: VCF Software Depot Structure Deep Dive for Install & Upgrade. Which confirms my struggle.
Sow I build my own offline depot
First, download the required bundle files from the Broadcom VMware portal. This includes:
Sizing for the disk is minimal 100GB I used 200GB thin Provisiond
I did a Ubuntu install on a 200GB disk (Partition without lvm)
Login with the user account that you create
Sudo passwd root
Vim /etc/ssh/sshd_config
if the following line exists, possibly commented out (with a # in front):
PermitRootLogin
Then change it to the following, uncommenting if needed (remove the # in front):
PermitRootLogin yes
sudo service ssh restart
now you can login als root
Sudo update
sudo apt install apache2 openssl apache2-utils unzip
nano ~/vcf-openssl.cnf
Paste the following:
[ req ]
default_bits = 2048
prompt = no
default_md = sha256
distinguished_name = dn
x509_extensions = v3_req
[ dn ]
C = US
ST = CA
L = LA
O = TS
OU = IT
CN = flt-depot.wardhomelab.nl
emailAddress = a@b.c
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[ alt_names ]
DNS.1 = flt-depot.wardhomelab.nl
IP.1 = 192.168.150.246
Replace the values with your organization’s details.
Generate the certificate
cd /etc/apache2
sudo mkdir ssl
sudo openssl req -x509 -nodes -days 365 \
-newkey rsa:2048 \
-keyout /etc/apache2/ssl/vcf.key \
-out /etc/apache2/ssl/vcf.crt \
-config ~/vcf-openssl.cnf
Sudo htpasswd -c /etc/apache2/.htpasswd vcfadmin
sudo nano /etc/apache2/sites-available/default-ssl.conf
Paste the following:
<VirtualHost *:443>
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html
SSLEngine on
SSLCertificateFile /etc/apache2/ssl/vcf.crt
SSLCertificateKeyFile /etc/apache2/ssl/vcf.key
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
RequestHeader unset Proxy early
<Directory /var/www/html>
Options Indexes FollowSymLinks
AllowOverride None
AuthType Basic
AuthName “VCF Depot”
AuthUserFile /etc/apache2/.htpasswd
Require valid-user
</Directory>
</VirtualHost>
sudo a2enmod ssl headers
sudo a2ensite default-ssl
sudo systemctl restart apache2
Sudo chmod 777 /home
Upload vcf-9.0.1.0-offline-depot-metadata.zip /home/ with Winscp
Sudo unzip vcf-9.0.1.0-offline-depot-metadata.zip -d /var/www/html
Make sure this file exists:
ls /var/www/html/PROD/metadata/productVersionCatalog/v1/productVersionCatalog.json
sudo rm -f /var/www/html/index.html
Create your Download token in de Broadcom portal
Create a token file in /home folder
Sudo mkdir /home/vcf-download-tool
Sudo chmod 777 /home/vcf-download-tool
cd /tmp/vcf-download-tool
Upload the tool in /tmp/vcf-download-tool folder
tar -xf vcf-download-tool-9.0.1.0.24962179.tar.gz
sudo reboot
Cd /tmp/vcf-download-tool/bin
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –automated-install –type=INSTALL
sudo ./vcf-download-tool binaries list –depot-download-token-file=/home/token –vcf-version=9.0.1 –type=INSTALL –sku=VCF
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component SDDC_MANAGER_VCF
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VCENTER
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VRSLCM
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VROPS
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component NSX_T_MANAGER
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VCF_OPS_CLOUD_PROXY
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VRA
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VRNI
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VRLI
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component HCX
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VRO
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VIDB
Upgrade Binaries will be visible
Open de VM console
Login with the root user
Vim /etc/ssh/sshd_config
if the following line exists, possibly commented out (with a # in front):
PermitRootLogin
Then change it to the following, uncommenting if needed (remove the # in front):
PermitRootLogin yes
systemctl restart sshd
scp root@ftt-depot.wardhomelab.nl:/etc/apache2/ssl/vcf.crt /tmp/vcf.crt
Import the certificate into the Java truststore
sudo keytool -import -trustcacerts -alias vcfDepotCert1 \
-file /tmp/vcf.crt \
-keystore /usr/lib/jvm/openjdk-java17-headless.x86_64/lib/security/cacerts \
-storepass changeit
Reboot
Now u should connect VCF to you offline depot
You want you upgrade to 9.0.1 ivm Edge Issue
Special thanks to vmtechnics for putting me in the right direction
After shutting down the VCF 9.0 environment completely, I had 2 strange things.
I followed the shutdown order for the management domain.
Edge servers lost from NIC 2 and NIC 3 the network configuration
Edge Servers did not start, not correctly
I deployed a third edge server. Two new distributed port groups were created.
These are missing for the edge01a and edge01b servers. These should not be deleted/missing after shutdown.
Create a distributed port group with a VLAN trunk
Set NIC 2 and 3 to this network.
Edge Nodes’ Edge Cluster is running fine again 😊
I found this on the Daniel Kruger Blog: https://sdn-warrior.org/posts/vcf9-ms-a2-special/
This is a known issue with VCF 9.0.0: When an edge is created by the Setup Network Connectivity UI, the system-created dvpg consumed by the edge gets deleted when the edge is powered on after 24 hrs. The port group assigned to the NSX Edge uplink has disappeared, making it impossible to use the network through NSX Edge. You can find the release notes here.
Build my lab nested, but not enough resources to do a full upgrade to 9.0.1.
So I build the lab again, but physically. More about that later!
I love testing things; you learn so much!
VCF 9 and 9.0.1 require 10GB nic.
I use for my homelab the MS-A2 with the 2 x 10GbE Intel X710 SFP+ (NIC 1 & 2 from the left). It is silent and fast and have enough power to run VCF 9!
So I bought also the 10GbE SFP+ to Copper Transceiver
But when you connect the MS-A2 to USW Flex 2.5G switch on the switch side is 2.5G.
But on the VMware side is 10GB 😊
No need to do the following: Disable 10GbE NIC Pre-Check in the VCF 9.0 Installer or
Enhancement in VCF 9.0.1 to bypass vSAN ESA HCL & Host Commission 10GbE NIC Check
It working great! No need to buy expensive 10GB switch!
For the deployment I used a server that has 16 cores en 32 cores total and a lot of ram!
Installation on Nested ESX VM’s
For the nested ESX VMs I am using the generic ESX image. In my lab I’m using 4 nested ESX nodes for vSAN ESA (Minimum is needed is 3).
To config the 4 Hosts for vSAN ESA I used my own powercli script where I blogged about here: Config vSAN ESA host or VCF ESA vSAN Host the easy way with Config-VSAN-ESA-VCF-Lab-Host Script
You can download the script HERE!
Installing the Vib works still great. No problem with ESA Pre-check: ✅
If you have problems check this great blog from William Lam: vSAN ESA Disk & HCL Workaround for VCF 9.0
At first you have to deploy the VMware Cloud Foundation
The name of the installer is a little off: VCF-SDDC-Manager-Appliance-9.0.0.0.24703748.ova
If you don’t have a download token from the Broadcom support then it’s a little complicated. I won’t go into this in depth now but here is a nice article if you have a Synology Nas: VCF 9.0 Offline Depot using Synology
(Note: For any vExpert reading this… You need to claim your own domain name because build your profile is not working with a @hotmail.com or @gmail accounts, Noted by Broadcom)
I looks something like this!
Next thing to do was: Disable 10GbE NIC Pre-Check in the VCF 9.0 Installer or this should also work: How to change the vmxnet3 link speed of a VM (Not tested)
I have deployed the beta in a earlier state. For deployment I used the same json file to start with.
VCF Automation will deployed automatically that was only the change that I changed it
Above the screenshot from the host with de Nested VCF lab deployment it’s quite cpu intensive (2 x CPU 8cores / 16 Threads)
After the deployment I did three things
– On the vSAN ESA Cluster (Disable Auto Disk Claim) (Keep the warning away)
– On the NSX VM reduce the cpu reservation from 6000 🡪 2000 (It helps but not enough)
– VCF automation VM 24 cores to 16 cores.
NSX en VCF automation are really CPU intensive.
I had deployed also two Edge servers but the server did not like that. Edges are also CPU intensive.
I am thinking about adding MS-01 or MS-A2 for splitting some load.
William Lam created the vSAN ESA HCL hardware mock VIB for Nested ESXi.
It works great for vSAN ESA or for VCF vSAN ESA Nested nodes.
You can download the needed vib here
A whale ago I found a easy way to the deployment of a VMware SDDC based on vSAN ESA: VCF automated lab deployment with vSAN ESA
It works great. I a few hours you have a working VCF management domain on vSAN ESA.
Sow what about the script
For the Workload domain domain I created de nested ESXi vm’s manually .
And I did some testing for VCF 9 which I created the nested ESXi host also manually.
Configure by hand takes a lot of time. So I did some scripting and I used some code
from: VCF automated lab deployment with vSAN ESA
So I created a script that does the following:
It works great for ESXi Nested nodes (ESXi 8.0u3b and ESXi9.X beta)
You can find the script HERE!!
On the 4th Oktober VMware Skyline was end of life.
VMware Skyline was great:
• Proactive Issue Identification
• Automated Insights
• Health Scans and Remediation
• Integration with support
VMware by Broadcom are building critical Findings and Self-Help recommendations directly in product starting with VCF (from 5.2) and Aria Operations (from v8.18 July 2024)
Many of the other Skyline features are being planned for inclusion in future releases in Cloud Foundation and Aria Operations. We will see what the future will bring.
But for now how do you get this working.
First Step:
Update Aria Operations to 8.18.2 (Lastest)
Second Steps:
1. vCenter (Don’t for get enable vSAN), NSX, VCF, Aria vRA
2. Configure log collection in Aria Logs for the following components:
• Configure vCenter server integration in Aria for Logs
• Configure log forwarding on vCenter server, ESXi hosts(automatically in Aria FOR logs), and SDDC manager
3. Integrating VMware Aria Operations for Logs and VMware Aria Operations
4. Connect Skyline Health Diagnostics (SHD)
5. In Aria LoginSight check the vRops integration checkboxes
Bij default Enable launch in context can be disabled when configured at first.
After upgrading and checking the settings its finally working 😊 (It can take some time).
So I interested to trying to deploy latest release of VMware Cloud Foundation (VCF) 5.0 on my Windows 11 Home PC witch have 128GB and 16 core intel cpu.
William Lee wrote a nice artikel about VMware Cloud Foundation 5.0 running on Intel NUC
Disclaimer: This is not officially supported by VMware, please use at your own risk.
Requirements:
Step 1 – Boot up the ESXi installer from de iso mount and then perform a standard ESXi installation.
Step 2 – Once ESXi is up and running, you will need to minimally configure networking along with an FQDN (ensure proper DNS resolution), NTP and specify which SSD should be used for the vSAN capacity drive. You can use the DCUI to setup the initial networking but recommend switching to ESXi Shell afterwards and finish the require preparations steps as demonstrated in the following ESXCLI commands:
esxcli system ntp set -e true -s pool.ntp.org
esxcli system hostname set –fqdn vcf-m01-esx01.wardvissers.nl
Note: Use vdq -q command to query for the available disks for use with vSAN and ensure there are no partitions residing on the 600GB disks.
Don’t change time server pool.ntp.org.
To ensure that the self-signed TLS certificate that ESXi generates matches that of the FQDN that you had configured, we will need to regenerate the certificate and restart hostd for the changes to go into effect by running the following commands within ESXi Shell:
/bin/generate-certificates
/etc/init.d/hostd restart
Step 3 – Deploy the VMware Cloud builder in a separate environment and wait for it to be accessible over the browser. Once CB is online, download the setup_vmware_cloud_builder_for_one_node_management_domain.sh setup script and transfer that to the CB system using the admin user account (root is disabled by default).
Step 4 – Switch to the root user and set the script to have the executable permission and run the script as shown below
su –
chmod +x setup_vmware_cloud_builder_for_one_node_management_domain.sh
./setup_vmware_cloud_builder_for_one_node_management_domain.sh
The script will take some time, especially as it converts the NSX OVA->OVF->OVA and if everything was configured successfully, you should see the same output as the screenshot above.
Step 4 – Download the example JSON deployment file vcf50-management-domain-example.json and and adjust the values based on your environment. In addition to changing the hostname/IP Addresses you will also need to replace all the FILL_ME_IN_VCF_*_LICENSE_KEY with valid VCF 5.0 license keys.
Step 5 – The VMnic in the Cloud Builder VM will acked als a 10GB NIC so I started the deployment not through powershell but normal way in Cloud Builder GUI.
Your deployment time will vary based on your physical resources but it should eventually complete with everything show success as shown in the screenshot below. (I have one retry for finish)
Here are some screenshots VCF 5.0 deployment running on my home PC.
Check this if you have problems logging in NSX:
https://www.wardvissers.nl/2023/07/26/nsx-endless-spinning-blue-cirle-after-login/
1. Reploy with use of the Holo-Router https://core.vmware.com/resource/holo-toolkit-20-deploy-router#deploy-holo-router
2. Testing if can deploy Single Host VCF Workload Domain, on same way by following this blog post HERE! 😁
If I can start another 64GB ESXi Server.
The VMware Cloud Foundation team is pleased to announce the availability of VMware Cloud Foundation 5.0
Waiting for VMUG Advantive to be updatet & Finding the time to test it out.
You must be logged in to post a comment.