ERROR LCMVROPCONFIG20013 while VCF Operations update from 9.0.0 to 9.01
ERROR LCMVROPCONFIG20013
Installed Tree: apt install tree # tree
File Exsist on the Offline Depot
File Remove from the Binary Management 
Download the upgrade file again
Download the file again: sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VROPS
Pre-staged the VROPS update again
Update successfully
Upgrade VMware VCF 9.0.1: Essential Setting Up an Offline Depot on Ubuntu Instructions
If you are planning to upgrade to latest release of VMware Cloud Foundation (VCF) 9.0.1 and you what to install/upgrade you need to have a offline depot.
I had some struggle with the VCF Fleet upgrades. I thought the VCF installer Offline Depot was sufficient. I seems not.
William Liam did a nice explaining about the options: VCF Software Depot Structure Deep Dive for Install & Upgrade. Which confirms my struggle.
Sow I build my own offline depot
First, download the required bundle files from the Broadcom VMware portal. This includes:
Sizing for the disk is minimal 100GB I used 200GB thin Provisiond
I did a Ubuntu install on a 200GB disk (Partition without lvm)
Login with the user account that you create
Sudo passwd root
Vim /etc/ssh/sshd_config
if the following line exists, possibly commented out (with a # in front):
PermitRootLogin
Then change it to the following, uncommenting if needed (remove the # in front):
PermitRootLogin yes
sudo service ssh restart
now you can login als root
Sudo update
sudo apt install apache2 openssl apache2-utils unzip
nano ~/vcf-openssl.cnf
Paste the following:
[ req ]
default_bits = 2048
prompt = no
default_md = sha256
distinguished_name = dn
x509_extensions = v3_req
[ dn ]
C = US
ST = CA
L = LA
O = TS
OU = IT
CN = flt-depot.wardhomelab.nl
emailAddress = a@b.c
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[ alt_names ]
DNS.1 = flt-depot.wardhomelab.nl
IP.1 = 192.168.150.246
Replace the values with your organization’s details.
Generate the certificate
cd /etc/apache2
sudo mkdir ssl
sudo openssl req -x509 -nodes -days 365 \
-newkey rsa:2048 \
-keyout /etc/apache2/ssl/vcf.key \
-out /etc/apache2/ssl/vcf.crt \
-config ~/vcf-openssl.cnf
Sudo htpasswd -c /etc/apache2/.htpasswd vcfadmin
sudo nano /etc/apache2/sites-available/default-ssl.conf
Paste the following:
<VirtualHost *:443>
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html
SSLEngine on
SSLCertificateFile /etc/apache2/ssl/vcf.crt
SSLCertificateKeyFile /etc/apache2/ssl/vcf.key
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
RequestHeader unset Proxy early
<Directory /var/www/html>
Options Indexes FollowSymLinks
AllowOverride None
AuthType Basic
AuthName “VCF Depot”
AuthUserFile /etc/apache2/.htpasswd
Require valid-user
</Directory>
</VirtualHost>
sudo a2enmod ssl headers
sudo a2ensite default-ssl
sudo systemctl restart apache2
Sudo chmod 777 /home
Upload vcf-9.0.1.0-offline-depot-metadata.zip /home/ with Winscp
Sudo unzip vcf-9.0.1.0-offline-depot-metadata.zip -d /var/www/html
Make sure this file exists:
ls /var/www/html/PROD/metadata/productVersionCatalog/v1/productVersionCatalog.json
sudo rm -f /var/www/html/index.html
Create your Download token in de Broadcom portal
Create a token file in /home folder
Sudo mkdir /home/vcf-download-tool
Sudo chmod 777 /home/vcf-download-tool
cd /tmp/vcf-download-tool
Upload the tool in /tmp/vcf-download-tool folder
tar -xf vcf-download-tool-9.0.1.0.24962179.tar.gz
sudo reboot
Cd /tmp/vcf-download-tool/bin
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –automated-install –type=INSTALL
sudo ./vcf-download-tool binaries list –depot-download-token-file=/home/token –vcf-version=9.0.1 –type=INSTALL –sku=VCF
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component SDDC_MANAGER_VCF
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VCENTER
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VRSLCM
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VROPS
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component NSX_T_MANAGER
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VCF_OPS_CLOUD_PROXY
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VRA
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VRNI
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VRLI
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component HCX
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VRO
sudo ./vcf-download-tool binaries download –depot-download-token-file=/home/token -d /var/www/html –vcf-version=9.0.1 –type UPGRADE –component VIDB
Upgrade Binaries will be visible
Open de VM console
Login with the root user
Vim /etc/ssh/sshd_config
if the following line exists, possibly commented out (with a # in front):
PermitRootLogin
Then change it to the following, uncommenting if needed (remove the # in front):
PermitRootLogin yes
systemctl restart sshd
scp root@ftt-depot.wardhomelab.nl:/etc/apache2/ssl/vcf.crt /tmp/vcf.crt
Import the certificate into the Java truststore
sudo keytool -import -trustcacerts -alias vcfDepotCert1 \
-file /tmp/vcf.crt \
-keystore /usr/lib/jvm/openjdk-java17-headless.x86_64/lib/security/cacerts \
-storepass changeit
Reboot
Now u should connect VCF to you offline depot
You want you upgrade to 9.0.1 ivm Edge Issue
Special thanks to vmtechnics for putting me in the right direction
After shutting down the VCF 9.0 environment completely, I had 2 strange things.
I followed the shutdown order for the management domain.
Edge servers lost from NIC 2 and NIC 3 the network configuration
Edge Servers did not start, not correctly
I deployed a third edge server. Two new distributed port groups were created.
These are missing for the edge01a and edge01b servers. These should not be deleted/missing after shutdown.
Create a distributed port group with a VLAN trunk
Set NIC 2 and 3 to this network.
Edge Nodes’ Edge Cluster is running fine again 😊
I found this on the Daniel Kruger Blog: https://sdn-warrior.org/posts/vcf9-ms-a2-special/
This is a known issue with VCF 9.0.0: When an edge is created by the Setup Network Connectivity UI, the system-created dvpg consumed by the edge gets deleted when the edge is powered on after 24 hrs. The port group assigned to the NSX Edge uplink has disappeared, making it impossible to use the network through NSX Edge. You can find the release notes here.
Build my lab nested, but not enough resources to do a full upgrade to 9.0.1.
So I build the lab again, but physically. More about that later!
I love testing things; you learn so much!
VCF 9 and 9.0.1 require 10GB nic.
I use for my homelab the MS-A2 with the 2 x 10GbE Intel X710 SFP+ (NIC 1 & 2 from the left). It is silent and fast and have enough power to run VCF 9!
So I bought also the 10GbE SFP+ to Copper Transceiver
But when you connect the MS-A2 to USW Flex 2.5G switch on the switch side is 2.5G.
But on the VMware side is 10GB 😊
No need to do the following: Disable 10GbE NIC Pre-Check in the VCF 9.0 Installer or
Enhancement in VCF 9.0.1 to bypass vSAN ESA HCL & Host Commission 10GbE NIC Check
It working great! No need to buy expensive 10GB switch!
Sometimes I write stuff for myself and publish it as a archive piece, so as this one.
For 8.0U3
esxcli software sources profile list -d https://dl.broadcom.com/<Your_Broadcom_Download_Token>/PROD/COMP/ESX_HOST/main/vmw-depot-index.xml | grep -i ESXi-8.0U3
For ESX(i) 9
esxcli software sources profile list -d https://dl.broadcom.com/<Your_Broadcom_Download_Token>/PROD/COMP/ESX_HOST/main/vmw-depot-index.xml | grep -i ESXi-9
If you got a memory error run the following commands 
If you do not get a list of versions run the following commands :
esxcli system settings advanced set -o /VisorFS/VisorFSPristineTardisk -i 0
cp /usr/lib/vmware/esxcli-software /usr/lib/vmware/esxcli-software.bak
sed -i ‘s/mem=300/mem=500/g’ /usr/lib/vmware/esxcli-software.bak
mv /usr/lib/vmware/esxcli-software.bak /usr/lib/vmware/esxcli-software -f
esxcli system settings advanced set -o /VisorFS/VisorFSPristineTardisk -i 1
Install latest Update (ESXi-8.0U3g-24859861-standard)
esxcli software profile update -p ESXi-8.0U3g-24859861-standard -d https://dl.broadcom.com/<Your_Broadcom_Download_Token>/PROD/COMP/ESX_HOST/main/vmw-depot-index.xml
For the deployment I used a server that has 16 cores en 32 cores total and a lot of ram!
Installation on Nested ESX VM’s
For the nested ESX VMs I am using the generic ESX image. In my lab I’m using 4 nested ESX nodes for vSAN ESA (Minimum is needed is 3).
To config the 4 Hosts for vSAN ESA I used my own powercli script where I blogged about here: Config vSAN ESA host or VCF ESA vSAN Host the easy way with Config-VSAN-ESA-VCF-Lab-Host Script
You can download the script HERE!
Installing the Vib works still great. No problem with ESA Pre-check: ✅
If you have problems check this great blog from William Lam: vSAN ESA Disk & HCL Workaround for VCF 9.0
At first you have to deploy the VMware Cloud Foundation
The name of the installer is a little off: VCF-SDDC-Manager-Appliance-9.0.0.0.24703748.ova
If you don’t have a download token from the Broadcom support then it’s a little complicated. I won’t go into this in depth now but here is a nice article if you have a Synology Nas: VCF 9.0 Offline Depot using Synology
(Note: For any vExpert reading this… You need to claim your own domain name because build your profile is not working with a @hotmail.com or @gmail accounts, Noted by Broadcom)
I looks something like this!
Next thing to do was: Disable 10GbE NIC Pre-Check in the VCF 9.0 Installer or this should also work: How to change the vmxnet3 link speed of a VM (Not tested)
I have deployed the beta in a earlier state. For deployment I used the same json file to start with.
VCF Automation will deployed automatically that was only the change that I changed it
Above the screenshot from the host with de Nested VCF lab deployment it’s quite cpu intensive (2 x CPU 8cores / 16 Threads)
After the deployment I did three things
– On the vSAN ESA Cluster (Disable Auto Disk Claim) (Keep the warning away)
– On the NSX VM reduce the cpu reservation from 6000 🡪 2000 (It helps but not enough)
– VCF automation VM 24 cores to 16 cores.
NSX en VCF automation are really CPU intensive.
I had deployed also two Edge servers but the server did not like that. Edges are also CPU intensive.
I am thinking about adding MS-01 or MS-A2 for splitting some load.
What are my things I would like to test
The VCF Cloud Foundation Installer makes lives a lot easier! More about this coming very soon
The VCF Operations Console is looking good! I used it in the VCF 9 beta
More about this also later!
William Lam created the vSAN ESA HCL hardware mock VIB for Nested ESXi.
It works great for vSAN ESA or for VCF vSAN ESA Nested nodes.
You can download the needed vib here
A whale ago I found a easy way to the deployment of a VMware SDDC based on vSAN ESA: VCF automated lab deployment with vSAN ESA
It works great. I a few hours you have a working VCF management domain on vSAN ESA.
Sow what about the script
For the Workload domain domain I created de nested ESXi vm’s manually .
And I did some testing for VCF 9 which I created the nested ESXi host also manually.
Configure by hand takes a lot of time. So I did some scripting and I used some code
from: VCF automated lab deployment with vSAN ESA
So I created a script that does the following:
It works great for ESXi Nested nodes (ESXi 8.0u3b and ESXi9.X beta)
You can find the script HERE!!
For my home lab I used for many years the Netgear GS108 unmanaged switch.
I works great.
Because the rest of my home work exist of Ubiquity devices I’ve wanted for a long timo to replace my netgear switch’s with a managed one. But the question was which one?
I used the Flex Mini behind the TV and for my PC and NAS. But not for my “server”.
With the release of the Flex 2.5G Switch it will make my homelab a little futher proof.
I have bought in de past the Synology DS723+ which support 2.5G en 10G with a optinal adapter. I wrote about it here
I have the luck that I don’t need to consult with the WAF ☺ Sow the decision was made quickly
now I can really use VLANs within the various virtualization solutions and keep things separate from each other
My labs are even more realistic because of this upgrade!
For your VCF homelab you wan to keep the resources small with a little bit overhead.
In this post I will talk about how i managed to deploy a VCF Workload Domain with a single NSX Manager, instead of the standard three nsx nodes.
Warning: Use this only in a Homelab!
The trick is to SSH into your SDDC Manager using the vcf user, and the password used during bring-up of the management domain.
When logged in, run su and log in as root using the password used during bring-up.
run: vi /etc/vmware/vcf/domainmanager/application-prod.properties
Hit i in your keyboard to go into insert mode. Go to the end of the file, and append the following:
nsxt.manager.formfactor=medium
nsxt.manager.resources.validation.skip=true
nsxt.manager.cluster.size=1
nsxt.manager.wait.minutes=120
This will make it so that any workload domain you deploy has one NSX Manager, and that it uses a smaller size. Once done, hit ESC in your keyboard, then type :wq and hit enter to save the file. (w = write, q = quit).
Then run systemctl restart domainmanager and you are good to go!
This worked in my nested Cloud Foundation deployment in my lab running 5.2.1.0.
You will still have to fill in the information for the extra nodes in the UI.
You must be logged in to post a comment.