And we’re back for post 3 of the VCP-DCV 2020. Section 3 is Planning and Design. VMware has no testable objectives for this section, so we won’t write anything on it. Perhaps we may cover it in a future blog if enough are interested in it, on to Section 4.
In the previous version of vSphere, you could deploy vCenter Server on a single server, which included the SSO components, or you could break it into multiple components. Sign-ons were serviced by the Platform Services Controller, and you could set more than one up to help with a massive load or multiple sites. With vSphere 7, this has been changed. You now only have the embedded vCenter Server, which has all components in a single appliance. You can scale out for larger loads. For resiliency, you can deploy vCenter High Availability. For multiple sites, you can leave the single vCenter in place, or you can have a second vCenter for the other sites and connect them with Enhanced Link Mode. Enhanced Link Mode enables you to manage all vCenters from a single HTML5 client.
vCenter Single Sign-On
This service provides authentication services for vSphere software components, not just vCenter Server. It allows communication between the components via a secure token exchange, and all communication is encrypted. The default domain is vsphere.local, but can be changed during setup. It is also good to note that you dont want to have your single sign-on domain be the same as your Active Directory domain name. This can cause authentication problems if you do. Single sign-on can also authenticate from other identity sources, as well cover in Objective 4.3.2 and .3. Components of SSO include:
Configuring a single sign-on (SSO) domain is done during the initial setup of the vCenter Server Appliance. The vCenter Server Appliance install has two stages. Single sign-on installation happens in the second stage. Here is the screen to set up SSO. You have the option of creating a new SSO or joining an existing one.
To create a new, you will need to supply a domain name and password. Once again, make sure the domain name chosen is not the same as your Active Directory domain. After created, you will need to add users and groups or configure an identity source to enable other users.
As with previous versions of vSphere, vCenter Server can be joined to an already existing SSO domain. This can be done both during the setup of the vCenter Server appliance, or it can be performed via command line later by repointing an appliance to the domain. Reasons to do this include:
Up to 15 vCenter Server appliances can be connected and displayed using a single-window view.
You configure virtual networking in different ways, depending on your environment. Configuring VSSs can be done using the ESXi HTML5 client as seen here
Physical NICs are how you access your Physical Network. You create VMKernel ports, which are how ESXi accesses the internal switch for management tasks, and you have Virtual switches to connect both. Finally, you have port groups, which is a grouping of vNICs or the virtual machine NICs. A better way to show this is with a picture.
1. These are the VMKernel ports – These are used for management tasks such as vMotion etc.
2. pNICS or Physical Network cards are on the other side and how you reach the physical network.
3. VM Network is the name of my Port Group, which is how I group all the NICs from the VMs underneath. I group them to perform tasks on all of them easier.
4. The construct in the middle is my Virtual Switch. This one is a VSS
Advanced options for virtual standard switches are:
The options we need to cover a bit more are Security and Teaming. Security settings include:
The last one we cover is Teaming and failover. You have 3 types of load balancing available + 1 failover:
Failover can use either link status or beaconing to detect a failure. You can decide what you want ESXi to do in case of failure. Failback will check to see if a failed NIC has recovered and, if so, return it to active duty. The notify switch option allows the host, if it has another physical NIC attached and active, to notify the switch of the failure.
Identity sources in vCenter Server allow users from other places, such as Active Directory, to log in to vCenter Server using the same username and password. Supported identity sources include
In this example, I add my Windows AD as an identity source. To add the identity source, do the following:
Click on the Menu and select Administration
Click on Configuration underneath Single Sign-On
If you havent done so already, you must join the Active Directory Domain before adding it as an identity source. You do that by selecting the Active Directory Domain in the center pane. Click on Join AD.
Fill out the needed information and select Join. You will need to reboot the vCenter Server node after this is done (just like a Windows machine, eh? )
You can reboot the node afterward, by selecting System Configuration from the left and then highlight the node, and click Reboot Node. Enter a reason on the popup and click reboot. You will lose connectivity to the vCenter Server during the reboot process.
When the system comes back up, login and navigate to administration and Single sign-on again, now you should be able to add the Identity Source. Click on Add.
In my case, I added a Windows Active Directory Domain, so it is already filled out for me. I can choose if I want to use a Machine Account (vCenter Server becomes a computer account in the domain) or if I want to use an SPN. I will leave it as a machine account. You could use SPN if you expect to change the computer name of vCenter at some point.
I will now change the default authentication source to be the AD domain. If you set this, instead of needing to use your [email protected], you only have to use username.
You may need to add access to a user. To do this, click on users and groups on the left and then click on Groups and the group you want to edit. Click the ellipsis in front and click Edit.
Then select the domain you will be pulling the user from and then type in the username. It should auto finish for you and then click Save
Now log out and try again. Success!
With vCenter Server federation, you are authenticating with Active Directory. How is this different from regular? With federation in place, you arent providing your credentials to vCenter Server at all. vCenter Server trusts Active Directory, and it redirects the credentials to AD. How is this better?
Currently, vCenter Server only supports Microsoft Active Directory Federation Services. The components needed are:
To configure it, you need to follow the flow set here. Here is the flow chart VMware provides for this.
Im only going to cover the vCenter piece of this. To do this, you will need to go back to the Administration > Configuration.
Click on the I next to Change Identity Provider on the right side. Copy both of the links that pop up. You will need these later.
You need to create an OpenID Connect configuration in AD FS and configure it for vCenter. Follow the directions here to do that in Windows 2016. Record the following when you created the AD FS group.
When that is complete, go back to your vCenter Server and click on the Change Identity Provider link on the right side.
This link brings up a wizard to configure the Main Identity Provider. Select Microsoft ADFS and click Next
This step, you will enter the information you wrote down from the AD FS server above.
For the next screen, enter user and group information for AD over LDAP connection.
Definitions for the fields are as follows:
Finish that up and then assign users using that domain as you would for other identity providers
LDAP has two supported options. Active Directory over LDAP and OpenLDAP. To choose one of these options, go back to the same place in Menu > Administration > Configuration and click on Add
Here are your options. If you click on Open LDAP, you must fill out the above information. The definitions and syntax will be the same as shown above – here it is again, so you dont need to click up.
For this one, see above Objective 4.3, as that is the one I chose to do first.
The vCenter Server Appliance can be configured both from a GUI and CLI. The steps I show here will be using the GUI.
The download from VMware is an ISO file. This ISO can either be unzipped or, using Windows 10, and you can double click on it to mount the CD on your computer. Then navigate down to the following folder (this is different if you are using Linux or Mac)
Run the executable shown in the picture. Once this is run, you have the following screen appear.
To install a new vCenter Server Appliance, click on Install. The following screen appears. It describes the first of two stages for the vCenter Server Appliance install.
Select Next. Accept the terms of the License Agreement and click Next.
You need to enter the ESXi host or vCenter Server you will install this vCenter to. You also need to tell it the port (default is 443) and then enter the username and password for the resource you want to use.
Accept the Certificate Warning and give the vCenter Server a name and password.
You now need to decide the size of your vCenter Server. This will be dictated by the number of hosts and services you are using.
Select the datastore you will use for the vCenter Server files.
Configure your network settings.
Review and start the install. When Stage 1 is complete, you will get the following screen.
Continue on to the Second Stage – this is where you configure the vCenter Server.
First step, set up time synch and SSH access.
Next, we need to create a new SSO domain or join an existing SSO domain.
You now need to decide if you want to join the VMware Customer Experience Improvement Program. This means you will be sending information back to VMware, although it is scrubbed of identifying data.
Review the information and then click Finish.
There are more configuration options you can do, such as backups, but that will be later.
We’ve already covered what VMware HA and its advanced options are/do. Let’s discuss how to configure it.
First, click on the cluster you want to work on. Next click on Configure, then on vSphere Availability
Then click on Edit for vSphere HA.
To activate HA, you will need to click on the toggle shown. This enables the rest of the options on here. You can configure them as your environment needs. Again, if you need to, go back to the first section (Objective 1.6.4) to remind yourself what those options do. The screens look like this
Admission Control
Heartbeat Datastores
Advanced Options
To enable Proactive HA, click on the Edit button on Proactive HA.
You will need to click on the toggle to enable this as well. Keep in mind that if no provider is found, it won’t have any automated response.
We covered the concept of vCenter HA back in Objective 1.2. Quick refresher for you though. vCenter Server HA works by using a total of 3 nodes. One Active, one passive, and one witness nodes. The active node is the only machine the admin will interact with. All the nodes communicate with each other in the background over a separate HA network to continually replicate data to the passive node and the witness provides quorum in case of split-brain scenarios (network connection loss). If something happens to the active node, the passive will automatically switch and pick up the slack.
What do we need for vCenter Server HA? A few things.
If you use the automatic configuration you will
What does this look like? I thought you’d never ask…
Setting up the network
Getting to the configuration page for vCenter Server HA
This next step is where you need to enter in information on the resources and networking
The last step here is to put in the network configuration.
After completed, click Finish. The installer will now begin its work. When complete, the config page will look something like this.
Content Libraries are containers for VMs, templates, and other types of files used for the vSphere infrastructure, such as ISOs. You can distribute this content to other vCenters if allowed. There are two types of content libraries you can create:
Before we dive into how to set up a Content Library, I did want to point out a major feature of vSphere 7 Content Libraries. And that is the ability to check out and save a history of all changes made to a template. This is awesome and allows you even revert if needed. Version control. Yes, ok back to set up.
Click Menu at the top bar and then click on Content Libraries
Click on Create
Type in a name and any notes you want to include. You also need to choose which vCenter to host it. Then click next.
Select if this content library will be local or subscribed. You also need to decide if local, do you want to allow publishing.
The next step is deciding where the files will be held. Specifically, which datastore you will use. (I don’t have all mine currently running, hence the red)
At this point, you are done. Look over the parameters and if they look correct, click finish.
You can now start adding templates and ISOs to your content library!
vCenter Servers are important data that should have backups. Fortunately, VMware has given us a utility to take backups of its important data. VMware supports FTP, FTPS, HTTP, HTTPS, SFTP, NFS, or SMB files share to store the backup. Keep in mind that in order to restore this data, you would have to use either the CLI or GUI install tool to install a new appliance. The second stage of the install is where it would take your information and restore it to the new appliance. Now let’s show how to configure it.
The first step is logging into the vCenter Server administration web UI. This is at https://[vCenter Server FQDN or IP address]:5480.
Click Backup on the left pane.
Click on Configure.
This allows us to create a schedule for our backups. If we only wanted to perform a single backup, we could just click on Backup Now. If we setup a schedule, we can perform a one-off backup using that configuration however. Put in the configuration you wish to use. You can also tell it how many backups to retain, as some of these can get quite large. I am going to say 5 backups. It lists the size there at the end.
When you click create it creates that schedule. It won’t automatically create a backup immediately. If you want to test it, click Backup Now and select at the top, Use backup location and username from backup schedule. When you click start it will attempt to take the backup. You do not have to have the folder created if using a user that has create rights. Once done, it will look like this.
VMware has come a long way from when I started troubleshooting their products. Their logs have gotten easier to get to and improved in their quality. What I will do here is give you a quick overview of where to find the logs and how to read them.
ESXi Logs
Before, the most straightforward option was to open an SSH session to the host and look at the logs; you can easily do that from within the host UI now. If you go to Monitor, you can see a list of all the logs available to peruse.
Here in the screenshot, you can see
Here is a list of logs on the ESXi host and a description of what the log does.
You can still access these logs through the DCUI or an SSH session as well.
All right, so you have the log now… How do you use it? Here is a sample taken from a VMKernel.log. This sample was after shutting down a switch port using a Software ISCSI controller to a SAN LUN.
2013-12-05T21:42:47.944Z cpu25:8753)<3>bnx2x 0000:04:00.0: vmnic4: NIC Link is Down
2013-12-05T21:43:12.090Z cpu16:8885)WARNING: iscsi_vmk: iscsivmk_StopConnection: vmhba45:CH:0 T:0 CN:0: iSCSI connection is being marked “OFFLINE” (Event:4)
2013-12-05T21:43:12.090Z cpu16:8885)WARNING: iscsi_vmk: iscsivmk_StopConnection: Sess [ISID: 00023d000001 TARGET: iqn.2001-05.com.equallogic:0-8a0906-0f6407f09-1173c8a93ab4f0f6-aim-2tb-1 TPGT: 1 TSIH: 0]
2013-12-05T21:43:12.090Z cpu16:8885)WARNING: iscsi_vmk: iscsivmk_StopConnection: Conn [CID: 0 L: 192.168.3.123:61632 R: 192.168.3.3:3260]
2013-12-05T21:43:22.093Z cpu31:8261)StorageApdHandler: 248: APD Timer started for ident [naa.6090a098f007640ff6f0b43aa9c87311]
2013-12-05T21:43:22.093Z cpu31:8261)StorageApdHandler: 395: Device or filesystem with identifier [naa.6090a098f007640ff6f0b43aa9c87311] has entered the All Paths Down state.
Lets decipher this a bit more.
Some entries are a bit more challenging to read than others, but the structure stays pretty close. You can also use something like Log Insight to help search through the logs and decipher them.
vCenter Server Logs
We have logs we may need to retrieve for vCenter Server as well. Unfortunately, it doesnt have a browser like the hosts. (Hint Hint VMware) Here is where you can get to them, in any case.
This picture shows accessing the Appliance Config at port 5480.
Once this is done downloading, you have a decent size .tar file. You have to unzip this a couple of times. When you finally have a typical directory structure, all the logs are under the /var/log/vmware folder. Here is a list of the files and locations and what they do.
Windows vCenter Server | vCenter Server Appliance | Description |
vmware-vpx\vpxd.log | vpxd/vpxd.log | The main vCenter Serverlog |
vmware-vpx\vpxd-profiler.log | vpxd/vpxd-profiler.log | Profile metrics for operations performed in vCenter Server |
vmware-vpx\vpxd-alert.log | vpxd/vpxd-alert.log | Non-fatal information logged about the vpxd process |
perfcharts\stats.log | perfcharts/stats.log | VMware Performance Charts |
eam\eam.log | eam/eam.log | VMware ESX Agent Manager |
invsvc | invsvc | VMware Inventory Service |
netdump | netdumper | VMware vSphere ESXi Dump Collector |
vapi | vapi | VMware vAPI Endpoint |
vmdird | vmdird | VMware Directory Service daemon |
vmsyslogcollector | syslog | vSphere Syslog Collector |
vmware-sps\sps.log | vmware-sps/sps.log | VMware vSphere Profile-Driven Storage Service |
vpostgres | vpostgres | vFabric Postgres database service |
vsphere-client | vsphere-client | VMware vSphere Web Client |
vws | vws | VMware System and Hardware Health Manager |
workflow | workflow | VMware vCenter Workflow Manager |
SSO | SSO | VMware Single Sign-On |
It would be simpler again to use a program like Log Insight to help you parse through the logs. And you wouldnt need to download them as they are streamed directly to Log Insight. Youll see output similar to what I mentioned above.
We’ve covered VMware attestation and the security piece of this earlier. The Trust Authority is enabled on a dedicated vCenter Server cluster (known as the vSphere Trust Authority Cluster) in order to attest VMware ESXi hosts are secure. There are Pre-Reqs required (here) before you can set up the Trust Authority. There are a number of tasks (10 in fact), each with their own respective steps that need to be accomplished to make this work. Those tasks are as follows
In 5.x and even in 6.x days configuring certificates wasn’t very easy. Matter of fact it was downright painful in some cases. In vSphere 7 there have been improvements made to try to make this simpler for admins. To be clear, the VMware Certificate Management (VMCA) is not a full-fledged PKI solution, so you can’t request certs for other purposes. For your VMware environment it is just enough.
There are a number of ways for you to manage your vCenter Server certificates. You can use
There are 4 modes you can run certificates through vCenter Server.
In the first mode, Fully Managed Mode, the vCenter Server generates a root certificate at first install and uses that to manage intra-cluster certificates as well as the certificate we use when we log on, the machine certificate. You can regenerate that root cert using your own company information if desired.
In Hybrid Mode, you replace the machine certificate that the vSphere client uses so that it can be accepted without intervention by default browsers. Something like a GoDaddy certificate for example. The VMCA still manages internal certificates making this simple and easy.
In subordinate mode, the VMCA will act as a subordinate CA. The vCenter Server still generates certificates but it generates them as part of the larger organization’s.
The last mode is Full Custom. In this mode, the VMCA isn’t used at all, and all certs must be installed and managed by a person.
To work with the certificates, you can go through the HTML5 client by going to Menu > Administration > Certificate Management. This is what that screen looks like.
To work with it, you can choose the Actions menu
Using the machine cert menu, you can renew, change, or generate a new one to replace an expired certificate. Under Trusted Root Certificates, you can select Add to import a certificate chain.
If you want to run the Certificate Manager Utility, the location is as follows.
/usr/lib/vmware-vmca/bin/certificate-manager
When you run that, it will present you options to choose from. You follow the options to complete the workflow.
In vSphere, certificates are used for, encryption of communication, authentication of vSphere services, and internal actions such as signing tokens. The internal VMware Certificate Authority can supply all the certificates needed for VMware services, but a company might institute a PKI or Public Key Infrastructure. A PKI can include, hardware, software, policies, processes, and procedures to manage certificates and their lifecycle and public keys.
A smaller business might decide to not use a full infrastructure, but should still have some sort of policies and procedures around how certificates are dealt with. VMware can work within a PKI to create certificates for an organization. It is suggested the best way to accomplish most business’ tasks would be to setup a Hybrid Mode VMware vCenter Server. This setup allows you to replace the machine certificate, or the one used to login to VMware vCenter Server and allows the VMCA to manage all the other certificates with its own self-signed certificates. VMware created a blog with some suggestions on how to implement certificates here.
In vSphere 7 Lifecycle Manager is a service that enables updating and upgrades for ESXi hosts. To setup Lifecycle Manager Click on Menu > Lifecycle Manager
There are a number of things you can do from there. Image Depot allows you to setup a base image for clusters and even add vendor drivers or installation bundles. The Updates tab allows you to remove updates from baselines. Imported ISOs allows you to import a ESXi ISO image to use for an update baseline. Baselines is one or more patch, extensions, or updates that you want to apply to your vSphere infrastructure. Settings allows you to configure LCM.
Once you have baselines and everything how you want them, you can apply them to hosts at an individual level or at a cluster level. Checking compliance will check the host against the baseline. If the host doesn’t have the updates on it, it will show out of compliance. You can then choose to stage, to download the updates to the host and then reboot at your leisure, or you can remediate the host immediately. If there is a new baseline, you can attach it to check compliance.
With the advent of UEFI firmware, Secure Boot is a feature that will refuse to load any driver or app unless it is cryptographically signed. You might be more familiar with operating systems such as Windows 10 or Ubuntu using this to prevent unwanted modification to the boot drive. Starting with vSphere 6.5 VMware has been able to use this as well. The host needs to be compatible and you can run validation script to check. Once enabled, you must use an ESXi bootloader that contains VMware’s public key. Trying to upgrade a system using esxcli commands with Secure Boot enabled will fail to update the bootloader and won’t work.
If a physical host has a TPM included, you can use host attestation to authenticate and securely boot the host as well.
TCP/IP stacks allow you to change DNS and gateway configuration for a specific traffic. You can also change congestion control, the number of connections allowed, and even the name of the stack. To do this, click on the host, and then click Configure > Networking > and TCP/IP configuration. You can see current settings below.
To change them, click on the one you want to change and then edit. The first screen allows you to either obtain settings from an existing VMkernel or manual
The next screen allows you to set a IPv4 or 6 gateway for routing. It mentions if you change this, you might lose connectivity between the host and vCenter.
The next screen allows you to name the stack.
And the last screen allows you to set connection limit and congestion algorithm type.
Finally, you CAN create a brand-new stack if you need to. This is done by using the following CLI command on a host.
esxcli network ip netstack add -N=”stack_name“
I will recycle this heading from my previous study guide since I don’t see any differences.
Host profiles provide a mechanism to automate and create a base template for your hosts. Using host profiles, you can make all your hosts the same. VMware will inform you if your host is not in compliance yet, and then you can take steps to remediate it.
You access it under Policies and Profiles
There is a process to it. Here it is:
At first, I believed they were looking for installation types, but after further study, I believe they are looking for boot options at the boot command line. This is accomplished by pressing Shift + O in the ESXi installer screen shown below.
At the command prompt that is displayed, enter in
ks=[location of installation script] boot command line options
Some of the boot options include (image grabbed from VMware documentation)
As shown, the location can be in several different places, including a USB drive or CD-ROM.
vSphere Quick Boot is pretty amazing. If you have a server that supports it, instead of doing a lengthy hardware reboot, where the server tests memory etc., it will just skip hardware initialization and just restarts the software. To determine if your host is compatible, check here or you can run this command at CLI on the host:
/usr/lib/vmware/loadesx/bin/loadESXCheckCompat.py
To configure vSphere to use it, click Update Manager or Lifecycle manager from the Menu.
Next click on Settings then Images under Host Remediation.
Now click Enable Quick Boot. – That’s it.
Tune back for our next object soon!