vmware-vpostgres service not starting after vCenter reboot

Customer raised an incident on Friday afternoon. vCenter is no longer accessible after patch installation. He already figured out that vmware-vpostgres is failing to start and so on vpxd and webclient will not start. You can see a lot of service are not started due to vpostgres is a dependency for them.

He also tried to manually start the vmware-vpostgres service without success

When looking into the log you can see the error causing the service not to start. Wait what is %(0)s?!

It all seems like the VMware KB76374 can help. But when looking into the vpostgres log you can see nothing. So, in fact, the vpostgres service is failing before it effectively tries to start!

After some googling I found the VMware KB80020 which seems to match my failure.

I checked this vCenter also ran into this issue with

/usr/lib/vmware-vmafd/bin/vecs-cli entry list –store TRUSTED_ROOT_CRLS | grep Number

Damn – that is it! That the services are not starting is caused due to corrupted certificates under /etc/ssl/certs , which causes an unexpectedly high number of certificate entries in TRUSTED_ROOT_CRLS store.

For problem solving you need to follow the resolution part of the VMware KB. You will need to download the attached script. Upload it to the vCenter appliance and let it run. It will fix the TRUSTED_ROOT_CRLS store. The you need to stop all services on the appliance and manually start them.

# service-control –stop –all
# service-control –start –all

vCenter will be available again!