PCA 3.x health monitoring and upgrade workflow overview

This overview is for appliance administrators of Private Cloud Appliance 3.x (Service Enclave). It explains where health information lives and how the upgrade process is structured. It is not a replacement for the Upgrade Guide: upgrade steps change between software versions, so always follow the guide and release notes for your exact target release.

Where health data lives

• Prometheus stores the metrics collected across the system.
• Loki stores logs collected from across the appliance (the Concepts Guide says Fluentd gathers them).
• Grafana is the central place to view both. As the Service Enclave admin user, open it from the Monitoring tile in the Service Web UI. Other Grafana users can go to https://grafana.pca-name.domain/login.

Grafana shows time in the user’s preferred time zone, which may differ from the appliance’s. Set your Grafana preferences to the appliance time zone so that timestamps line up with logs.

Dashboards worth knowing

• Platform Health Check (folder PCA 3.0 Service Advisor): results of the built-in health checkers. Filter by platform service or status; each result’s timestamp links to the related Loki logs.
• Server Stats (folder Service Monitoring) and Node Exporter Full: host-level telemetry.
• Kubernetes and OKE folders: health of the internal container platform and, if used, the Kubernetes Engine.

The health checks run at regular intervals as Kubernetes CronJobs and report healthy or not healthy. Examples named in the Concepts Guide include certificate, etcd, Vault and MySQL cluster checkers. Do not edit the Oracle-provided dashboards: Oracle may need them for support, and the My Dashboards folder is read-only. Create custom dashboards in a new folder.

Capacity checks

The Service CLI provides quick capacity views, such as getFaultDomainInfo for compute capacity and list ZfsPool / show ZfsPool for storage. The ZFS pool manager raises a fault when pool usage passes a threshold (default 0.8, that is 80 percent), which you can view with show ZfsPoolManager.

How upgrades are delivered

Upgrade content (packages, archives, deployment charts) comes as ISO images. During preparation the ISO is downloaded to shared storage and the upgrader updates itself to the version in the ISO. The target versions are compared with what is installed, and the result is recorded in an upgrade plan that drives every later step in a fixed order. You can view it in the Service Web UI under Maintenance > Upgrade Plan.

Pre-checks and locks

• Pre-checks are run automatically by the upgrade code whenever an upgrade command is entered. All checks must pass, even for a single-component upgrade.
• Since January 2024 releases, prerequisite software versions are enforced: you may need to install a minimum version before the version you actually want.
• Some procedures require a provisioning lock and a maintenance lock. The large workflows set and release them automatically; after manual procedures you must release them so the system returns to normal operation.

Full rack upgrade

For releases from April 2025 onward, Oracle calls the full rack upgrade the preferred method and strongly recommends it; per-component procedures remain for special cases. The Concepts Guide lists the workflow order as: ZFS Storage Appliance firmware, compute nodes, management nodes (host OS, MySQL cluster database, secret service, Kubernetes platform, microservices), OCI images, ILOM firmware, and switch firmware. The exact order for your release is whatever the upgrade plan says.

In the Service CLI, the documented flow looks like this. Run the health check (or a verify-only run) first, and only start the upgrade once preparation in the Upgrade Guide is complete:

PCA-ADMIN> upgradeRack action=HEALTHCHECK
PCA-ADMIN> upgradeRack type=ISO action=APPLY verifyOnly=True
PCA-ADMIN> getUpgradeJobs requestId=UWS-<request-id>

PCA-ADMIN> upgradeRack type=ISO action=APPLY
PCA-ADMIN> getUpgradeStatus requestId=UWS-<request-id>
PCA-ADMIN> getUpgradeJobs requestId=UWS-<request-id>
PCA-ADMIN> show job id=<job-id>

The upgrade is done when the request shows Status = Passed, all jobs show Passed, and show job reports Run State = Succeeded. getUpgradeRequests helps if you lose track of a request ID. The Service Web UI offers a force option; Oracle says to use it only when instructed by Oracle.

Planning tips

• Read the release notes and the Upgrade Guide’s preparation chapter before every upgrade, including sections on Oracle Linux 8 migration of the management cluster and compute node upgrades, which affect timing and service impact.
• Check the Platform Health Check dashboard and fix existing faults before you begin.
• Tell tenancy users about the maintenance window, since compute node upgrades can affect running instances.

Official documentation

• Oracle PCA Administrator Guide (3.0): Monitoring with Grafana
• Oracle PCA Concepts Guide (3.0): Upgrade
• Oracle PCA Upgrade Guide (3.0): Full Rack Upgrade