Build a Proxmox VE cluster with pvecm and live-migrate VMs

A Proxmox VE cluster is a group of nodes managed together. All nodes share the configuration in /etc/pve, you can manage everything from any node’s web UI, and you can migrate guests between nodes. The cluster stack uses Corosync for communication and decides by quorum (a majority of votes) whether the cluster may change its state.

Before you start

• Install Proxmox VE on every node with its final hostname and IP address. They cannot be changed after the node joins a cluster.
• All nodes must reach each other on UDP ports 5405-5412 (Corosync) and TCP port 22 (SSH).
• Time must be synchronized on all nodes.
• Corosync needs a reliable, low-latency network (under about 5 ms). A dedicated NIC for cluster traffic is recommended, plus a second link on a different network for redundancy.
• For HA you need at least three nodes. A two-node cluster can use a QDevice as a third vote.
• A node that joins a cluster cannot contain guests, and its /etc/pve is overwritten. Back up and remove guests from a node before joining, then restore them under unused IDs afterwards.
• Keep all nodes on the same Proxmox VE version.

1. Create the cluster on the first node

The cluster name must be unique in the network and cannot be changed later. With --link0 you choose the address (and therefore the network) Corosync uses; a --link1 adds a redundant link.

pvecm create lab-cluster --link0 10.0.0.11 --link1 192.0.2.11
pvecm status

The same is available in the web UI under Datacenter → Cluster → Create Cluster.

2. Join the other nodes

Log in to the node that should join and point it at an existing member. It asks for that node’s root password and shows its certificate fingerprint to confirm. Use the IP address of the existing node, and the joining node’s own cluster-network address for --link0.

pvecm add 10.0.0.11 --link0 10.0.0.12 --link1 192.0.2.12

In the web UI, copy the Join Information from an existing node (Datacenter → Cluster) and paste it into Join Cluster on the new node. After joining, the node gets a new certificate, so reload the browser and log in again.

3. Check the cluster

pvecm status
pvecm nodes

In pvecm status, look for Quorate: Yes, the expected and total vote counts, and every node listed under membership information. Each node has one vote by default. A cluster that loses quorum switches /etc/pve to read-only, so you cannot start guests or change configuration on the minority side.

4. Live-migrate a VM

With guests on shared storage, live migration only copies the VM’s memory and device state, and the switchover pause is typically well under a second. It also works with local disks by copying them over the network, which takes longer.

qm migrate 110 pve2 --online
qm migrate 111 pve2 --online --with-local-disks
pct migrate 200 pve2 --restart

• Live migration requires both nodes in the same cluster, a reliable network between them, and the same or a newer Proxmox VE version on the target.
• VMs with passed-through PCI or USB devices cannot be live-migrated.
• CPUs should be from the same vendor with similar features. Using a generic virtual CPU type rather than host helps when nodes have different CPU models. Test before relying on it.
• Containers cannot be live-migrated; --restart stops, moves and restarts them, with a short downtime.
• Migration traffic is encrypted through SSH by default. --migration_network selects which network to use.

Removing a node

Warning: removing a node is not reversible, and a removed node must never come back online with its old configuration, or it can break the cluster. Before removing, move all guests off the node, remove it from replication jobs and back up its local data. Then power it off permanently and run pvecm delnode from a remaining node:

pvecm nodes
pvecm delnode pve4

To reuse that server in the cluster, reinstall Proxmox VE and join it again. If the node was part of a Ceph cluster, follow the Ceph removal steps in the documentation first.

Official documentation

• pvecm(1): Cluster Manager
• qm(1): Migration