![[Google Cloud Documentation]](https://www.gstatic.com/devrel-devsite/prod/v0f3a5b8b76e4a120372a69044e02756e703b6140f9f1812c6e4a0407b19864b7/clouddocs/images/lockup_full_color.svg){kind=link}
![[]](https://docs.cloud.google.com/_static/clouddocs/images/icons/products/compute-engine-color.svg){kind=link}
Sole-tenancy lets you create VMs on a specific sole-tenant node or in a group of nodes. If you create a VM in a group of nodes, Compute Engine optimizes the space available for VMs in the node group by using a bin-packing algorithm to determine the node to place the VM on. For information about sole-tenancy, see Sole-tenancy overview.
As your workload runs, you might want to move VMs to a different node or node group. To move sole-tenant VMs to a different node or node group, you can manually initiate a live migration. You can also manually initiate a live migration to move a multi-tenant VM into sole-tenancy.
Use cases for manual live migration
The following list shows some use cases for manually live migrating VMs:
Increase utilization and optimize costs. You might be able to consolidate VMs onto fewer sole-tenant nodes.
Logically reorganize VMs. Use different sole-tenant node groups or nodes to separate VMs based on their workload type.
Isolate workloads to meet compliance standards or improve performance. Manually live migrate multi-tenant workloads that require hardware isolation into sole-tenancy to meet compliance standards or to improve performance.
Increase portability of VMs. You can't modify certain node template settings, such as the maintenance policy, the maintenance window, and settings related to local SSD. By using manual live migration, you can migrate VMs to a node group with different settings.
- Improve performance by rebalancing oversubscribed sole-tenant nodes. If you are overcommitting CPUs on sole-tenant VMs, you can manually live migrate any VMs that are underperforming to other sole-tenant nodes.
Examples
To understand how manual live migration supports the preceding use cases, review the following examples.
Manual bin packing
To more efficiently arrange VMs in a node group to fit additional VMs, you can use manual live migration to choose which nodes to put sole-tenant VMs on.
Consider a sole-tenant node group with the following initial state, on which you are trying to schedule an additional VM with 16 vCPUs:
| Initial state | Node 1 | Node 2 | Total |
|---|---|---|---|
| vCPU capacity | 80 | 80 | 160 |
| VM vCPUs | 72 | 64, 8 | 144 |
| Unused capacity | 8 | 8 | 16 |
There is not enough space on any node to schedule a VM with 16 vCPUs. However, there is enough aggregate space.
To make space for the 16 vCPU VM, initiate a live migration of the 8 vCPU VM from Node 2 to Node 1. The following table shows the new VM configuration:
| Final state | Node 1 | Node 2 | Total |
|---|---|---|---|
| vCPU capacity | 80 | 80 | 160 |
| VM vCPUs | 72, 8 | 64, 16 | 160 |
| Unused capacity | 0 | 0 | 0 |
The following figure summarizes this process:
[Manual bin packing of VMs to make room for a larger VM.]
Figure 1: Manual bin packing of VMs to make space for a VM that has more vCPUs.
![[Manual bin packing of VMs to make room for a larger VM.]](https://cloud.google.com/static/compute/images/sole-tenancy/sole-tenant-node-live-migration-1.svg){kind=link}
Autoscale after bin packing
After bin packing, there might be sole-tenant nodes without any VMs. In this case, the sole-tenant node autoscaler can remove the empty node.
Consider a sole-tenant node group with the following initial state. If you move the 8 vCPU VM, the node group autoscaler can remove a node:
| Initial state | Node 1 | Node 2 | Total |
|---|---|---|---|
| vCPU capacity | 80 | 80 | 160 |
| VM vCPUs | 8 | 72 | 80 |
| Unused capacity | 72 | 8 | 80 |
To notify the node group autoscaler of an empty node, initiate a live migration of the 8 vCPU VM from Node 1 to Node 1. The following table shows the new VM configuration:
| Final state | Node 1 | Node 2 | Total |
|---|---|---|---|
| vCPU capacity | 80 | 80 | 160 |
| VM vCPUs | 0 | 72, 8 | 80 |
| Unused capacity | 80 | 0 | 80 |
Now that Node 1 is empty, the autoscaler can remove it from the node group. The following table shows the new VM configuration:
| Final state | Node 1 | Node 2 | Total |
|---|---|---|---|
| vCPU capacity | - | 80 | 80 |
| VM vCPUs | - | 72, 8 | 80 |
| Unused capacity | - | 0 | 80 |
The following figure summarizes this process:
[Manual bin packing of VMs to consolidate on one node. Then, the autoscaler can remove the empty node.]
Figure 2: Manual bin packing of VMs to consolidate nodes.
![[Manual bin packing of VMs to consolidate on one node. Then, the autoscaler can remove the empty node.]](https://cloud.google.com/static/compute/images/sole-tenancy/sole-tenant-node-live-migration-2.svg){kind=link}
Limitations
The following limitations apply when manually live migrating VMs:
Capacity limitations. During manual live migration of a VM within sole-tenancy, the VM consumes capacity from both the source sole-tenant node and the destination sole-tenant node until live migration completes. If there is not enough capacity on the destination host, Compute Engine does not move the VM.
General limitations. Manual live migration requests might fail if there are incompatible scheduling properties or other competing live migration requests. For information about how to troubleshoot this, see VM scheduling failures.
Managed instance group (MIG) limitations. You cannot manually live migrate VMs that are in a MIG to another sole-tenant node.
VM instance lifecycle limitations. You cannot update some properties of a VM, such as the machine type, without restarting the VM. Also, you cannot update these properties at the same time as you update node affinities. For more information about these properties, see Updating instance properties.
Pricing
There are no additional charges for manually live migrating VMs. For more information about how you are billed for sole-tenant nodes, see Sole-tenant node pricing.
If a sole-tenant node is empty after the migration and you have enabled the sole-tenant node autoscaler, manually live migrating VMs might lower your charges.