About instance flexibility in MIGs Stay organized with collections Save and categorize content based on your preferences.

Instance flexibility in managed instance groups (MIGs) lets you specify multiple machine types for the virtual machine (VM) instances in the group, rather than being limited to a single machine type. If a machine type is unavailable, the MIG can automatically select another compatible type from your list based on the current resource availability. This flexibility helps to improve resource obtainability, particularly for applications that can operate on different machine types and that require large-scale capacity or hardware with high demand.

This document describes how instance flexibility works in a MIG and its limitations. To know how to configure a MIG to use instance flexibility, see Add instance flexibility.

Instance flexibility

A MIG creates all its VMs based on the instance template that you select. The instance template specifies the machine type to use for each VM. With instance flexibility, you can list all the machine types that are compatible with your application.

The following image compares a MIG that uses a single machine type with a MIG that uses instance flexibility. As depicted in the image, a MIG with a single machine type creates identical VMs based on an instance template, whereas a MIG with instance flexibility can create VMs with different machine types and sizes.

[Comparison of a MIG that uses a single machine type with a MIG that uses instance flexibility]

How instance flexibility works

By default, a MIG creates all VMs using a single machine type from the instance template. You can set multiple machine types in a MIG by configuring an instance selection (instanceSelections) in the instance flexibility policy (instanceFlexibilityPolicy).

You can also specify machine types that run on different CPU platforms and those with different architectures (x86 and Arm architecture). To use machine types with different requirements for minimum CPU platform or disks, use the instance flexibility policy to override corresponding properties from the instance template. For more information, see How minCpuPlatform and disks overrides work.

Example for machine type overrides

If your workload can operate on a number of different machine types, you can configure a list of all machine types compatible with your application in a single instanceSelection as follows:

  "instanceFlexibilityPolicy": {
    "instanceSelections": {
      "instance-selection-1": {
        "machineTypes": ["c2-standard-8","c2d-standard-8","c3-standard-8", "c3d-standard-8"]
      }
    }
  }
  

The machine types within an instance selection have equal preference. If you want a MIG to choose machine types in a specific order, you can configure multiple instance selections, each with a list of machine types and a rank to define preference. A lower value of rank means higher preference. The MIG attempts to create its VMs using machine types with a higher preference. If these preferred machine types are not available, the MIG uses a machine type with a lower preference.

  "instanceFlexibilityPolicy": {
    "instanceSelections": {
      "most-preferred": {
        "rank": 1,
        "machineTypes": ["c3-standard-8","c2-standard-8"]
      },
      "least-preferred": {
        "rank": 2,
        "machineTypes": ["n2-standard-16","n1-standard-16"]
      }
    }
  }
  

Your application needs to be ready to run on any of the listed machine types as the availability of hardware changes over time and specific hardware is not assured unless you use reservations.

Instance selections can also override the instance properties like minCpuPlatform and disks. For more information, see the following section about How instance selection overrides work.

How minCpuPlatform and disks overrides work

Preview — Minimum CPU platform and disk overrides

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

In an instance selection, you can specify additional instance properties, such as minimum CPU platform (minCpuPlatform) and disks (disks), to override those defined in the instance template. When a MIG selects a machine type from an instance selection, the MIG creates a VM by using the instance properties specified in that instance selection. If you don't specify any of these properties for an instance selection, the MIG will use the corresponding properties from the template for that selection.

Overriding these properties at the instance selection level provides more control and flexibility, especially when using different hardware in a MIG.

• Override minimum CPU platform when mixing machine types that run on different CPU platforms. You can specify a minimum CPU platform for instances created from the machine types within a selection. This override lets you mix different machine types but selectively apply minimum CPU platforms. Alternatively, you can configure the instance selection to automatically select a CPU platform by setting the minCpuPlatform field to AUTOMATIC. This automatic selection overrides the instance template's minimum CPU platform setting and allows the MIG to use any compatible CPU platform for the selected machine types.

  For example, you can mix machine types from N1 and N2 machine series in a selection, while ensuring that N1 machines run on Intel Skylake to achieve performance gains.

• Override disks when mixing machine types that have different architectures. You can specify a custom disk configuration to override the boot disk or additional disks specified in the instance properties, or to attach a specific storage. This override lets you mix machine types that have different architectures or those that don't share disk technologies, and specify the appropriate boot images or disks required for each machine type.

  For example, you can mix C4 (x86 architecture) and C4A (Arm architecture) machine types. As x86 and Arm machines require different boot images, you can use separate instance selections to override the boot disk for each architecture.

  When you specify the disks for an instance selection, the MIG merges the disks in the instance template and in the instance selection by their deviceName keys. The following diagram shows an example.

  [Disks from instance template and instance selection merges]

To know about the limitations when using instance selection overrides, see the Limitations section in this document.

Example for minimum CPU platform and disk overrides

Consider an instance template with the following configuration:

  {
    ...
      "disks": [
        {
          "autoDelete": true,
          "deviceName": "boot",
          "boot": true,
          "diskEncryptionKey": {},
          "initializeParams": {
            "sourceImage": "projects/debian-cloud/global/images/family/debian-12",
            "diskSizeGb": "10",
            "diskType": "pd-balanced"
          }
        }
      ],
      "machineType": "n2-standard-2",
      ...
    }
  

The following example of an instance flexibility policy overrides the template configuration by specifying a minimum CPU platform and disk configuration.

  "instanceFlexibilityPolicy": {
    "instanceSelections": {
      "first-preference": {
        "rank": 1,
        "machineTypes": ["n4-standard-4 "],
        "disks": [
          {
            "autoDelete": true,
            "deviceName": "boot",
            "boot": true,
            "diskEncryptionKey": {},
            "initializeParams": {
              "sourceImage": "projects/debian-cloud/global/images/family/debian-12",
              "diskSizeGb": "10",
              "diskType": "hyperdisk-balanced"
            }
          }
        ]
      },
      "second-preference": {
        "rank": 2,
        "machineTypes": ["n2-standard-4"],
        "minCpuPlatform": "Intel Ice Lake",
        "disks": [
          {
            "autoDelete": true,
            "deviceName": "boot",
            "boot": true,
            "initializeParams": {
              "sourceImage": "projects/debian-cloud/global/images/family/debian-12",
              "diskSizeGb": "10",
              "diskType": "pd-ssd"
            }
          }
        ]
      },
      "third-preference": {
        "rank": 3,
        "machineTypes": ["n1-standard-4"],
        "minCpuPlatform": "Intel Skylake",
      }
    }
  }
  

How a MIG selects machine types

When you configure multiple machine types, the MIG selects a machine type based on current resource availability and prioritizes unused reservations. If you've configured ranks, the MIG first considers your preference order. Within a ranked instance selection, the MIG then checks resource availability, followed by reservations.

The zones in the MIG's region might also impact the machine type selection because resource availability varies across zones. For more information, see the following section about how a MIG selects zones.

How a MIG selects zones

You can configure instance flexibility in a regional MIG that has a BALANCED, ANY, or ANY_SINGLE_ZONE target distribution shape. Instance flexibility doesn't support EVEN target distribution shape.

With instance flexibility, when a regional MIG creates a VM, the MIG selects a zone depending on the selected target distribution shape as follows:

• BALANCED: The MIG selects a zone with available resources that can balance the VM distribution across zones as evenly as possible. Within a selected zone, the MIG prioritizes machine types based on ranks, if configured, and available reservations.

• ANY: The MIG selects a zone with available resources and reservations. If ranks are configured, then the MIG prioritizes the zone in which a machine type based on ranks is available.

• ANY_SINGLE_ZONE: The MIG selects a zone which has the most machine types based on the ranks, if configured, and where the MIG can create most VMs out of unused reservations.

For more information about how a target distribution shape works, see Regional MIG target distribution shape.

Observability and monitoring

Because a MIG with instance flexibility dynamically selects machine types and zones based on real-time capacity and obtainability, you can observe this runtime behavior using a Cloud Monitoring dashboard.

Google Cloud provides a preconfigured dashboard, GCE MIG Instance Distribution Monitoring, which visualizes the instance distribution by machine type and zone in real-time. This observability helps you confirm high availability, track fallback machine type allocations, and diagnose capacity distribution in your MIG.

To learn how to access the monitoring dashboard, see Monitor instance distribution in MIGs.

Instance flexibility and reservations

You can use instance flexibility with reservations. Based on the reservation affinity property you set in the instance template, a MIG uses reservations as follows:

• ANY_RESERVATION (recommended for instance flexibility): When using any reservation, the MIG selects the machine type with unused reservation first. If you want to change the order in which the MIG selects machine types, you can configure multiple instance selections and specify your preference using ranks. The MIG then first follows the ranks and then considers available reservations for the machine types within a rank.

• SPECIFIC: When using a specific reservation, the MIG selects the machine type that matches the reservation properties. If your instance selections don't include the machine type that matches the specific reservation properties, then instance creation fails.

For more information about reservations, see About reservations.

Instance flexibility and Spot VMs

If your MIG with instance flexibility uses Spot VMs, then the MIG automatically selects a machine type with the lowest observed preemption rate to help ensure the least disruption for your workload. If you want to override this behavior, then complete the following steps:

• View resource availability. Before you create a MIG with Spot VMs, you can view resource availability in the region or zone where you want to create your MIG. This action helps reduce your chances of encountering resource availability errors.

  For instructions, see View the availability of Spot VMs.

• Configure multiple instance selections. You can specify the machine type and zones in which to create Spot VMs by using ranks. The MIG then first follows your preference order, and then considers the reliability of Spot VMs within an instance selection.

  For instructions, see Create a MIG with multiple machine types and preferences.

For more information about using Spot VMs in a MIG, see Spot VMs in a managed instance group.

Instance flexibility and VM repairs

By default, if a VM in a MIG fails, the MIG repairs the VM by recreating it using the same configuration, including the machine type. If you configure instance flexibility and enable updates on repair, then you can improve the chances to obtain resources if a stockout occurs. When a MIG attempts to recreate a VM, if the original machine type of the VM is unavailable, then you will get a stockout error. With updates on repair enabled, the MIG then uses your specified instance selections and current availability to choose another machine type for the VM creation. Enabling updates on repair also helps to ensure that the recreated VM uses the latest instance flexibility configuration.

For more information about repairs, see About repairing VMs for high availability.

Instance flexibility and autoscaling

When you configure autoscaling in a MIG that has multiple machine types, the following happens:

• When an autoscaler scales out a MIG, the MIG creates VMs by selecting machine types in the same manner as in a non-autoscaled MIG. For more information, see the How a MIG selects machine types in this document.

• When an autoscaler scales in a MIG, the MIG selects the VMs to delete as follows:

  1. The MIG first prioritizes deleting VMs that best preserve the target distribution shape.
  2. If you configured ranks, then among the prioritized VMs, the MIG prioritizes deleting VMs that use lower-ranked machine types.
  3. Finally, the MIG checks the autoscaler recommendation to select the VMs to delete.

Autoscaling works effectively when your MIG has machine types of similar performancefor example, machines with the same vCPU count. Otherwise, when the MIG scales out or scales in, the autoscaler might attempt to add or remove the highest-performing machine type more frequently. This behavior can happen regardless of the rank that you configured for that machine type.

For more information about autoscaling, see Autoscaling groups of instances.

Use cases

The following sections describe some use cases of instance flexibility in MIGs.

Improve obtainability

You can configure MIG instance flexibility to improve resource obtainability:

• Machine types: The basic setup is to configure multiple machine types within a single rank. This single rank gives them equal preference, letting the MIG select any of them based on resource availability. However, if you have preferences, then you can assign different ranks to specify a preference order for the machine types.

• Location: You can further increase the chance of obtaining the resources by choosing all zones when creating your MIG and using the ANY target distribution shape. The ANY shape allows the MIG to search for capacity across all zones.

• Spot VMs and preemption: When you use Spot VMs, to maximize the chances of obtaining the VMs and to minimize their preemptions, Google recommends that you include as many compatible machine types as possible within the same rank. This configuration lets MIG choose machine types with the lowest preemption rates to minimize disruption for your workload. If you configure machine types with different ranks, then MIG prioritizes machine types based on rank instead of preemption rates, which might result in selecting a machine type with a higher preemption rate.

Optimize costs

A MIG can automatically provision the most cost-effective instances for your infrastructure when you configure the instance flexibility settings of your MIG as follows:

• Machine types: Select the machine types that offer the preferred cost-performance ratio for your workload. You don't need to specify all the available sizes of each machine type; specify only the sizes that are most cost-optimal for your needs.

• Rank: Assign ranks to instance selections according to your cost preferences, with the lowest-cost machine type as most preferred. This helps to ensure that you obtain the lowest-cost machine type based on availability.

• Location: When creating the MIG, select all zones and the ANY target distribution shape. This increases the chances of obtaining the lowest-cost machine types in zones where they are available.

Based on your instance flexibility settings, the MIG automatically provisions the best possible combination of available machine types to meet your cost criteria and the total capacity requested. This ensures the lowest possible cost at a given time.

Maximize performance

You can configure MIG instance flexibility to maximize performance. To do so, select machine families and sizes that best serve your workload. Then rank the machine types based on performance, from highest to lowest, using the most preferred size within each family. Based on the ranking and the machine types, the MIG selects the highest-performing machines from all available resources.

Because newer machine generations typically offer higher per-core performance, you need not limit the machine selection to use the same size (vCPU count) across different families to achieve equivalent performance. For example, when selecting across machine families, you can mix different CPU counts, such as mixing n1-standard-32 with n2-standard-16.

Additionally, specifying a minCpuPlatform for the instance selection can ensure that the MIG selects a high-performing CPU platform.

Seamless transition to latest machine types

You can configure MIG instance flexibility to prioritize the adoption of latest generation machine types while using older generation machine types as a fallback for capacity. To do so, create an instance selection containing the latest generation machine types and assign a lower rank (higher preference). If these machine types require different disk configurations, use the disk overrides within this instance selection to define the specific disk properties. Create a second instance selection containing the older generation machine types to serve as a fallback. Assign a higher rank (lower preference) to this instance selection.

When creating the MIG, select all zones in the region and the ANY target distribution shape. This configuration ensures that the MIG first searches for the latest generation machines across all zones in the region. The MIG will only provision the older generation machines if the MIG can't find enough latest generation capacity, ensuring that your requested total capacity is always met while maximizing the share of the newest hardware.

Limitations

Instance flexibility has the following limitations:

• Zonal MIGs don't support instance flexibility. You can use a regional MIG with a single zone instead.

• Flex-start VMs don't support instance flexibility.

• Instance flexibility doesn't support the following MIG features:

  • Regional MIGs with target distribution shape set to EVEN
  • Automatic updating of VM configuration and canary updates (two instance templates)
  • Suspending or stopping VMs
  • Resize requests that create GPU VMs all at once

• When overriding minCpuPlatform and disks, the following limitations also apply:

  • You must enable updates on repairs.

  • When configuring the minCpuPlatform or disks for an existing instance selection, you must also include the machineTypes field. Without the machineTypes field, the operation fails with an error.

  • You can't apply stateful policy or per-instance configuration to disks overridden through an instance selection.

  • You can override local SSDs only for those machine types that allow you to choose a number of local SSD disks. You can't override local SSD for -lssd variants of machine types, which automatically attach local SSD disks.

  • You must specify unique machine type across all instance selections. For example, you cannot specify n1-standard-2 in one selection with a Skylake CPU and also in another selection with a Broadwell CPU.

• To help ensure effective load balancing or autoscaling in a MIG that has instance flexibility, use machine types with similar performancefor example, the same vCPU count. However, "similar performance" is not limited to using the same size across different families; because newer machine generations typically have higher performance per vCPU, you can mix different vCPU counts (for example, mixing n1-standard-32 with n2-standard-28).

  • In Cloud Load Balancing, similar performance machine types can prevent over-utilization of smaller hardware and under-utilization of faster machines.

  • In autoscaling, similar performance machine types can prevent addition or deletion of the highest-performing machine type.

What's next?

• Learn more about different machine types.
• Add instance flexibility to a MIG.
• View instance flexibility in a MIG.
• Monitor instance distribution in MIGs.