Request TPU Spot VMs

Spot VMs offer unused capacity at significantly discounted rates. While Spot VMs are preemptible at any time, they don't have a maximum runtime duration, unlike preemptible TPUs. To restart a Spot VMs instance or MIG, you must delete and then recreate it.

For more information about Spot VMs, see the Compute Engine documentation about Spot VMs.

Create TPU Spot VMs

You can create TPU Spot VMs as individual instances or as part of a managed instance group (MIG).

Using MIGs for multi-host slices (Recommended)

To create TPU Spot VMs with a MIG, specify the Spot provisioning model in your instance template.

1. Create an instance template:

   gcloud

   gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
       --project=PROJECT_ID \
       --machine-type=MACHINE_TYPE \
       --maintenance-policy=TERMINATE \
       --instance-termination-action=STOP \
       --provisioning-model=SPOT \
       --image-family=IMAGE_FAMILY \
   --image-project=IMAGE_PROJECT
   

   REST

   curl -X POST \
   -H "Authorization: Bearer $(gcloud auth print-access-token)" \
   -H "Content-Type: application/json" \
   -d '{
       "name": "INSTANCE_TEMPLATE_NAME",
       "properties": {
       "machineType": "MACHINE_TYPE",
       "disks": [
           {
           "boot": true,
           "initializeParams": {
               "sourceImage": "projects/IMAGE_PROJECT/global/images/family/IMAGE_FAMILY"
           }
           }
       ],
       "scheduling": {
           "onHostMaintenance": "TERMINATE",
           "provisioningModel": "SPOT",
           "instanceTerminationAction": "STOP"
       },
       "networkInterfaces": [
           {
           "network": "global/networks/default"
           }
       ]
       }
   }' \
   "https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/instanceTemplates"
   

   Replace the following placeholders:

   • PROJECT_ID: The ID of your Google Cloud project.
   • INSTANCE_TEMPLATE_NAME: The name of your instance template.
   • MACHINE_TYPE: The machine type for the TPU VM, for example, ct6e-standard-8t.
   • IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
   • IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.

2. Create a workload policy

   The following command creates a workload policy. This is optional for single-host slices.

   gcloud

   gcloud compute resource-policies create workload-policy WORKLOAD_POLICY_NAME \
   --type=high-throughput \
   --accelerator-topology=TOPOLOGY \
   --region=REGION
   

   REST

   curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "WORKLOAD_POLICY_NAME",
       "workloadPolicy": {
         "type": "HIGH_THROUGHPUT",
         "acceleratorTopology": "TOPOLOGY"
       }
     }' \
     "https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/resourcePolicies"
   

   Replace the following placeholders:

   • PROJECT_ID: The ID of your Google Cloud project.
   • WORKLOAD_POLICY_NAME: The name of your workload policy.
   • TOPOLOGY: The topology of the TPU VMs, for example, 4x4x8. For more information about topology for each version of TPU, see TPU versions.
   • REGION: The region for your workload policy.

3. Create the MIG:

   gcloud

   gcloud compute instance-groups managed create MIG_NAME \
   --size=MIG_SIZE \
   --target-size-policy-mode=bulk \
   --template=INSTANCE_TEMPLATE_URL \
   --zone=ZONE \
   --default-action-on-vm-failure=do-nothing \
   --workload-policy=WORKLOAD_POLICY_URL
   

   REST

   curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json" \
     -d '{
           "name": "MIG_NAME",
           "targetSize": MIG_SIZE,
           "targetSizePolicy": {
             "mode": "BULK"
           },
           "instanceTemplate": "INSTANCE_TEMPLATE_URL",
           "instanceLifecyclePolicy": {
             "defaultActionOnFailure": "DO_NOTHING"
           },
           "resourcePolicies": {
             "workloadPolicy": "WORKLOAD_POLICY_URL"
           }
         }' \
       "https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers"
   

   Replace the following placeholders:

   • PROJECT_ID: The ID of your Google Cloud project.
   • MIG_NAME: The name of your MIG.
   • MIG_SIZE: The size of the MIG.
   • INSTANCE_TEMPLATE_URL: the URL of the instance template that you want to use to create instances in the MIG. The URL can contain either the ID or name of the instance template. Specify one of the following values:
     • For a regional instance template: projects/PROJECT_ID/regions/REGION/instanceTemplates/INSTANCE_TEMPLATE_ID
     • For a global instance template: INSTANCE_TEMPLATE_ID
   • ZONE: The zone for your MIG.
   • WORKLOAD_POLICY_URL: The URL of the workload policy that you want to use to create instances in the MIG. For example: projects/PROJECT_ID/regions/WORKLOAD_POLICY_REGION/resourcePolicies/WORKLOAD_POLICY_NAME.
   Note: For multi-host TPU slices, you must define a workload policy and use the --target-size-policy-mode=bulk flag when creating the MIG to ensure that all VMs are created in the same network and can communicate.

Using single TPU VMs

You can also create single TPU Spot VMs.

gcloud compute instances create TPU_NAME \
  --machine-type=MACHINE_TYPE \
  --image-family=IMAGE_FAMILY \
  --image-project=IMAGE_PROJECT \
  --zone=ZONE \
  --provisioning-model=SPOT \
  --instance-termination-action=DELETE \
  --maintenance-policy=TERMINATE

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "TPU_NAME",
    "machineType": "zones/ZONE/machineTypes/MACHINE_TYPE",
    "disks": [
      {
        "boot": true,
        "initializeParams": {
          "sourceImage": "projects/IMAGE_PROJECT/global/images/family/IMAGE_FAMILY"
        }
      }
    ],
    "scheduling": {
      "onHostMaintenance": "TERMINATE",
      "provisioningModel": "SPOT",
      "instanceTerminationAction": "DELETE"
    },
    "networkInterfaces": [
      {
        "network": "global/networks/default"
      }
    ]
  }' \
  "https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances"

Replace the following placeholders:

• PROJECT_ID: The ID of your Google Cloud project.
• TPU_NAME: A name for your TPU VM.
• MACHINE_TYPE: The machine type for the TPU VM (for example ct6e-standard-8t).
• IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
• IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.
• ZONE: The zone for the TPU VM (for example us-central1-b).

For more information about Spot VMs in Compute Engine, see Spot VMs.

Pricing and quota

Pricing for TPU Spot VMs is significantly lower than for on-demand and reserved TPUs. For more information about pricing, see Cloud TPU pricing.

You need preemptible quota to use TPU Spot VMs. For more information, see Quotas.

What's next

• Learn how to manage TPUs