Multiple network interfaces

This page provides an overview of multiple network interfaces for Compute Engine instances. Instances with multiple network interfaces are referred to as multi-NIC instances.

An instance network interface connects to a Virtual Private Cloud (VPC) network and can be either a virtual network interface (vNIC) or a Dynamic NIC, where a Dynamic NIC is a child sub-interface of a parent vNIC. An instance's vNICs are backed by one or more physical NICs on the host server that runs the instance. For more information, see Network interface types.

Use cases

Multi-NIC instances are useful in the following scenarios:

• To connect to resources in separate VPC networks: multi-NIC instances can connect to resources located in different VPC networks that aren't connected to each other through VPC Network Peering or NCC.

  • Because each interface of a multi-NIC instance is in a separate VPC network, you can use each interface for a unique purpose. For example, you can use some interfaces to route packets between VPC networks that carry production traffic and another interface for management or configuration purposes.

  • Within the guest OS of each multi-NIC instance, you must configure route policies and local route tables.

• To route packets between VPC networks: multi-NIC instances can be used as next hops for routes to connect two or more VPC networks.

  • Software running within the guest OS of a multi-NIC instance can perform packet inspection, network address translation (NAT), or another network security function.

  • When connecting VPC networks using multi-NIC instances, it's a best practice to configure two or more multi-NIC instances, using them as backends for an internal passthrough Network Load Balancer in each VPC network. For more information, see Use cases in the Internal passthrough Network Load Balancers as next hops documentation.

• To increase the total network bandwidth of an instance: multi-NIC instances support increased bandwidth as follows:

  • For some machine types in Google Cloud, instances are backed by multiple physical NICs on the host server. Creating an instance with multiple vNICs where each vNIC uses a different physical NIC increases the total network bandwidth of the instance. For more information, see Physical NICs and Multiple network interfaces in the same VPC network.

  • For most machine types in Google Cloud, instances are backed by a single physical NIC on the host server, and each vNIC uses the same physical NIC. When vNICs share a physical NIC, adding vNICs doesn't increase network bandwidth.

You can also use multi-NIC instances with Private Service Connect interfaces to connect service producer and consumer networks in different projects.

Network interface types

See the following sections for details about network interface types in Google Cloud.

Instance network interfaces

Compute Engine instances connect to VPC networks by using network interfaces that you configure, which can be either vNICs or Dynamic NICs. These instance network interfaces correspond to the instances.networkInterfaces[] resource and property in the Compute Engine API.

For more information about each type of instance network interface, see the following:

• Virtual network interface (vNIC): a software-defined network interface exposed to your instance's guest OS that lets your instance connect to VPC networks. Each instance must have at least one vNIC.

  • You can only configure vNICs when creating an instance.

  • The name of a vNIC assigned by Google Cloud has the following format: nicN, such as nic0. The corresponding guest OS-assigned name depends on the operating system, such as ens4 for Ubuntu.

  • A vNIC can be one of the following types, depending on the associated VPC network. Each type corresponds to a specific network driver in the guest OS:

    • Regular VPC networks accept attachments from the following vNIC types: GVNIC, IDPF, and VIRTIO_NET.

    • VPC networks with an RDMA network profile accept attachments from vNIC types IRDMA or MRDMA, which are also called RDMA vNICs.

    For information about which machine types support which vNIC types, see Machine series comparison.

• Dynamic NIC: a child of a parent vNIC that is implemented as a sub-interface in the guest OS of your instance.

  • You can configure Dynamic NICs when you create an instance, or add them later.

  • The name of a Dynamic NIC assigned by Google Cloud has the following format: nicN.VLAN_ID, such as nic0.2. The corresponding guest OS-assigned name depends on the operating system. For example, if you use the guest agent to manage Dynamic NICs in Ubuntu, the name might be a-gcp.ens4.2.

  • Dynamic NICs aren't supported in VPC networks with an RDMA network profile.

  For more information, see Dynamic NICs.

Physical NICs

A physical NIC is the hardware component directly installed on the host server where an instance runs. Depending on the machine type of your instance, the instance's vNICs map to one or more physical NICs on a host server as follows:

• Instances backed by one physical NIC: For most machine types, instances run on a host server with a single physical NIC. For a given instance, all of the instance's vNICs map to the single physical NIC.

• Instances backed by multiple physical NICs: For a limited set of machine types, such as C4N machine types and GPU machine types, instances run on a host server with multiple physical NICs. For a given instance, Compute Engine maps the instance's vNICs to different physical NICs.

  • Instances backed by multiple physical NICs support use cases such as increased network bandwidth and GPU-to-GPU communication over an RoCE VPC network. For more information, see Network support for C4N instances and Networking and GPU machines.

  • When you create the instance, Compute Engine maps each vNIC to a unique physical NIC, up to the total physical NIC count of the instance's machine type.

  • To understand support for multiple physical NICs, view the documentation for a given machine series. If applicable, the table of machine types includes a column titled Physical NIC count. For example, see C4N machine types.

Additionally, for most machine types, instances share a host server with other instances. Instances that share a host server also share its physical NIC or set of physical NICs. However, for some machine types, such as bare metal machine types or H3 and H4D machine types, a single instance occupies an entire host server.

Specifications

The following specifications apply to instances with multiple network interfaces:

• Instances and network interfaces: every instance has a nic0 interface. The maximum number of network interfaces varies depending on the instance's machine type.

  • Each interface has an associated stack type, which determines the supported subnet stack types and IP address versions. For more information, see Stack type and IP addresses.

• Network and subnetwork requirements: each network interface must use a unique subnet in a VPC network.

  • Attaching network interfaces to subnets in different VPC networks is supported.

  • Attaching network interfaces to different subnets in the same VPC network is supported in the following scenarios:

    • For regular VPC networks, multiple network interfaces can use the same VPC network as the nic0 interface. For more information, see Multiple network interfaces in the same VPC network (Preview).

  • For VPC networks created with an RDMA network profile, multiple RDMA NICs can use the same VPC network. For more information, see the table for each network profile in Supported and unsupported features.

  • A VPC network and subnet must exist before you can create an instance whose network interface uses the network and subnet. For more information about creating networks and subnets, see Create and manage VPC networks.

• Project of instance and subnets: for multi-NIC instances in standalone projects, each network interface must use a subnet located in the same project as the instance.

  • For instances in Shared VPC host or service projects, see Shared VPC .

  • Private Service Connect interfaces provide a way for a multi-NIC instance to have network interfaces in subnets in different projects. For more information, see About network attachments.

• IP forwarding, MTU, and routing considerations: multi-NIC instances require careful planning for the following instance and interface specific configuration options:

  • The IP forwarding option is configurable on a per instance basis, applying to all network interfaces. For more information, see Enable IP forwarding for instances.

  • Each network interface can use a unique maximum transmission unit (MTU), matching the MTU of the associated VPC network. For more information, see Maximum transmission unit.

  • Each instance receives a default route using DHCP Option 121, as defined by RFC 3442. The default route is associated with nic0. Unless manually configured otherwise, any traffic leaving an instance for any destination other than a directly connected subnet will leave the instance using the default route on nic0.

    On Linux systems, you can configure custom rules and routes within the guest OS using the /etc/iproute2/rt_tables file and the ip rule and ip route commands. For more information, consult the guest OS documentation. For an example, see the following tutorial: Configure routing for an additional interface.

Multiple network interfaces in the same VPC network

Preview

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

You can attach additional vNICs of an instance to the same VPC network as the nic0 vNIC of the instance.

You might want to attach multiple vNICs to the same VPC network to increase the bandwidth of an instance in the VPC network:

• For some machine types in Google Cloud, instances are backed by multiple physical NICs on the host server as described in Physical NICs.

  Creating an instance with multiple vNICs where each vNIC uses a different physical NIC increases the network bandwidth of the instance for the VPC network that the vNICs are attached to.

• For most machine types in Google Cloud, instances are backed by one physical NIC. For these machine types, we recommend that you attach each vNIC to a separate VPC network.

Limitations for multiple network interfaces in the same VPC network

See the following limitations for attaching multiple network interfaces of an instance to the same VPC network:

• The additional network interfaces must be attached to the same network as the nic0 interface.
• To get additional network bandwidth for the non-nic0 interfaces, you must use a machine type that is backed by multiple physical NICs.
• The following are not supported:
  • Dynamic NICs.
  • Private Service Connect interfaces.
  • For Cloud Load Balancing, instances with multiple vNICs attached to the same VPC network can be added to instance group or zonal network endpoint group (NEG) backends. However, from the set of vNICs attached to the same VPC network, traffic can be distributed to only nic0.
  • For static routes, you can't send packets to the non-nic0 interfaces by using a next hop instance specified by name and zone (next-hop-instance). Instead, use a next hop instance specified by address (next-hop-address). For more information, see Next hops and features.
  • Network interface bonding with link aggregation.

Dynamic NICs

If your use case requires any of the following, use Dynamic NICs. Ensure that you also review the properties and limitations of Dynamic NICs.

• You need to add or remove network interfaces to or from existing instances. Adding or removing Dynamic NICs doesn't require restarting or recreating the instance.

• You need more network interfaces. The maximum number of vNICs for most machine types in Google Cloud is 10; however, you can configure up to 16 total interfaces by using Dynamic NICs. For more information, see Maximum number of network interfaces.

• You need to configure multi-NIC Compute Engine bare metal instances, which only have one vNIC.

Properties of Dynamic NICs

See the following information about the properties of Dynamic NICs:

• Dynamic NICs are VLAN interfaces that use the IEEE 802.1Q standard packet format. See the following considerations:

  • The VLAN ID of a Dynamic NIC must be an integer from 2 to 255.
  • The VLAN ID of a Dynamic NIC must be unique within a parent vNIC. However, Dynamic NICs that belong to different parent vNICs can use the same VLAN ID.

• Google Cloud uses the following format for the name of a Dynamic NIC: nicNUMBER.VLAN_ID, where

  • nicNUMBER is the name of the parent vNIC, such as nic0.
  • VLAN_ID is the VLAN ID that you set, such as 4.

  An example Dynamic NIC name is nic0.4.

• Creating an instance with Dynamic NICs or adding Dynamic NICs to an existing instance requires additional steps to install and manage the corresponding VLAN interfaces in the guest OS. You can use one of the following methods:

  • Configure automatic management of Dynamic NICs by using the guest agent.

  • Configure the guest OS manually.

  For more information, see Configure the guest OS for Dynamic NICs.

• Dynamic NICs share the bandwidth of their parent vNIC. To prevent any of the network interfaces from consuming all of the bandwidth, you must create an application-specific traffic policy in the guest OS to prioritize or distribute traffic, such as by using Linux Traffic Control (TC).

• Dynamic NICs share the same receive and transmit queues as their parent vNIC. To configure a network interface to use different receive and transmit queues, use vNICs instead of Dynamic NICs.

• The stack type of a Dynamic NIC can be the same as or different than its parent vNIC. For example, you can create IPv6-only and dual-stack Dynamic NICs under an IPv4-only parent vNIC.

Limitations of Dynamic NICs

See the following limitations of Dynamic NICs:

• You can't modify the following properties of a Dynamic NIC after it is created:

  • The parent vNIC to which the Dynamic NIC belongs.
  • The VLAN ID of the Dynamic NIC.

• Attaching multiple network interfaces of an instance to the same VPC network is not supported for Dynamic NICs.

• A Dynamic NIC can't be deleted if the Dynamic NIC is a load-balanced network interface.

• Dynamic NICs don't support the following:

  • Advanced network DDoS protection and network edge security policies for Google Cloud Armor
  • Configuring IP addresses by using per-instance configurations for MIGs.
  • Features that rely on packet intercept, such as firewall endpoints
  • Compute Engine Windows drivers

• Dynamic NICs aren't supported for use with GPU instances. For more information, see Creation errors and decreased performance when using Dynamic NICs with GPU instances.

• Dynamic NICs require that you manually create local routes in certain scenarios. For more information, see Dropped packets when using Dynamic NICs with alias IP ranges, protocol forwarding, or Passthrough Network Load Balancers.

Stack types and IP addresses

When you create a network interface, you specify one of the following interface stack types:

• IPv4-only
• Dual-stack
• IPv6-only

A VM instance can have network interfaces that have different stack types.

The following table describes supported subnet stack types and IP address details for each interface stack type:

Changing network interface stack type

You can change the stack type of a network interface as follows:

• You can convert an IPv4-only interface to dual-stack if the interface's subnet is a dual-stack subnet or if you stop the instance and assign the interface to a dual-stack subnet.

• You can convert a dual-stack interface to IPv4-only.

You can't change the stack type of an IPv6-only interface. IPv6-only interfaces are only supported when creating instances.

IPv4 address details

Each IPv4-only or dual-stack network interface receives a primary internal IPv4 address. Each interface optionally supports alias IP ranges and an external IPv4 address. The following are the IPv4 specifications and requirements:

• Primary internal IPv4 address: Compute Engine assigns the network interface a primary internal IPv4 address from the primary IPv4 address range of the interface's subnet. The primary internal IPv4 address is allocated by DHCP.

  • You can control which primary internal IPv4 address is assigned by configuring a static internal IPv4 address or by specifying a custom ephemeral internal IPv4 address.

  • Within a VPC network, the primary internal IPv4 address of each VM network interface is unique.

• Alias IP ranges: optionally, you can assign the interface one or more alias IP ranges. Each alias IP range can come from either the primary IPv4 address range or a secondary IPv4 address range of the interface's subnet.

  • Within a VPC network, each interface's alias IP range must be unique.

• External IPv4 address: optionally, you can assign the interface an ephemeral or reserved external IPv4 address. Google Cloud ensures the uniqueness of each external IPv4 address.

IPv6 address details

Compute Engine assigns each dual-stack or IPv6-only network interface a /96 IPv6 address range from the /64 IPv6 address range of the interface's subnet:

• Whether the /96 IPv6 address range is internal or external depends on the IPv6 access type of the interface's subnet. Google Cloud ensures the uniqueness of each internal and external IPv6 address range. For more information, see IPv6 specifications.

  • If an instance needs both an internal IPv6 address range and an external IPv6 address range: you must configure two dual-stack interfaces, two IPv6-only interfaces, or one dual-stack interface and one IPv6-only interface. The subnet used by one interface must have an external IPv6 address range, and the subnet used by the other interface must have an internal IPv6 address range.

• The first IPv6 address (/128) is configured on the interface by DHCP. For more information, see IPv6 address assignment.

• You can control which /96 IPv6 address range is assigned by configuring a static internal or external IPv6 address range. For internal IPv6 addresses, you can specify a custom ephemeral internal IPv6 address.

If you are connecting an instance to multiple networks by using IPv6 addresses, install google-guest-agent version 20220603.00 or later. For more information, see I can't connect to a secondary interface's IPv6 address.

Maximum number of network interfaces

For most machine types, the maximum number of network interfaces that you can attach to an instance scales with the number of vCPUs as described in the following tables.

The following are machine-specific exceptions:

• Compute Engine bare metal instances support a single vNIC.

• The maximum number of vNICs is different for some accelerator optimized machine types, such as A3, A4, and A4X. For more information, see Accelerator-optimized machine family.

Example distributions of Dynamic NICs

You don't have to distribute Dynamic NICs evenly across vNICs. However, you might want an even distribution because Dynamic NICs share the bandwidth of their parent vNIC.

An instance must have at least one vNIC. For example, an instance that has 2 vCPUs can have one of the following configurations:

• 1 vNIC
• 2 vNICs
• 1 vNIC and 1 Dynamic NIC

The following tables provide example configurations that evenly distribute Dynamic NICs across vNICs while using the maximum number of network interfaces for a given number of vCPU.

Product interactions

This section describes interactions between multi-NIC instances and other products and features in Google Cloud.

Shared VPC

Except for Private Service Connect interfaces, the subnet and project relationship of a multi-NIC instance in a Shared VPC host or service project is as follows:

• Each network interface of a multi-NIC instance located in a Shared VPC host project must use a subnet of a Shared VPC network in the host project.

• Each network interface of a multi-NIC instance located in a Shared VPC service project can use either of the following:

  • A subnet of a VPC network in the service project.
  • A subnet of a Shared VPC network in the host project.

For more information about Shared VPC, see:

• Shared VPC overview
• Provision Shared VPC

Compute Engine internal DNS

Compute Engine creates internal DNS name A and PTR records only for the primary internal IPv4 address of the nic0 network interface of an instance. Compute Engine doesn't create internal DNS records for any IPv4 or IPv6 address associated with a network interface different from nic0.

For more information, see Compute Engine internal DNS.

Static routes

Static routes can be scoped to specific instances by using network tags. When a network tag is associated with an instance, the tag applies to all network interfaces of the instance. Consequently, adding a network tag to or removing a network tag from an instance might change which static routes apply to any of the instance's network interfaces.

Load balancers

Instance group backends and zonal NEG backends each have an associated VPC network as follows:

• For managed instance groups (MIGs), the VPC network for the instance group is the VPC network assigned to the nic0 interface in the instance template.

• For unmanaged instance groups, the VPC network for the instance group is the VPC network used by the nic0 network interface of the first instance that you add to the unmanaged instance group.

The following table shows which backends support distributing connections or requests to any network interface.

1Throughout the table, Any NIC doesn't apply when multiple network interfaces of an instance are in the same VPC network (Preview). For more information, see Limitations for multiple network interfaces in the same VPC network.

Target pool-based Regional external passthrough Network Load Balancers don't use instance groups or NEGs and only support load balancing to nic0 network interfaces.

Firewall rules

The set of firewall rules—from hierarchical firewall policies, global network firewall policies, regional network firewall policies, and VPC firewall rules—are unique to each network interface. Ensure that each network has appropriate firewall rules to allow the traffic that you want to allow to and from a multi-NIC instance. To determine which firewall rules apply to a network interface, and the source for each rule, see Get effective firewall rules for a VM interface.

Firewall rules can be scoped to specific VM instances by using network tags or secure tags, both of which apply to all network interfaces of an instance. For more information, see Comparison of secure tags and network tags.

Known issues

This section describes known issues related to using multiple network interfaces in Google Cloud.

Creation errors and decreased performance when using Dynamic NICs with GPU instances

Dynamic NICs aren't supported for use with GPU instances. If you create a GPU instance with Dynamic NICs, or add Dynamic NICs to an existing GPU instance, the following issues might occur:

• The operation fails with an error such as the following:

  Internal error. Please try again or contact Google Support. (Code: 'CODE')

• The operation succeeds, but the instance experiences decreased performance, such as significantly lower network bandwidth.

These issues occur because the Dynamic NIC configuration leads to errors when Compute Engine attempts to distribute the instance's vNICs across physical NICs on the host server.

Dropped packets when using Dynamic NICs with alias IP ranges, protocol forwarding, or Passthrough Network Load Balancers

The guest agent automatically add local routes in the following scenarios for vNICs, but not for Dynamic NICs:

• When you configure an alias IP range, the guest agent creates a local route for the alias IP range.
• When you create a target instance that references a compute instance for protocol forwarding, the guest agent creates a local route for the associated forwarding rule IP address.
• When you add a backend to a Passthrough Network Load Balancer, the guest agent creates a local route for the associated forwarding rule IP address.

Because the local routes aren't added for Dynamic NICs, the Dynamic NIC might experience dropped packets.

To resolve this issue, add the IP addresses manually as follows:

1. Connect to the instance by using SSH.

2. If you are configuring an alias IP range, do the following. Otherwise, you can skip this step.

   1. In /etc/default/instance_configs.cfg, ensure that the ip_aliases setting is set to true.

   2. If the ip_aliases setting is set to false, modify the file to change it to true and then restart the guest agent:

      systemctl restart google-guest-agent
      

3. Configure a local route for the alias IP range or the forwarding rule IP address by using the following command:

   ip route add to local IP_ADDRESS dev DYNAMIC_NIC_DEVICE_NAME proto 66
   

   Replace the following:

   • IP_ADDRESS: the alias IP range or forwarding rule IP address that you want to add a local route for.
   • DYNAMIC_NIC_DEVICE_NAME: the device name of the Dynamic NIC that you want to add a local route for. For example, a-gcp.ens4.3.

Issues with installation and management of Dynamic NICs in guest agent versions 20250901.00 to 20251120.01

If you configure automatic management of Dynamic NICs and your instance is running the guest agent at a version from 20250901.00 to 20251120.01, you might encounter the following issues:

• The guest agent fails to install and manage Dynamic NICs in the guest OS of your instance.

  You might receive an error that includes Cannot find device when running commands in the guest OS that reference Dynamic NICs.

• Deleting multiple Dynamic NICs causes the metadata server to become inaccessible.

Root cause

Starting with version 20250901.00, the guest agent migrated to a new plugin-based architecture to improve modularity. The new architecture didn't initially support the automatic installation and management of Dynamic NICs.

Resolution

To resolve these issues, update your instance to use guest agent version 20251205.00 or later:

1. To update the guest agent to the latest version, see Update the guest environment.
2. To confirm the guest agent version that your instance is running, see View installed packages by operating system version.

If necessary, you can temporarily work around these issues for instances that are running guest agent versions 20250901.00 to 20251120.01 by following the instructions in Backward compatibility to revert to the previous guest agent architecture.

Dynamic NIC operations fail when a compute instance is in multiple instance groups

If you add or delete a Dynamic NIC, and the operation gets stuck in a RUNNING state at 0% progress and eventually fails with an INTERNAL_ERROR, this might be because the compute instance is in multiple instance groups (managed or unmanaged).

To let the operation complete, remove the instance from all but one instance group. You can remove an instance from a managed instance group or remove an instance from an unmanaged instance group.

What's next

• Create VMs with multiple network interfaces