[ Web Proxy ]
URL:
Viewing: https://docs.cloud.google.com/bigquery/docs/shopify-transfer [Back]  [Original]
Load Shopify data into BigQuery  |  Google Cloud Documentation Skip to main content
Google Cloud Documentation [Google Cloud Documentation]
Technology areas Cross-product tools
Search:
Console
[]
Start free
Overview Guides Reference Samples Resources
Google Cloud Documentation [Google Cloud Documentation]
Send feedback Stay organized with collections Save and categorize content based on your preferences.

Load Shopify data into BigQuery

Preview

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

Note: To get support or provide feedback for this feature, contact dts-preview-support@google.com.

You can load data from Shopify to BigQuery using the BigQuery Data Transfer Service for Shopify connector. With the BigQuery Data Transfer Service, you can schedule recurring transfer jobs that add your latest data from Shopify to BigQuery.

The Shopify connector supports the data transfer of GraphQL-based resources, such as Collections and Orders, from the following resources:

Limitations

Shopify data transfers are subject to following limitations:

Before you begin

The following sections describe the prerequisites that you need to do before you create a Shopify data transfer.

Shopify prerequisites

BigQuery prerequisites

Required BigQuery roles

To get the permissions that you need to create a BigQuery Data Transfer Service data transfer, ask your administrator to grant you the BigQuery Admin (roles/bigquery.admin) IAM role on your project. For more information about granting roles, see Manage access to projects, folders, and organizations.

This predefined role contains the permissions required to create a BigQuery Data Transfer Service data transfer. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to create a BigQuery Data Transfer Service data transfer:

You might also be able to get these permissions with custom roles or other predefined roles.

For more information, see Grant bigquery.admin access.

If you intend to set up transfer run notifications for Pub/Sub, ensure that you have the pubsub.topics.setIamPolicy Identity and Access Management (IAM) permission. Pub/Sub permissions aren't required if you only set up email notifications. For more information, see BigQuery Data Transfer Service run notifications.

Set up a Shopify data transfer

Add Shopify data into BigQuery by setting up a transfer configuration using one of the following options:

Console

  1. Go to the Data transfers page in the Google Cloud console.

    Go to Data transfers

  2. Click Create transfer.

  3. In the Source type section, for Source, choose Shopify.

  4. In the Data source details section, do the following:

    • For Shop name, enter the name of your Shopify shop. For example, if your merchant account ID is storename.myshopify.com, then your shop name is storename.
    • For Client ID and Client Secret, enter the client ID and secret for your Shopify app. For more information, see About client secrets.
    • For Shopify objects to transfer, click Browse:
      • Select any objects to be transferred to the BigQuery destination dataset. You can also manually enter any objects to include in the data transfer in this field.

  5. In the Destination settings section, for Dataset, choose the dataset that you created to store your data.

  6. In the Transfer config name section, for Display name, enter a name for the data transfer.

  7. In the Schedule options section:

    • In the Repeat frequency list, select an option to specify how often this data transfer runs. To specify a custom repeat frequency, select Custom. If you select On-demand, then this transfer runs when you manually trigger the transfer.
    • If applicable, select either Start now or Start at set time, and provide a start date and run time.

  8. Optional: In the Notification options section, do the following:

    • To enable email notifications, click the Email notification toggle. When you enable this option, the transfer administrator receives an email notification when a transfer run fails.
    • To enable Pub/Sub transfer run notifications for this transfer, click the Pub/Sub notifications toggle. You can select your topic name, or you can click Create a topic to create one.

  9. Click Save.

bq

Enter the bq mk command and supply the transfer creation flag --transfer_config:

bq mk
    --transfer_config
    --project_id=PROJECT_ID
    --data_source=DATA_SOURCE
    --display_name=NAME
    --target_dataset=DATASET
    --params='PARAMETERS'

Where:

The following command creates an incremental Shopify data transfer in the default project.

bq mk
    --transfer_config
    --target_dataset=mydataset
    --data_source=shopify
    --display_name='My Transfer'
    --params='{"assets": ["Orders"]
        "connector.shopName": "storename",
        "connector.authentication.accessToken":"sk_test_123456789"}'
When you save the transfer configuration, the Shopify connector automatically triggers a transfer run according to your schedule option. With every transfer run, the Shopify connector transfers all available data from Shopify into BigQuery.

To manually run a data transfer outside of your regular schedule, you can start a backfill run.

Supported tables

The Shopify connector supports the following data objects:

Data type mapping

The following table maps Shopify data types to the corresponding BigQuery data types:

Shopify data type BigQuery data type
String STRING
Int INT64
Decimal NUMERIC
Double FLOAT64
Long BIGNUMERIC
Bool BOOL
Datetime TIMESTAMP

Pricing

There is no cost to transfer Shopify data into BigQuery while this feature is in Preview.

Troubleshoot transfer setup

If you are having issues setting up your data transfer, see Shopify transfer issues.

What's next

Send feedback

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2026-08-12 UTC.

Need to tell us more? [[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2026-08-12 UTC."],[],[]]
Web Proxy Viewer  |  New URL  |  Original Page