[ Web Proxy ]
URL:
Viewing: https://developers.google.com/analytics/bigquery/backfill-traffic-source-data [Back]  [Original]
Backfill Google Ads traffic source data as soon as possible (GA 360)  |  Google Analytics  |  Google for Developers Skip to main content
Google Analytics [Google Analytics]
Overview Developer documentation Related products Blog Community
Search:
Guides Reference Libraries & samples Support
Google Analytics [Google Analytics]
Send feedback

Backfill Google Ads traffic source data as soon as possible (GA 360) Stay organized with collections Save and categorize content based on your preferences.

outlined_flag

If you use BigQuery Fresh Daily exports, you might see Data Not Available for some traffic_source fields. This guide helps you automatically backfill most of the missing traffic source data in existing exports as soon as it's available, typically by 5 AM each day.

Here are the steps to automate the backfill:

  1. Listen for the daily completeness signal from BigQuery.
  2. Identify the events with missing traffic source data in your BigQuery export.
  3. Query the complete data for those events from Google Ads.
  4. Join the complete event data with your BigQuery export.

Create a Pub/Sub topic

  1. Open Pub/Sub in the left navigation menu of the Google Cloud console. If you don't see Pub/Sub, search for it in the Google Cloud console search bar: Search for Pub/Sub [Search for Pub/Sub]
  2. Click + CREATE TOPIC in the Topics tab: Create topic button [Create topic button]
  3. Enter a name in the Topic ID field.
  4. Select Add a default subscription, leave the other options blank: create subscription button [create subscription button]
  5. Click Create.

Create a Log Router sink

  1. Open Log router in the Google Cloud console: Log router button [Log router button]
  2. Click Create sink: Create sink button [Create sink button]
  3. Enter a name and description for your sink, then click Next.
  4. Choose Cloud Pub/Sub topic as the sink service.
  5. Choose the topic you created, then click Next.

  6. Enter the following code in Build inclusion filter:

    logName="projects/YOUR-PROJECT-ID/logs/analyticsdata.googleapis.com%2Ffresh_bigquery_export_status"

    Replace YOUR-PROJECT-ID with the ID for your Google Cloud console project.

  7. Click Next, then click Create sink. You don't need to filter out any logs.

  8. Verify the sink is now listed under Log Router Sinks.

Join the missing data

Use a Cloud Run function to automatically execute the code to backfill traffic source data when Pub/Sub detects the completeness signal:

  1. Open Cloud Run functions: Cloud Run functions button [Cloud Run functions button]
  2. Click CREATE FUNCTION: Create function button [Create function button]
  3. Choose Cloud Run function for the Environment.
  4. Enter a name for your function.
  5. Choose Cloud Pub/Sub as the Trigger type, and the topic you created as the Cloud Pub/Sub topic.
  6. Click Next, then enter your code to join the Google Ads attribution data with your BigQuery export in the box.
Previous
Access Google Analytics custom reports from BigQuery
Send feedback

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2024-12-19 UTC.

Need to tell us more? [[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2024-12-19 UTC."],[],["The core process addresses missing traffic source data in BigQuery Fresh Daily exports. It involves listening for a daily completeness signal via Pub/Sub, identifying incomplete events, and then querying Google Ads for the complete data. This complete data is then joined with the existing BigQuery export. To achieve this, a Pub/Sub topic and a Log Router sink are created to monitor the completeness signal. Finally, a Cloud Run function is employed to execute the data backfill using a custom code when triggered by the completeness signal.\n"]]
Web Proxy Viewer  |  New URL  |  Original Page