[FDC] Deploy should provision Cloud SQL async - #9004
Conversation
Sorry, something went wrong.
There was a problem hiding this comment.
Summary of Changes
Hello @fredzqm, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request introduces significant improvements to the Firebase Data Connect deployment process, primarily by making Cloud SQL instance and database provisioning asynchronous. This change allows deployments to proceed more quickly, providing immediate feedback to the user while the Cloud SQL resources are being set up in the background, with data temporarily stored until the instance is ready. Beyond this core enhancement, the PR includes refactorings to simplify IAM user setup, clarifies logging messages during schema migrations, and refines deployment confirmation prompts for better user control and safety.
Highlights
- Asynchronous Cloud SQL Provisioning: Cloud SQL instance and database creation during deployment are now asynchronous, allowing the deployment process to continue without waiting for these long-running operations to complete.
- Simplified IAM User Setup: The setupIAMUsers function has been refactored to no longer require the databaseId parameter, streamlining its usage across the codebase.
- Enhanced Schema Migration Logging: Logging messages related to schema generation and compatibility checks have been updated for improved clarity and user feedback.
- Improved Cloud SQL Connection Management: The logic for ensuring service connectivity to Cloud SQL instances has been made more robust, including better handling of instance/database switching and the introduction of an ephemeral property for PostgreSQL datasources.
- Refined Deployment Prompts: The confirmation prompt for deleting services not present in firebase.json has been adjusted, explicitly setting the force option to false and adding a default false response, preventing accidental deletions.
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review | Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary | Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help | Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
Sorry, something went wrong.
There was a problem hiding this comment.
Code Review
This pull request refactors the Cloud SQL provisioning process to be asynchronous, which is a significant improvement for the user experience during deployment. The code is generally well-structured, with logic broken down into smaller, more manageable functions. I've identified a potential bug in the schema migration logic and an opportunity to refactor some duplicated code for better maintainability. My detailed feedback is in the comments below.
Sorry, something went wrong.
| let diffs: Diff[] = []; | ||
|
|
||
| // Make sure database is setup. | ||
| await setupSchemaIfNecessary(instanceId, databaseId, options); |
There was a problem hiding this comment.
NOTE: This should never be here.
firebase dataconnect:sql:diff must not mutate SQL state, as it's a read-only command.
setupSchemaIfNecessary may update some SQL schema.
Sorry, something went wrong.
| `Only users with 'roles/cloudsql.admin' can grant SQL roles. If you do not have this role, ask your database administrator to run this command or manually grant ${fdcSqlRole} to ${user}`, | ||
| ); | ||
| } | ||
| await ensureServiceIsConnectedToCloudSql( |
There was a problem hiding this comment.
Added this so that dataconnect:sql:grant would error out quick if Cloud SQL is still being created.
Sorry, something went wrong.
| `Missing required IAM permission to setup SQL schemas. SQL schema setup requires 'roles/cloudsql.admin' or an equivalent role.`, | ||
| ); | ||
| } | ||
| await setupIAMUsers(instanceId, databaseId, options); |
There was a problem hiding this comment.
This was repetitive.
The caller is responsive to make sure IAM users are already setup.
It's used in two places, both with the same flow:
setupIAMUsers getSchemaMetadata setupSQLPermissions
Sorry, something went wrong.
joehan
left a comment
There was a problem hiding this comment.
LGTM with some minor changelog fixes
Sorry, something went wrong.
Co-authored-by: Joe Hanley <joehanley@google.com>
Co-authored-by: Joe Hanley <joehanley@google.com>
Description
firebase deploy will no longer hang due to Cloud SQL provision delay.
Similar to Firebase console, it would save the Cloud SQL metadata in FDC Schema, then starts a UpdateSchema(MIGRATE_COMPATIBLE) LRO to wait for Cloud SQL creation and setup SQL schema afterwards.
During the Cloud SQL provision window, firebase deploy would fail because there is a blocking LRO on Schema. That's expected. During that period, customers can still use the ephermeral database to prototype.
firebase init dataconnect
firebase deploy right after that
Update dataconnect.yaml and then run firebase deploy
firebase deploy Again