FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Make migrations work · bpg130/python-spanner-orm@04f7138 · GitHub

/ python-spanner-orm Public
forked from google/python-spanner-orm

Commit 04f7138

Browse files
dcbrandao
committed
Make migrations work
Ran through creating a database, creating a migration, executing the migration, and rolling back the migration, updated all the code to make it actually work. Also updated the documentation to describe the migration process
1 parent 16c1978 commit 04f7138

7 files changed

Lines changed: 88 additions & 16 deletions

File tree

‎README.md‎

Lines changed: 37 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -50,18 +50,21 @@ class TestModel(spanner_orm.Model):
5050
```
5151

5252
If the model does not refer to an existing table on Spanner, we can create
53-
the corresponding table on the database through the ORM in one of two ways:
53+
the corresponding table on the database through the ORM in one of two ways. If
54+
the database has not yet been created, we can create it and the table at the
55+
same time by:
5456

5557
``` python
56-
spanner_orm.connect_admin(
58+
admin_api = spanner_orm.connect_admin(
5759
'instance_name',
5860
'database_name',
5961
create_ddl=spanner_orm.model_creation_ddl(TestModel))
60-
spanner_orm.spanner_admin_api().create()
62+
admin_api.create()
6163
```
6264

63-
or by executing a Migration where the upgrade method returns a CreateTable for
64-
the model you have just defined (see section on migrations)
65+
If the database already exists, we can execute a Migration where the upgrade
66+
method returns a CreateTable for the model you have just defined (see section
67+
on migrations)
6568

6669

6770
### Retrieve data from Spanner
@@ -137,4 +140,32 @@ complex use cases, but you will have to do more work in order to use those
137140
correctly. See the documentation on those methods for more information.
138141

139142
## Migrations
140-
TODO(dbrandao): work in progress
143+
### Creating migrations
144+
Running ```spanner-orm generate <migration name>``` will generate a new
145+
migration file to be filled out in the directory specified (or 'migrations' by
146+
default). The ```upgrade``` function is executed when migrating, and the
147+
```downgrade``` function is executed when rolling back the migration. Each of
148+
these should return a single SchemaUpdate object (e.g., CreateTable, AddColumn,
149+
etc.), as Spanner cannot execute multiple schema updates atomically.
150+
151+
### Executing migrations
152+
Running ```spanner-orm migrate <Spanner instance> <Spanner database>``` will
153+
execute all the unmigrated migrations for that database in the correct order,
154+
using the application default credentials. If that won't work for your use case,
155+
```MigrationExecutor``` can be used instead:
156+
157+
``` python
158+
connection = spanner_orm.SpannerConnection(
159+
instance_name,
160+
database_name,
161+
credentials)
162+
executor = spanner_orm.MigrationExecutor(connection)
163+
executor.migrate()
164+
```
165+
166+
Note that there is no protection against trying execute migrations concurrently
167+
multiple times, so try not to do that.
168+
169+
If a migration needs to be rolled back,
170+
```spanner_orm rollback <migration_name> <Spanner instance> <Spanner database>```
171+
or the corresponding ```MigrationExecutor``` method should be used.

‎setup.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616
from setuptools import setup
1717
setup(
1818
name='spanner-orm',
19-
version='0.1.8',
19+
version='0.1.9',
2020
description='Basic ORM for Spanner',
2121
maintainer='Derek Brandao',
2222
maintainer_email='dbrandao@google.com',

‎spanner_orm/__init__.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,9 +20,11 @@
2020
from spanner_orm import decorator
2121
from spanner_orm import error
2222
from spanner_orm import field
23+
from spanner_orm import index
2324
from spanner_orm import model
2425
from spanner_orm import relationship
2526
from spanner_orm.admin import api as admin_api
27+
from spanner_orm.admin import migration_executor
2628
from spanner_orm.admin import update
2729

2830
# add NullHandler to root-module logger so that individual modules
@@ -50,6 +52,7 @@
5052
Boolean = field.Boolean
5153
Field = field.Field
5254
Integer = field.Integer
55+
Index = index.Index
5356
Relationship = relationship.Relationship
5457
String = field.String
5558
StringArray = field.StringArray
@@ -84,3 +87,5 @@
8487
DropIndex = update.DropIndex
8588
NoUpdate = update.NoUpdate
8689
model_creation_ddl = update.model_creation_ddl
90+
91+
MigrationExecutor = migration_executor.MigrationExecutor

‎spanner_orm/admin/metadata.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,8 +34,10 @@ class SpannerMetadata(object):
3434
"""Gathers information about a table from Spanner."""
3535

3636
@classmethod
37-
def _class_name_from_table(cls, table_name: str) -> str:
38-
return 'table_{}_model'.format(table_name)
37+
def _class_name_from_table(cls, table_name: Optional[str]) -> Optional[str]:
38+
if table_name:
39+
return 'table_{}_model'.format(table_name)
40+
return None
3941

4042
@classmethod
4143
def models(cls) -> Dict[str, Type[model.Model]]:

‎spanner_orm/admin/migration_executor.py‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,9 @@ def rollback(self, target_migration: str) -> None:
110110
self._hangup()
111111

112112
def _connect(self) -> None:
113-
admin_api.from_connection(self._connection)
113+
api_connection = admin_api.from_connection(self._connection)
114+
if not self._connection.database.exists():
115+
api_connection.create_database()
114116

115117
def _hangup(self) -> None:
116118
admin_api.hangup()

‎spanner_orm/admin/scripts.py‎

Lines changed: 37 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,8 @@
1919
import argparse
2020
from typing import Any
2121

22+
from spanner_orm import api
23+
from spanner_orm.admin import migration_executor
2224
from spanner_orm.admin import migration_manager
2325

2426

@@ -27,23 +29,53 @@ def generate(args: Any) -> None:
2729
manager.generate(args.name)
2830

2931

32+
def migrate(args: Any) -> None:
33+
connection = api.SpannerConnection(args.instance, args.database)
34+
executor = migration_executor.MigrationExecutor(connection, args.directory)
35+
executor.migrate(args.name)
36+
37+
38+
def rollback(args: Any) -> None:
39+
connection = api.SpannerConnection(args.instance, args.database)
40+
executor = migration_executor.MigrationExecutor(connection, args.directory)
41+
executor.rollback(args.name)
42+
43+
3044
def main(as_module: bool = False) -> None:
3145
prog = 'spanner-orm' if as_module else None
3246
parser = argparse.ArgumentParser(prog=prog)
3347
subparsers = parser.add_subparsers(
34-
title='subcommands', description='valid subcommands')
48+
dest='subcommand',
49+
title='subcommands',
50+
description='valid subcommands',
51+
required=True)
3552

3653
generate_parser = subparsers.add_parser(
3754
'generate', help='Generate a new migration')
3855
generate_parser.add_argument('name', help='Short name of the migration')
3956
generate_parser.add_argument('--directory')
4057
generate_parser.set_defaults(execute=generate)
4158

59+
migrate_parser = subparsers.add_parser(
60+
'migrate', help='Execute unmigrated migrations')
61+
migrate_parser.add_argument(
62+
'--name', help='Stop migrating after this migration')
63+
migrate_parser.add_argument('--directory')
64+
migrate_parser.add_argument('instance', help='Name of Spanner instance')
65+
migrate_parser.add_argument('database', help='Name of Spanner database')
66+
migrate_parser.set_defaults(execute=migrate)
67+
68+
rollback_parser = subparsers.add_parser(
69+
'rollback', help='Roll back migrated migrations')
70+
rollback_parser.add_argument(
71+
'name', help='Keep rolling back past this migration')
72+
rollback_parser.add_argument('--directory')
73+
rollback_parser.add_argument('instance', help='Name of Spanner instance')
74+
rollback_parser.add_argument('database', help='Name of Spanner database')
75+
rollback_parser.set_defaults(execute=rollback)
76+
4277
args = parser.parse_args()
43-
if hasattr(args, 'execute'):
44-
args.execute(args)
45-
else:
46-
parser.print_help()
78+
args.execute(args)
4779

4880

4981
if __name__ == '__main__':

‎spanner_orm/api.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,7 @@ def __init__(self,
126126
client = spanner.Client(project=project, credentials=credentials)
127127
instance = client.instance(instance)
128128
self.database = instance.database(
129-
database, pool=pool, ddl_statements=create_ddl)
129+
database, pool=pool, ddl_statements=create_ddl or ())
130130

131131

132132
class SpannerApi(SpannerReadApi, SpannerWriteApi):

0 commit comments

Comments
 (0)
Back | FazBrowse Home | New Git URL