FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

feat!: migrations move under il migration; Python script becomes inputlayer-migrate by jsam · Pull Request #101 · inputlayer/inputlayer · GitHub

feat!: migrations move under il migration; Python script becomes inputlayer-migrate - #101

Merged
jsam merged 6 commits into
mainfrom
py/il-migrate-rename
Aug 20, 2026
Merged

feat!: migrations move under il migration; Python script becomes inputlayer-migrate#101
jsam merged 6 commits into
mainfrom
py/il-migrate-rename

Conversation

jsam commented Aug 19, 2026
edited
Loading

Copy link
Copy Markdown
Contributor

What

Three related changes that make migrations part of the single il CLI and language-independent:

1. Vocabulary: il migration

You type Delegates to
il migration generate --models myapp.models inputlayer-migrate makemigrations ...
il migration apply --url ws://... --kg production inputlayer-migrate migrate ...
il migration revert inputlayer-migrate revert ...
il migration status inputlayer-migrate showmigrations ...

il owns the vocabulary; the delegation layer (#98, with integration tests) translates. Console script renamed il -> inputlayer-migrate. --migrations-dir accepted after the subcommand on every subcommand.

2. Language-neutral migration files (step 1-2 of #104)

Migration artifacts were Python source loaded via importlib - unusable by any other language. They are now JSON documents: {format, dependencies, operations (typed, with enough structure to derive apply AND revert IQL), state}. The writer emits .json; the loader reads .json plus legacy .py (read-only back-compat). test_writer rewritten around the JSON document incl. a no-code guarantee and operation roundtrip; JSON state normalizes column pairs to tuples so the autodetector sees no phantom changes. This unblocks the JS frontend and native il apply (#104 steps 3-5).

3. Recorder bugs: migrations never worked against a real engine

Found by live end-to-end verification, invisible to mocked unit tests:

  • Relation name inputlayer_migrations is rejected by schema declarations but silently auto-created untyped by inserts, and unqueryable - applied-state was write-only. Renamed to inputlayer_migrations. Engine inconsistency filed as Engine: inserts bypass relation-name validation, creating unreadable ghost relations #106.
  • get_applied() used '?X <- rel(...)', a form the engine rejects - status always empty, revert never fired, apply re-applied every run. Switched to the plain query form.

Verified

Known limits

Merge order

With or right after #98. No conflicts with other open PRs.

…mes inputlayer-migrate

il is the single product CLI and migrations are a noun group under it:
il migration generate / apply / revert / status. The delegation layer in
the il binary translates these to the Python tool's subcommands, so
Django-flavored names (makemigrations) never surface in the product CLI
and 'il migrate migrate' never exists.

The SDK's console script is renamed from 'il' to 'inputlayer-migrate'
(matching its existing argparse prog) so the name 'il' belongs to the
product CLI alone. Docs updated to the new vocabulary.

Breaking for scripts invoking the old Python 'il' directly: use
'il migration <verb>' (or 'inputlayer-migrate <subcommand>' where only
the SDK is installed).
jsam force-pushed the py/il-migrate-rename branch from 1545250 to 55cdf29 Compare August 19, 2026 17:13
jsam changed the title feat!: migrations move under il migrate; Python script becomes inputlayer-migrate feat!: migrations move under il migration; Python script becomes inputlayer-migrate Aug 19, 2026
Found by running the full lifecycle (generate/apply/status/revert)
end to end via il migration against a live server. Two bugs, both
invisible to the mocked unit tests:

1. The recorder relation was named __inputlayer_migrations__. The
   engine rejects that name in schema declarations (must start with a
   lowercase letter) but silently auto-creates it untyped on insert,
   and queries against it fail - so applied-state was written into a
   relation that could never be read. Renamed to inputlayer_migrations.
   (Engine-side inconsistency filed separately: inserts bypass the
   relation-name validation that schema declarations enforce.)

2. get_applied() used '?Name, At <- rel(Name, At)', which the engine
   rejects ('Query cannot contain a rule definition'). Every status
   read returned [], so showmigrations always showed unapplied, revert
   never found anything, and apply re-applied migrations every run.
   Switched to the plain query form and deduplicated.

Also: --migrations-dir is now accepted after the subcommand on every
subcommand (argparse.SUPPRESS so the global default survives), since
delegating callers always place flags after the subcommand.

Verified live: generate -> apply -> status [X] -> re-apply no-op ->
second migration -> revert to 0001 -> status shows [X]/[ ]. Parser
position tests added; full SDK suite 960 passed.
jsam added 2 commits August 20, 2026 09:30
Migration artifacts were Python source (operations rendered as code,
loaded via importlib) - unusable by any other SDK language, and even
listing history required a Python runtime. Migrations are now data:

- writer emits NNNN_name.json: {format, dependencies, operations, state}
  using the to_dict/from_dict serialization the operation classes
  already had. No code, no imports, nothing language-specific.
- loader reads both formats: JSON (current) and legacy .py (back-compat,
  read-only). JSON state normalizes column pairs back to tuples so the
  autodetector sees no phantom changes.
- test_writer rewritten for the JSON document (structure, operation
  roundtrip incl. forward/backward command equality, no-code guarantee);
  legacy .py loading stays covered in test_loader. Suite: 956 passed.
- migrations guide: anatomy section now shows the JSON document.

This is step 1-2 of #104: the on-disk format is now consumable by a
future JS frontend and by il natively (apply/revert/status without any
SDK installed). Verified live: generate -> apply -> status -> revert
against a running engine with JSON files.
…y, loader validation, stale docs

Adversarial review of this PR found (all reproduced by execution):

1. BLOCKER: a directory containing both 0001_x.py and 0001_x.json (a
   stale legacy twin of a regenerated migration) loaded twice and
   APPLIED twice in one run - destructive for Drop/Replace operations.
   load_migrations now rejects duplicate names and duplicate numbers
   with a clear error.
2. Engine errors were invisible: the WS protocol reports failures as an
   'error'-column result, and nothing in the migration stack checked -
   a failed op mid-migration was recorded as applied with exit 0, and
   get_applied() could return the error text as a migration name. New
   errors.py check_engine_result is applied to every recorder and
   executor statement; a failed migration is never recorded.
3. The JSON loader had zero validation: malformed JSON, missing op
   fields, or unknown op types produced raw tracebacks without
   filenames, and the format field was write-only. All now raise
   MigrationError naming the file; format > supported rejects with an
   upgrade hint. The CLI catches MigrationError cleanly.
4. Migration names were unvalidated: --name a/b wrote into a
   nonexistent subdirectory and quotes broke the recorder's IQL.
   Writer enforces [a-z0-9_]+; recorder escapes interpolated names.
5. Stale docs taught the removed 'il' script: the PyPI-shipping SDK
   README, python-sdk.mdx, and the generated GUI docs bundle
   (regenerated). migrations.mdx output examples now match actual CLI
   output.

New tests/test_migration_safety.py covers all of it (13 tests: dual
format collisions, the legit mixed directory, malformed JSON, format
version, error-frame propagation, no-record-on-failure, name safety).
Full SDK suite: 969 passed.
jsam added 2 commits August 20, 2026 12:12
Pre-1.0 decision: no legacy formats. The importlib-based .py loader,
the Migration base class it existed for (and its package exports), and
the dual-format handling are gone. Stray .py files from older SDKs are
inert - ignored by discovery, never imported, never double-applied.
Duplicate-number detection remains for renumber collisions.

test_loader rewritten around JSON; safety tests updated (stray .py twin
is ignored); docs and GUI bundle updated. Suite: 969 passed.
Found by live demo verification: an unreachable server printed a raw
InputLayerConnectionError traceback. The CLI now catches InputLayerError
alongside MigrationError and prints a one-line error with exit 1.
Test added; suite 970 passed.
jsam force-pushed the py/il-migrate-rename branch from 5416fd3 to 862fe72 Compare August 20, 2026 10:20
jsam merged commit dde43d0 into main Aug 20, 2026
11 checks passed
jsam deleted the py/il-migrate-rename branch August 20, 2026 11:33
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant


Back | FazBrowse Home | New Git URL