FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

feat(entities): new entity APIs — cursor pages, count, aggregate, upsert by arosenan · Pull Request #287 · base44/javascript-sdk · GitHub

feat(entities): new entity APIs — cursor pages, count, aggregate, upsert - #287

Merged
arosenan merged 12 commits into
mainfrom
entities-scan-free-primitives
Sep 22, 2026
Merged

arosenan merged 12 commits into
mainfrom
entities-scan-free-primitives

Conversation

Copy link
Copy Markdown
Contributor

Why

Fifty apps with the deepest skip reads on the entities cluster were reviewed. None of them pages because a user asked for page N. Every deep-skip loop exists to get something the SDK cannot give:

Need Apps of 50 New API
A number: count, sum, group-by, avg, max-per-key, count-distinct ~37 aggregate(), count()
"Does this key already exist?" before insert; duplicate cleanup ~23 upsert(), aggregate({ having })
Walk a big table in order: export, sitemap, resumable job ~15 cursor on list() / filter()
Distinct values for dropdowns / autocomplete 5 distinct option on list() / filter()

Three apps built binary search with limit-1 probes as a substitute for count (skips of 10.2M, 900k and 64k). Builders persist lastSkip as a resume token because nothing else exists.

The server routes land in base44-dev/apper#24939. This PR is the SDK half, and its purpose right now is to settle the API shape. Version stays 0.8.x: everything is additive, so existing apps on ^0.8.x pick it up on their next install.

Proposed API

// Cursor pages: same list()/filter(), an options object instead of positional args.
// The token carries the query, sort and fields, so later pages need only the cursor.
let page = await base44.entities.Order.filter(
  { status: 'open' },
  { sort: '-created_date', limit: 1000 }
);                                        // -> { items, next_cursor, has_more }
while (page.has_more) {
  page = await base44.entities.Order.filter({ status: 'open' }, { cursor: page.next_cursor, limit: 1000 });
}

await base44.entities.Order.list('-created_date', 100);   // unchanged, -> Order[]

// A number, not rows.
const open = await base44.entities.Order.count({ status: 'open' });

// Distinct values of a field, for dropdowns and autocomplete: a page of values, not records.
const { items: brands } = await base44.entities.Product.filter({ active: true }, { distinct: 'brand' });
// items: ['Adidas', 'Nike', ...]  (ascending, cursor-paged like any other page)

// Group-by aggregates, computed server-side.
const { rows } = await base44.entities.Sale.aggregate({
  query: { sale_date: { $gte: '2026-09-01' } },
  groupBy: 'agent_id',
  sum: 'amount',
  sort: '-sum_amount',
});
// rows: [{ agent_id, count, sum_amount }, ...]; also dateBucket, avg/min/max, countDistinct, having, limit

// Create or update by a key of your own, up to 500 records per call.
const { created, updated, records } = await base44.entities.Contact.upsert(rows, { key: 'crm_id' });

Design notes:

  • The cursor token carries the query, sort and fields of the walk, like Wix Data's cursorPaging: a later page needs only cursor and limit, and a different query or sort with a cursor is a 400 rather than silently ignored. The server re-sanitizes the query and applies scope and RLS on every page, so the token adds no new trust.
  • distinct is an option on list()/filter(), like distinct() on a Wix Data or Mongo query, rather than a group-by spelled through aggregate(). It returns the same page envelope with values as items.
  • The cursor lives on list()/filter() rather than a new method, matching Stripe/Firestore/Prisma. Passing an options object is what switches the return type to a page envelope; positional calls keep returning arrays, so no existing code changes. skip stays and is documented as deprecated for loops. The options form defaults to 100 rows per page (Notion-style), the positional form keeps its 5,000 default.
  • Vocabulary follows the SDK, not Mongo: the aggregate filter is query like filter(query), spec keys are camelCase like the SDK's methods; server-produced fields stay snake_case (next_cursor, has_more, sum_amount) like created_date.
  • updateMany is unchanged here. Making it update every match, instead of batches the caller loops over, is a server-side change tracked in the backend PR.

What

  • list(options) / filter(query, options) → GET /{entity}/v2/list → EntityPage<T>; EntityListOptions<T, K> typed against the schema. With EntityDistinctOptions<T, K> ({ distinct, limit, cursor }) the same route returns EntityPage<T[K]>. Same route family as list, versioned like /conversations/v2/… in the agents module; the backend route takes this path once the API is closed.
  • count(query?) → GET /{entity}/count?q= → number.
  • aggregate(spec) → POST /{entity}/aggregate → EntityAggregateResult (server caps at 1000 rows); EntityAggregateSpec<T> typed against the schema. Spec keys are query, groupBy, dateBucket, count, sum, avg, min, max, countDistinct, having, sort, limit; the spec is the wire body.
  • upsert(records, { key }) → POST /{entity}/upsert → EntityUpsertResult<T>.

New types are exported from the package entry and listed in types-to-expose.json for the docs pipeline. JSDoc follows the sdk-docs-writing skill: description, params, returns, examples on every method.

Tests

  • tests/unit/entities-primitives.test.ts: nock tests for each route, params and body; positional list/filter still hit the list route and return arrays.
  • tests/types/entities-primitives.types.ts: spec and option fields are tied to the entity type; unknown fields, units, sorts and keys fail to compile.
  • npm run build, npm run test:types, npm run test:unit (308 tests) and npm run lint pass.

No version bump, following the repo's separate chore: bump version commits.

🤖 Generated with Claude Code

arosenan and others added 11 commits September 16, 2026 12:50
…egate, upsert

Wraps the scan-free entity routes added in base44-dev/apper#24939. Fifty apps with
the deepest `skip` reads were reviewed; none paged because a user asked for page N.
Every loop existed to get a number, to check whether a key already exists, to walk a
table with a resume point that is not an offset, or to list a field's distinct values.

- list(options) / filter(query, options): pass an options object {sort, limit,
  cursor, fields} instead of positional args to read one cursor page; returns
  {items, next_cursor, has_more}. Positional calls are unchanged. skip is
  documented as deprecated for loops.
- count(query?): number of readable records matching a filter.
- distinct(field, query?): {values, truncated}, capped at 5000 values.
- aggregate(spec): group_by / date_bucket / count / sum / avg / min / max /
  count_distinct / having / sort / limit; returns {rows, truncated}.
- upsert(records, {key}): create or update by a natural key, up to 500 records.

New public types are exported and listed in types-to-expose.json.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…rameter

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- aggregate spec: `query` (not `match`), camelCase keys (`groupBy`, `dateBucket`,
  `countDistinct`), and no rule against combining countDistinct with other measures
- cursor pages default to 100 rows; the maximum stays 5,000

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A cursor token now carries the query, sort and fields of the walk, so a
later page needs only cursor and limit (same model as Wix Data cursorPaging).
distinct moves out of aggregate-only usage into an option on list()/filter()
that returns a page of values, matching how query APIs usually expose it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
aggregate(stages[]) posts { pipeline } to the same route; field names are the
entity's own and the server translates them. The spec form stays the primary,
typed API; the pipeline is the escape hatch for what the spec cannot express.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

github-actions Bot commented Sep 18, 2026
edited
Loading

Copy link
Copy Markdown

🚀 Package Preview Available!


Install this PR's preview build with npm:

npm i @base44-preview/sdk@0.8.48-pr.287.c4af256

Prefer not to change any import paths? Install using npm alias so your code still imports @base44/sdk:

npm i "@base44/sdk@npm:@base44-preview/sdk@0.8.48-pr.287.c4af256"

Or add it to your package.json dependencies:

{
  "dependencies": {
    "@base44/sdk": "npm:@base44-preview/sdk@0.8.48-pr.287.c4af256"
  }
}

Preview published to npm registry — try new features instantly!

github-actions Bot added the docs-draft PR has auto-drafted documentation suggestions label Sep 18, 2026
The entities primitives (and every other route that raises ApiError) answer
with {"error": {"code", "message", "details"}}. The error mapper only read the
legacy top-level code/message, so a caller catching an invalid_cursor 400 saw
code undefined and axios's generic message. Found while running the preview
build of this branch inside a Base44 app.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
arosenan merged commit dea1829 into main Sep 22, 2026
7 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs-draft PR has auto-drafted documentation suggestions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants


Back | FazBrowse Home | New Git URL