FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

docs(skills): encode integration-hardening learnings into the authoring skills by waleedlatif1 · Pull Request #7271 · simstudioai/sim · GitHub

docs(skills): encode integration-hardening learnings into the authoring skills - #7271

Closed
waleedlatif1 wants to merge 7 commits into
stagingfrom
docs/integration-authoring-learnings
Closed

docs(skills): encode integration-hardening learnings into the authoring skills#7271
waleedlatif1 wants to merge 7 commits into
stagingfrom
docs/integration-authoring-learnings

Conversation

Copy link
Copy Markdown
Collaborator

Eight traps found empirically across a 15-PR integration-hardening sweep, each costing real debugging and currently written down nowhere. Each learning goes into the skill that owns it, with a file:line citation so the guidance stays checkable; the other skills cross-reference rather than duplicate.

Additive and surgical — no restructuring, one deliberate correction noted below.

add-tools

  • Reserved parameter names. apps/sim/tools/request-transport.ts reads timeout (:191), proxyUrl (:198), and method (:167) off params for its own use before request sees them. timeout is the outbound HTTP deadline in milliseconds, so daytona/execute_command.ts:49 — documented as "Timeout in seconds (defaults to 10 seconds)" — aborts the call after 10ms. Still live on staging.
  • Path traversal. encodeURIComponent does not stop ./..: both are unreserved, and the WHATWG URL parser removes dot segments after decoding. Documents when each of the three tools/url-path.ts helpers applies (safeUrlPathSegment / safeUrlPath / safeEncodedUrlPathSegment) and why only safeUrlPath refuses to trim.
  • params.x?.trim() guards undefined, not the type. A <Block.output> resolving to a number throws a raw TypeError; toGuardedString (url-path.ts:98) is why the helpers do not.

add-block

  • Omitting a key from tools.config.params does NOT drop it. executor/handlers/generic/generic-handler.ts:191 does { ...inputs, ...transformedParams }, so the raw subBlock value merges back in. Clearing a reserved name needs an explicit undefined.
  • Renaming a subBlock id orphans saved workflow state. Rename the tool param and map it; scripts/check-block-registry.ts:181 / :225 enforce it, and _removed_ entries in subblock-migrations.ts cover genuine removals.
  • Declared outputs do not drive variable resolution. executor/utils/block-reference.ts:239 walks the runtime object; the schema is consulted only at :242 when the value is already undefined. Changing an output's shape therefore breaks saved references that were never declared — spread raw keys last.

validate-integration

  • Path-safety harness design. A harness that fuzzes every param at once with catch { return } lets one guarded param hide all its siblings — x_manage_block.targetUserId and okta_remove_user_from_group.userId were fully unguarded and passed everything. The sound shape: enumerate (tool, param) pairs and fuzz one at a time; assert named rejection, not path shape (shape misses %2F and a trailing bare .); probe conditional and presence branches; assert the skipped/unbuildable ledger is empty.
  • Test files are type-checked by nothing. apps/sim/tsconfig.json excludes **/*.test.ts and vitest.config.ts declares no typecheck block.
  • Corrects existing guidance: the two checklist lines teaching `${params.id.trim()}` as the path pattern are replaced — that is the exact defect this sweep fixed.

add-integration

Four new entries under Common Gotchas, pointers only.

Gates

bun run lint, bun run check:audits (39 audits, includes check:skills), and bun run skills:sync (36 skills already in sync — no projection to commit) all pass.

…ng skills

Eight traps found across a 15-PR integration-hardening sweep, each written
into the skill that owns it and cross-referenced rather than duplicated.

add-tools
- Reserved param names. The shared transport reads `timeout`, `proxyUrl`, and
  `method` off `params` before `request` sees them; `timeout` is its own HTTP
  deadline in milliseconds, so Daytona's documented 10-second sandbox timeout
  aborts the call after 10ms.
- Path traversal. `encodeURIComponent` does not stop `.`/`..` — they are
  unreserved and the URL parser removes dot segments after decoding. Documents
  when each of the three `tools/url-path.ts` helpers applies, and why
  `params.x?.trim()` guards `undefined` rather than the type.

add-block
- Omitting a key from `tools.config.params` does not drop it; the executor
  merges the patch over the raw inputs, so clearing a key needs an explicit
  `undefined`.
- Renaming a subBlock id orphans saved workflow state. Rename the tool param
  and map it; `_removed_` migrations cover genuine removals.
- Declared `outputs` do not drive variable resolution — the resolver walks the
  runtime object, so changing an output's shape breaks references that were
  never declared.

validate-integration
- Path-safety harness design: enumerate (tool, param) pairs, fuzz one at a time,
  assert named rejection rather than path shape, probe conditional and presence
  branches, and assert the skip ledger is empty.
- Test files are type-checked by nothing — tsconfig excludes them and Vitest
  transpiles without checking.
- Replaces the two checklist lines that taught the now-known-defective
  `${params.id.trim()}` path pattern.

add-integration gets pointers only.

vercel Bot commented Aug 29, 2026
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Preview Aug 29, 2026 7:12am

greptile-apps Bot commented Aug 29, 2026
edited
Loading

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds integration-authoring guidance based on recurring hardening failures.

  • Documents reserved transport parameters, URL path safety, and runtime input coercion.
  • Explains subBlock persistence, parameter patch merging, and runtime output compatibility.
  • Adds robust path-safety test guidance and corrects the validation sequence.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
.agents/skills/add-block/SKILL.md Documents durable subBlock IDs, explicit parameter clearing, and backward-compatible output transformations.
.agents/skills/add-integration/SKILL.md Adds concise cross-references for reserved parameters, path safety, and block-state compatibility.
.agents/skills/add-tools/SKILL.md Adds detailed transport-reservation and URL-hardening guidance with an explicit precondition for helpers not yet available on staging.
.agents/skills/validate-integration/SKILL.md Replaces unsafe path guidance and adds comprehensive validation patterns for traversal guards and trustworthy tests.

Reviews (5): Last reviewed commit: "docs(skills): correct the Enrow billing ..." | Re-trigger Greptile

cubic-dev-ai Bot left a comment
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

All reported issues were addressed across 4 files

Tip: instead of fixing issues one by one fix them all with cubic

Re-trigger cubic

… a succeeding one

The most valuable learning of the sweep, and the one every other check
missed — sixteen PRs' own suites, both review bots, and the author.

Trimming a path identifier looks strictly safer. It is not when the
identifier previously went out through a bare encodeURIComponent and
reaches a destructive endpoint: box_sign_cancel_request went from a 404
no-op to cancelling a real signature request, and delete_r2_bucket from
naming no bucket to destroying prod-data. BigQuery's delete_dataset and
delete_table had the same shape on projectId.

Records the reasoning error that hid it: "trimming is the helper's
contract at all 137 sites" is an average, and the question is an
intersection — parameters whose normalisation actually changed, crossed
with irreversible operations. On that PR the answer was one of 137.
The resolution is strictUrlPathSegment, argued from the values (no
legitimate id carries surrounding whitespace, and the previous behaviour
was already a clean failure), not from consistency.

Two smaller ones folded in:

- safeUrlPath rejects only a truly empty path component, never a
  whitespace-only one. Git tracks a file and a directory named only
  spaces, and the parser never removes %20%20%20.
- A test that calls a function directly can pass while the wrapper does
  the opposite. executeTool catches a postProcess throw and restores the
  submit response, so eleven green Enrow failure-path tests sat over a
  production success: true.

Copy link
Copy Markdown
Collaborator Author

Added the late learning — the most valuable one this effort produced — plus the two smaller ones. Pushed as 37e99e97cf.

validate-integration → new subsection in Step 8

A hardening change must not turn a failing request into a succeeding one.

Stated in exactly those words, with both confirmed instances verified against their branches:

  • box_sign_cancel_request — apps/sim/tools/box_sign/cancel_request.ts:34 (fix/storage-path-safety): /2.0/sign_requests/%20%20<uuid>%20%20/cancel → 404 no-op before; /2.0/sign_requests/<uuid>/cancel → cancels a real signature request after.
  • cloudflare_delete_r2_bucket — apps/sim/tools/cloudflare/delete_r2_bucket.ts:30 (fix/cloudflare-discord-path-safety): " prod-data " names no bucket that can exist → fails; trimmed → destroys prod-data.
  • google_bigquery_delete_dataset — delete_dataset.ts:53, same shape on projectId.

The section records the reasoning error, because that is the part that makes it hard to see: "trimming is the helper's contract at all 137 sites" is an average. The question is an intersection — parameters whose normalisation this change actually alters, crossed with irreversible operations (DELETE, cancel, revoke, purge, drop). One of 137, after four review passes escalating P2→P2→P1→P1 and two pushbacks.

The resolution is written up as argued from the values, not from consistency: no legitimate identifier for these providers carries surrounding whitespace (Box Sign UUID, GCP [a-z][a-z0-9-]{5,29}, R2 ^[a-z0-9][a-z0-9-]*[a-z0-9]), and the previous behaviour was already a clean failure — so refusing preserves it and upgrades an opaque 404 to a named error. strictUrlPathSegment / assertNoSurroundingWhitespace (apps/sim/tools/strict-url-path.ts:41, :51). Parameters already trimmed before the change keep plain safeUrlPathSegment, explicitly.

Four checklist items, and a cross-reference from add-tools where the path guards are introduced.

add-tools → safeUrlPath whitespace asymmetry

Folded into the same cross-reference: safeUrlPath rejects only a truly empty component (url-path.ts:317, the if (!segment) after 515b9516cc), never a whitespace-only one — git tracks a file and a directory named only spaces, and new URL('https://x/a/%20%20%20/b').pathname keeps the segment where a dot segment is removed. safeUrlPathSegment still rejects an all-whitespace value, because it trims opaque ids first. The asymmetry is stated so it is not "fixed" back.

validate-integration → alongside the type-check note

A test that calls a function directly can pass while the wrapper does the opposite. executeTool catches a postProcess throw and restores the pre-postProcess result (apps/sim/tools/index.ts:1977 and :2062) — the submit response, success: true with every field null — and the hosted-key cost hook, gated on finalResult.success (:1987), bills it. Eleven green Enrow failure-path tests sat over that.

All gates re-run clean: bun run lint, bun run check:audits (39 audits), bun run skills:sync (no projection).

The most-repeated defect of the whole effort — four separate instances on
one PR, each a blanket tolerance that made an assertion unable to fail.
Promoted from a note to a first-class rule in the harness-design step.

A tolerated throw must be tolerated BY NAME, in an explicit allowlist,
with the reason recorded. A blanket `catch { return }` converts every
case it covers from tolerated to untested. The line that keeps the rule
usable: tolerating a failed probe during discovery is legitimate, since
probing a guarded param is meant to throw — tolerating a throw inside an
assertion is the bug.

The fourth instance earns its own paragraph because it fails in the
opposite direction from everything else this sweep was about: swallowing
the throw meant the origin, prefix and inert-probe assertions never ran,
so a guard that OVER-tightened passed silently. A path-safety suite that
only catches under-guarding is half a suite. The resolution shape —
enumerate every pair against every inert value, measure which legitimately
throw, then make a throw a failure unless the param is in an explicit
strictlyValidated list — is written out, with the measured answer of ten
pairs (Supabase table and functionName).

Adds "every new assertion is verified red before it is kept" to the
checklist, which is the practice that would have caught all four.

Also groups the type-check and bypassed-wrapper notes under one
"Your tests can lie to you" heading, and renumbers the path-safety step
to 9 (it collided with Memory Load Safety).

Copy link
Copy Markdown
Collaborator Author

Promoted the assertion-that-cannot-fail pattern to a first-class rule. Pushed as 40a4ee6996.

validate-integration Step 9 → ### Never let a catch stand in for an assertion

A tolerated throw must be tolerated by name, in an explicit allowlist, with the reason recorded. A blanket catch { return } converts every case it covers from tolerated to untested.

Kept the distinction that makes it usable rather than absolutist: tolerating a failed probe during discovery is legitimate — probing a guarded param is meant to throw. Tolerating a throw inside an assertion is the bug.

The four instances are tabulated with what each blanket tolerance hid, and the fourth gets its own paragraph because it fails in the opposite direction from everything else this sweep was about: catch { return } meant the origin check, prefix check and inert-probe assertion never ran, so a guard that over-tightened passed silently. A path-safety suite that only catches under-guarding is half a suite.

The resolution shape is written out as three steps — enumerate every (tool, param) pair against every inert value, measure which legitimately throw, then make a throw a failure unless the param is in an explicit allowlist — with the measured answer: exactly ten pairs, Supabase table via validateDatabaseIdentifier and functionName via validateFunctionName, correct because abc#fragment is a fine URL segment but not a SQL identifier.

Two new checklist items, including the one that would have caught all four: every new assertion is verified red before it is kept.

Grouping

Took the suggestion — These tests are type-checked by nothing and A test that calls a function directly can pass while the wrapper does the opposite are now #### under one ### Your tests can lie to you. The catch rule stays in harness design, since it is about how you write the suite rather than about a suite lying after the fact.

Verified (file:line, origin/fix/storage-path-safety)

  • apps/sim/tools/__tests__/path-safety.ts:441 — the strictlyValidated option, and its TSDoc already states the rule ("Listing the exceptions by name is what keeps 'tolerated' from silently becoming 'untested'").
  • :500 and :592 — both former catch {} sites, now expect(strictlyValidated.includes(paramName), …).toBe(true). Commit 1f395a0334 records the non-vacuity check: over-tightening strictUrlPathSegment to reject # fails three box_sign cases.
  • :416 preservesWhitespace — commit 72612813d5, the branch whose docstring says padding must survive to the wire was tolerating a refusal of it.
  • :371 toolsWithoutPathParams — commit e35e19154c, the vacuous pin; 11 tools across four services, invisible because the inventory was enumerated through a request.url-is-a-function narrowing.
  • apps/sim/tools/supabase/path_safety.test.ts:109 strictlyValidated: ['table', 'functionName'].
  • Sibling-masking swallow: fcb1118259 (622/622 green with x_manage_block.targetUserId fully unguarded) — already cited in property 1 of the harness-design list.

One correction while in here: my path-safety section was numbered Step 8, colliding with the pre-existing Step 8: Validate Memory Load Safety. Renumbered to Step 9; Error Handling and Report were already 10 and 11.

Gates clean: lint, check:audits (39), skills:sync (no projection).

Copy link
Copy Markdown
Collaborator Author

@greptile review

Copy link
Copy Markdown
Collaborator Author

@cubic review

cubic-dev-ai Bot commented Aug 29, 2026

Copy link
Copy Markdown

@cubic review

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

Comment thread .agents/skills/add-tools/SKILL.md Outdated
…counterweight

Nine recorded instances now, not four, and the three newest are each a
different shape that does not look like a catch:

- A namesParam matcher that was a substring scan, so "projectId cannot
  have leading whitespace" satisfied the assertion for param `id` and
  "pathological failure" satisfied `path` — a guard naming the WRONG
  identifier passed the very check meant to catch that.
- An assertion that guarded itself out of existence:
  `if (serialized?.includes('projectId')) { expect(...) }`.
- describe.each over a derived array a rename had silently emptied, so a
  whole block vanished emitting neither tests nor failures.

Adds the counterweight rule: assert exact error text and exact encoded
output, never a bare toThrow(). Three upstream changes to url-path.ts
landed underneath a downstream suite and only the exact assertions
noticed — trimming dropped, !segment.trim() narrowed to !segment, and a
rebase rewording "cannot have" to "must not have".

Review fixes:

- Qualifies the subBlock-rename mapper example. check-block-registry
  narrows to required + user-only params and demands a subBlock key equal
  to the tool param id, which a rename-at-execution mapper does not
  satisfy. (cubic, correct.)
- Settles branch/ref explicitly: GitHub's branches route is greedy on its
  final parameter, so `feature/api` takes safeUrlPath and a %2F would
  404. safeEncodedUrlPathSegment is for a non-greedy single value such as
  a label name. (cubic claimed the opposite; the shipped tools disagree.)
- Adds an availability note and converts every citation into an unlanded
  path-safety file from file:line to module + symbol. Those branches are
  actively rebasing — strict-url-path.ts has already been deleted and its
  symbols folded into url-path.ts — so a line number is stale on arrival.
  Exact file:line is kept for everything that is on staging.

cubic-dev-ai Bot left a comment
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Review completed against the latest diff

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread .agents/skills/add-tools/SKILL.md Outdated
…qualifies

Greptile re-raised at P1 that an author following this section cannot import
the helpers it prescribes. The note existed but sat six paragraphs below the
table, so it read as a footnote rather than a precondition.

It now leads the section, and says what to DO rather than only what is
missing: add the helper to url-path.ts with the semantics specified here,
never hand-roll a local encoder at the call site.

Copy link
Copy Markdown
Collaborator Author

@greptile review

Copy link
Copy Markdown
Collaborator Author

@cubic review

cubic-dev-ai Bot commented Aug 29, 2026

Copy link
Copy Markdown

@cubic review

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai Bot left a comment
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

All reported issues were addressed across 4 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread .agents/skills/add-block/SKILL.md Outdated
Comment thread .agents/skills/add-block/SKILL.md Outdated
…g assertions

Instances 10 and 11, both with a different CAUSE from the first nine —
which is why auditing catch blocks and filters did not find them:

- Fixture drift. `expect(serialized).not.toContain('  my-project  ')` was
  written when the fixture was padded; a strict guard made padding throw,
  the fixture was unpadded, and the assertion stayed, asserting the
  absence of a string that can no longer occur. Teaches the better fix —
  derive one literal from the other so a test named "agrees" asserts
  agreement rather than two constants that happen to match.
- A globally-mocked dependency. vitest.setup.ts:112 stubs
  @/tools/registry as `{ tools: {} }`, so a guard iterating the registry
  passes over an empty set; four real failures only reproduced after
  vi.unmock.

Review fixes, all four valid and all four my own guidance failing its own
standard:

- Step 9 recommended `toThrow(new RegExp(paramName))` three paragraphs
  above the section documenting the substring-matcher trap. Replaced with
  the capture-and-assert form the reference harness actually uses.
- The exactness rule recommended `toThrow('<message>')`, which Vitest
  treats as a SUBSTRING match — so the rule asserting exactness was
  itself inexact. Now pins with toBe on a captured message.
- The subBlock-rename qualification offered "keep the tool param name and
  clear the reserved key" for a required user-only param, which cannot
  satisfy both rules at once. Rename plus a migration is the only answer.
- The reserved-key checklist item was unconditional, which would forbid a
  block legitimately setting the transport's timeout, proxy, or method.
  Scoped to a collision where the subBlock means something provider-specific.

Copy link
Copy Markdown
Collaborator Author

@greptile review

Copy link
Copy Markdown
Collaborator Author

@cubic review

cubic-dev-ai Bot commented Aug 29, 2026

Copy link
Copy Markdown

@cubic review

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai Bot left a comment
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

All reported issues were addressed across 4 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

… list

Two review findings, both valid.

The swallowed postProcess throw does NOT bill on Enrow. Both getCredits
implementations return 0 when the output carries no `qualification`, and
the fall-back submit response has none — deliberately, per the comment in
verify_email.ts. The real consequence is that a stale SUCCESS reaches both
the user and the pricing hook, and whether that charges depends entirely
on the tool's own getCost. Restated as the rule that matters: write
getCost so it cannot charge for a result the poll never produced, and do
not rely on the failure propagating, because it does not.

The Step 3 checklist named all three url-path helpers without the
availability caveat that Step 9 and add-tools carry, so following it
against staging produces an import that will not compile. Gated.

Copy link
Copy Markdown
Collaborator Author

@greptile review

Copy link
Copy Markdown
Collaborator Author

@cubic review

cubic-dev-ai Bot commented Aug 29, 2026

Copy link
Copy Markdown

@cubic review

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

2 issues found across 4 files

Confidence score: 3/5

  • In .agents/skills/validate-integration/SKILL.md, the path-safety assertion may accept an unsafe URL builder by conflating encoded %2F with a real separator, weakening protection against incorrect path handling; assert url.pathname directly.
  • In .agents/skills/validate-integration/SKILL.md, the fallback submit-result description incorrectly implies every field is null while id remains populated, which could make failure-path tests encode the wrong execution contract; specify that only email and qualification are null.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".agents/skills/validate-integration/SKILL.md">

<violation number="1" location=".agents/skills/validate-integration/SKILL.md:549">
P2: This assertion cannot distinguish an encoded `%2F` from a real path separator, so a path-safety test can pass an unsafe builder. Assert `url.pathname` directly to pin the encoded output.</violation>

<violation number="2" location=".agents/skills/validate-integration/SKILL.md:578">
P3: The fallback submit result does not have every field null: `id` remains populated. Describe `email` and `qualification` as null while preserving the non-null job id, so failure-path tests do not encode the wrong executor result shape.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

equality, and pin the output with `toBe`:

```typescript
expect(decodeURIComponent(url.pathname)).toBe('<the exact expected path>')

cubic-dev-ai Bot Aug 29, 2026
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

P2: This assertion cannot distinguish an encoded %2F from a real path separator, so a path-safety test can pass an unsafe builder. Assert url.pathname directly to pin the encoded output.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .agents/skills/validate-integration/SKILL.md, line 549:

<comment>This assertion cannot distinguish an encoded `%2F` from a real path separator, so a path-safety test can pass an unsafe builder. Assert `url.pathname` directly to pin the encoded output.</comment>

<file context>
@@ -329,13 +340,265 @@ If any tool lists, searches, exports, imports, downloads, uploads, paginates, ba
+equality, and pin the output with `toBe`:
+
+```typescript
+expect(decodeURIComponent(url.pathname)).toBe('<the exact expected path>')
+
+let message = ''
</file context>


`executeTool` wraps every `postProcess` call in a catch that logs and then restores the
pre-`postProcess` result (`apps/sim/tools/index.ts:1977` and `:2062`). For a submit-then-poll tool
that pre-`postProcess` result is the **submit** response — `success: true` with every result field

cubic-dev-ai Bot Aug 29, 2026
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

P3: The fallback submit result does not have every field null: id remains populated. Describe email and qualification as null while preserving the non-null job id, so failure-path tests do not encode the wrong executor result shape.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .agents/skills/validate-integration/SKILL.md, line 578:

<comment>The fallback submit result does not have every field null: `id` remains populated. Describe `email` and `qualification` as null while preserving the non-null job id, so failure-path tests do not encode the wrong executor result shape.</comment>

<file context>
@@ -329,13 +340,265 @@ If any tool lists, searches, exports, imports, downloads, uploads, paginates, ba
+
+`executeTool` wraps every `postProcess` call in a catch that logs and then restores the
+pre-`postProcess` result (`apps/sim/tools/index.ts:1977` and `:2062`). For a submit-then-poll tool
+that pre-`postProcess` result is the **submit** response — `success: true` with every result field
+null. So a `postProcess` that throws on a timed-out or exhausted poll is reported to the user as a
+successful lookup that simply found nothing — and that stale success is also what reaches the
</file context>

Copy link
Copy Markdown
Collaborator Author

Closing for now — not because of a defect. This batch grew to 17 PRs across ~700 changed call sites, and we would rather revisit it as smaller, independently testable pieces than merge this much at once.

Nothing here is lost: the branch docs/integration-authoring-learnings is preserved and this PR can be reopened. Review state, the reasoning on every thread, and the red-first verification all stay attached.

waleedlatif1 deleted the docs/integration-authoring-learnings branch August 29, 2026 07:16
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant


Back | FazBrowse Home | New Git URL