Hydrofoil

Requirements

Turn tickets into testable criteria, then check each one against the code and the running app.

  1. 01Acceptance criteria generationTurns the ticket, the PR description and the diff into a numbered list of testable criteria.AI
  2. 02Acceptance criteria verificationChecks each criterion against both the code and the running app, and flags changes no ticket explains.AI
MinorHigh confidence

Ticket has no acceptance criteria

PAY-142 describes the goal but lists nothing testable. Hydrofoil drafted 4 criteria from the description, a comment and the diff.

Source: PAY-142 description
Example finding Dashboard GitHub check
MajorHigh confidence

Criterion not met: expired codes are accepted

Applying an expired promo code lowered the order total instead of showing an expiry error.

3 browser steps and a screenshot
Example finding Dashboard GitHub check
01 / 02

Acceptance criteria generation

Turns the ticket, the PR description and the diff into a numbered list of testable criteria.

Type
AI
Status
[TBD: Live / Beta / Coming soon]
Default
[TBD: On / Off]
Est. cost
[TBD: X credits per run]

What it catches

  • Tickets with unclear or missing acceptance criteria
  • Requirements buried in comments or subtasks
  • PRs that have no ticket at all
  • Behaviour the diff implies that nobody wrote down

Example finding

Ticket text on the left, the numbered criteria Hydrofoil drafts from it on the right. Each criterion carries a coloured tag showing where it came from.

Hydrofoil · Acceptance criteria
PAY-142 · Ticket

Apply promo codes at checkout

Customers can enter a promo code on the checkout page and see the discount applied to their total.1

Codes can expire. Invalid codes should show an error.

Comment · Removing a code should put the price back.
Criteria
  1. 1.

    Given a valid promo code, when I apply it, then the order total is reduced.

    Jira
  2. 2.

    Given an expired code, when I apply it, then I see an expiry error.

    Jira2
  3. 3.

    Given an invalid code, when I apply it, then I see an error and my total is unchanged.

    PR
  4. 4.

    Given a code is applied, when I remove it, then the original total is restored.

    inferred3
  1. 1
    Source textEach criterion links back to the ticket field, PR description line or diff hunk it came from.
  2. 2
    Source tagsEvery criterion is tagged Jira, PR or inferred.
  3. 3
    Inferred criteriaHydrofoil's best reading of the diff, not a requirement.

How it works

Hydrofoil reads the Jira ticket (description, acceptance-criteria field, comments, subtasks) or GitHub issue, the PR description and the diff. It writes numbered, Given/When/Then-style criteria and tags each with its source: Jira, PR, or inferred. You can edit, add or remove criteria in the dashboard, and your edits persist across runs. Criteria can optionally be posted back to Jira as a comment. [TBD: confirm Jira write-back is live]

  1. 01

    Read the sources

    Ticket or issue, PR description and the diff.

  2. 02

    Draft criteria

    Numbered, Given/When/Then-style and testable.

  3. 03

    Tag each source

    Jira, PR or inferred.

  4. 04

    Edit and keep

    Your edits persist across runs.

Output

Findings appear in the Hydrofoil dashboard and in your GitHub check, with a severity (blocker, major, minor or nit) and a confidence level. Low-confidence findings are marked Needs reviewand don't fail the check.

A numbered criteria list on the PR page, each tagged with its source.

Severity
BlockerMajorMinorNit
Confidence
HighMediumLow · Needs reviewDoesn't fail the check
Where it appears
Hydrofoil dashboard GitHub check

Evidence

What each finding includes.

  • A link to the source text, whether a ticket field, a PR description line, or a diff hunk.

Known limits

  • Criteria tagged inferred are Hydrofoil's best reading of the diff, not a requirement.
  • A vague ticket produces vague criteria. Edit them to sharpen the review.
  • Reads Jira Cloud and GitHub Issues. [TBD: Linear and Jira Data Center not supported]

Configuration

Turn this review on or off for each repository in repository settings.Choose which Jira projects to read and which field holds acceptance criteria [TBD: confirm labels].

02 / 02

Acceptance criteria verification

Checks each criterion against both the code and the running app, and flags changes no ticket explains.

Type
AI
Status
[TBD: Live / Beta / Coming soon]
Default
[TBD: On / Off]
Est. cost
[TBD: X credits per run]

What it catches

  • Criteria the implementation doesn't meet
  • Criteria that are only partly met
  • Scope creep: changes no ticket or criterion explains
  • Criteria that can't be proven on the preview

Example finding

Each criterion gets a verdict. Expand one to see the browser steps and screenshot behind it.

Hydrofoil · Criteria verdicts
  • 1.

    A valid promo code reduces the order total

    Met1
  • 2.

    An expired code shows an expiry error

    Not met
    Browser steps
    1. 1. Open checkout with a cart
    2. 2. Enter an expired code
    3. 3. Select Apply

    The total dropped instead of showing an error.

    2
  • 3.

    An invalid code shows an error and leaves the total unchanged

    Partially met
  • 4.

    Removing a code restores the original total

    Unable to verify
Unexplained changesrc/lib/analytics.ts changes tracking. No ticket or criterion explains it.
3
  1. 1
    Verdict per criterionMet, Partially met, Not met or Unable to verify.
  2. 2
    Runtime evidenceBrowser steps and a screenshot show what happened on the preview.
  3. 3
    Unexplained changesChanges that no ticket or criterion accounts for.

How it works

For each criterion, Hydrofoil reasons about the diff, then the browser agent tries it in the running preview. Each criterion gets one of four results: Met, Partially met, Not met, Unable to verify.

  1. 01

    Reason about the diff

    Check the code against each criterion.

  2. 02

    Try it in the preview

    The browser agent exercises the running app.

  3. 03

    Give a verdict

    Met, Partially met, Not met or Unable to verify.

  4. 04

    Flag unexplained changes

    List changes no ticket explains.

Output

Findings appear in the Hydrofoil dashboard and in your GitHub check, with a severity (blocker, major, minor or nit) and a confidence level. Low-confidence findings are marked Needs reviewand don't fail the check.

A verdict per criterion, plus a list of unexplained changes. Not-met criteria become findings.

Severity
BlockerMajorMinorNit
Confidence
HighMediumLow · Needs reviewDoesn't fail the check
Where it appears
Hydrofoil dashboard GitHub check

Evidence

What each finding includes.

  • Browser steps and a screenshot for runtime checks; code context for static checks.

Known limits

  • Unable to verify means the preview couldn't prove it, such as missing seed data, a missing persona or a backend behaviour the UI doesn't show. It does not mean "failed".
  • Backend behaviour is only visible through the UI.
  • Scope-creep flags can include legitimate refactors. [TBD: confirm handling]

Configuration

Turn this review on or off for each repository in repository settings.[TBD: add any check-specific options.]

Related reviews

See it on your next pull request

Hydrofoil opens each preview in a real browser and posts what it finds to GitHub.

Review your first PR free See how it works