Skip to content

feat(smol): @e2e-dev/smol, browsers and the app under test in branchable smol machines microVMs - #956

Open
BinSquare wants to merge 1 commit into
tester-army:mainfrom
BinSquare:smol-browsers
Open

BinSquare wants to merge 1 commit into
tester-army:mainfrom
BinSquare:smol-browsers

Conversation

@BinSquare

@BinSquare BinSquare commented Oct 7, 2026 •

Copy link
Copy Markdown
  • Adds @e2e-dev/smol, a BrowserProvider that runs Chromium in smol machines microVMs on the runner's own computer (macOS on Apple Silicon, Linux with KVM) through the smolmachines SDK. No service account or key.
  • Each worker slot boots one warm browser machine. With the default scope: 'attempt', every attempt runs in a copy-on-write branch of that running machine (Chromium already up, under a second), deleted when the attempt ends.
  • prepare(cdpEndpoint) drives the warm browser once before it is branched, so every attempt starts signed in. It stands in for sessions, which a per-attempt lease cannot use.
  • app runs the app under test inside the same machine instead of on the host. Its code no longer runs with the user's permissions (the gap docs/security.mdx names), and each attempt's branch holds its own copy of the running app and its data, so parallel tests and explorers stop sharing one database (the setup docs/bug-bash.mdx works around).
  • Lives entirely behind the web engine's provider seam: no change to e2e or @e2e-dev/web. Docs page at docs/integrations/smol.mdx.
web({
  browser: smol({
    app: { source: '.', setup: 'apk add --no-cache nodejs npm && npm ci', start: 'npm start', port: 3000 },
    prepare: async (cdpEndpoint) => { /* sign in once */ },
  }),
})
// with app: { url: 'http://localhost:3000' }
flowchart LR
  W[worker slot] -->|first attempt| B[boot machine: Chromium + app, prepare]
  B -->|every attempt| C1[branch: own browser, app, data]
  B --> C2[branch]
  C1 -->|attempt ends| D[delete branch]
Loading

Verified

Ran it locally: yes (macOS, Apple Silicon, smolmachines 1.22.2)

  • Testbed with the testbed app moved inside the machines (app: { source: 'app', start: 'node server.mjs', port: 4271 }), e2e run tests/forms.e2e.ts tests/board.e2e.ts tests/controls.e2e.ts, 2 workers: 23/23 passed; branches took 0.7 to 1.6 s after each slot's first boot; the runner's sweep left no machine behind.
  • Isolation: a todo app that stores its list on disk, three tests that each expect an empty list and add one item:
one app on the host (app.command) app inside smol, branched per attempt
test 1 ✓, test 2 × (sees test 1's todo), test 3 × test 1 ✓, test 2 ✓, test 3 ✓
  • prepare setting a cookie: every attempt saw it, and a cookie one attempt set never reached the next.
  • scope: 'worker' runs (one machine per slot, booted in prepare); prepare with it is refused at config load, since each attempt there gets a new context.

Review in cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

11 issues found across 24 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/smol/README.md">

<violation number="1" location="packages/smol/README.md:40">
P2: `scope: 'worker'` reuses the worker machine and app disk across attempts, so this isolation promise does not hold for that supported scope; qualify it as applying only to `scope: 'attempt'`.</violation>
</file>

<file name="docs/integrations/smol.mdx">

<violation number="1" location="docs/integrations/smol.mdx:12">
P2: These guarantees apply only to the default `'attempt'` scope; `'worker'` makes no branches and shares app data across attempts on a slot. Qualify the introduction so users do not rely on isolation that worker scope does not provide.</violation>

<violation number="2" location="docs/integrations/smol.mdx:15">
P2: This guarantee also covers externally hosted state, but copy-on-write branches isolate only the machine; attempts using a shared remote database can still see each other's changes. Limit the guarantee to data stored on the machine and tell users to isolate external services separately.</violation>

<violation number="3" location="docs/integrations/smol.mdx:65">
P2: This promises sub-second startup without the known caveat: smolmachines versions through 1.23.7 can intermittently fail sub-second timeouts because of published-port latency. Qualify the timing and document the affected versions so users do not rely on it as a guarantee.</violation>

<violation number="4" location="docs/integrations/smol.mdx:76">
P2: The `prepare` example imports `playwright-core`, but none of the install commands add it as a project dependency; pnpm users cannot import `@e2e-dev/web`'s transitive dependency from their config. Add `playwright-core` at the web engine's pinned version to the documented install commands.</violation>

<violation number="5" location="docs/integrations/smol.mdx:188">
P3: Attempt-scope machines are nonpersistent and the smol engine deletes them when their worker exits; only worker-scope machines are deliberately left for the runner's sweep. Qualify this cleanup claim as worker-scope behavior.</violation>
</file>

<file name="packages/smol/src/index.ts">

<violation number="1" location="packages/smol/src/index.ts:4">
P3: This summary says every attempt gets a branch, but `scope: 'worker'` keeps one browser per worker without branching. Qualify branching as the default behavior.</violation>
</file>

<file name="packages/smol/src/provider.ts">

<violation number="1" location="packages/smol/src/provider.ts:225">
P2: Alpine's BusyBox `wget` does not use GNU Wget's exit status 8 for HTTP errors, so a healthy app returning 404 at `/` is treated as unready and boot fails after 180 seconds. Use a probe that accepts HTTP error responses with the image's available tools, or install a client with that behavior.</violation>

<violation number="2" location="packages/smol/src/provider.ts:307">
P2: A worker-scope CDP reconnect reuses the original machine name, so `Machine.create` conflicts instead of creating a replacement. Give each acquisition a unique name while keeping the run/target prefix for sweeping.</violation>
</file>

<file name="README.md">

<violation number="1" location="README.md:62">
P2: `smol({ scope: 'worker' })` does not branch per attempt, so this summary overstates the isolation behavior. Qualify per-attempt branching as the default.</violation>
</file>

<file name="docs/integrations/index.mdx">

<violation number="1" location="docs/integrations/index.mdx:21">
P2: This description promises a signed-in browser and per-test branches for every configuration, but sign-in requires optional `prepare` and `scope: 'worker'` disables branching. Qualify both as optional/default behavior so users do not expect capabilities their configuration does not provide.</violation>
</file>

Reply to a comment to ask cubic a question or push back. It learns from your replies.

Turn on auto-fix | Re-trigger cubic

Comment thread packages/smol/README.md
ports on your computer's loopback the browser reaches as its own `localhost`.
`setup` runs a shell script in the machine before Chromium starts.
`app: { source, setup, start, port }` runs the app under test inside the
machine instead of on your computer, so every attempt also gets its own copy

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: scope: 'worker' reuses the worker machine and app disk across attempts, so this isolation promise does not hold for that supported scope; qualify it as applying only to scope: 'attempt'.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/smol/README.md, line 40:

<comment>`scope: 'worker'` reuses the worker machine and app disk across attempts, so this isolation promise does not hold for that supported scope; qualify it as applying only to `scope: 'attempt'`.</comment>

<file context>
@@ -0,0 +1,48 @@
+ports on your computer's loopback the browser reaches as its own `localhost`.
+`setup` runs a shell script in the machine before Chromium starts.
+`app: { source, setup, start, port }` runs the app under test inside the
+machine instead of on your computer, so every attempt also gets its own copy
+of the running app and its data.
+`scope: 'worker'` keeps one browser machine per worker slot instead.
</file context>

[browser provider](/browser#hosted-browsers) for
[smol machines](https://smolmachines.com) microVMs on the computer the run
starts on. Each worker slot boots one Chromium machine, and every test
attempt runs in a copy-on-write branch of that running browser, made in under

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: These guarantees apply only to the default 'attempt' scope; 'worker' makes no branches and shares app data across attempts on a slot. Qualify the introduction so users do not rely on isolation that worker scope does not provide.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/integrations/smol.mdx, line 12:

<comment>These guarantees apply only to the default `'attempt'` scope; `'worker'` makes no branches and shares app data across attempts on a slot. Qualify the introduction so users do not rely on isolation that worker scope does not provide.</comment>

<file context>
@@ -0,0 +1,189 @@
+[browser provider](/browser#hosted-browsers) for
+[smol machines](https://smolmachines.com) microVMs on the computer the run
+starts on. Each worker slot boots one Chromium machine, and every test
+attempt runs in a copy-on-write branch of that running browser, made in under
+a second and deleted when the attempt ends. With `app`, the app under test
+runs in the same machine, so it is sandboxed away from your computer and
</file context>

scope,
async acquire(request: BrowserRequest): Promise<BrowserLease> {
if (scope === 'worker') {
const name = `${prefixFor(request.runId, request.targetName)}-s${request.slot}`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: A worker-scope CDP reconnect reuses the original machine name, so Machine.create conflicts instead of creating a replacement. Give each acquisition a unique name while keeping the run/target prefix for sweeping.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/smol/src/provider.ts, line 307:

<comment>A worker-scope CDP reconnect reuses the original machine name, so `Machine.create` conflicts instead of creating a replacement. Give each acquisition a unique name while keeping the run/target prefix for sweeping.</comment>

<file context>
@@ -0,0 +1,360 @@
+    scope,
+    async acquire(request: BrowserRequest): Promise<BrowserLease> {
+      if (scope === 'worker') {
+        const name = `${prefixFor(request.runId, request.targetName)}-s${request.slot}`;
+        const { endpoint } = await boot(request, name);
+        request.log(`browser machine ${name}`);
</file context>

Comment thread README.md
| [`@e2e-dev/github`](https://www.npmjs.com/package/@e2e-dev/github) | Reporter that posts results as a pull request comment. |
| [`@e2e-dev/kernel`](https://www.npmjs.com/package/@e2e-dev/kernel) | Kernel hosted browsers for the web engine. |
| [`@e2e-dev/eas`](https://www.npmjs.com/package/@e2e-dev/eas) | EAS Simulators hosted iOS simulators and Android emulators for the mobile engine. |
| [`@e2e-dev/smol`](https://www.npmjs.com/package/@e2e-dev/smol) | smol machines browsers for the web engine: each attempt branches a warm Chromium microVM. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: smol({ scope: 'worker' }) does not branch per attempt, so this summary overstates the isolation behavior. Qualify per-attempt branching as the default.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At README.md, line 62:

<comment>`smol({ scope: 'worker' })` does not branch per attempt, so this summary overstates the isolation behavior. Qualify per-attempt branching as the default.</comment>

<file context>
@@ -59,6 +59,7 @@ suite.
 | [`@e2e-dev/github`](https://www.npmjs.com/package/@e2e-dev/github) | Reporter that posts results as a pull request comment. |
 | [`@e2e-dev/kernel`](https://www.npmjs.com/package/@e2e-dev/kernel) | Kernel hosted browsers for the web engine. |
 | [`@e2e-dev/eas`](https://www.npmjs.com/package/@e2e-dev/eas) | EAS Simulators hosted iOS simulators and Android emulators for the mobile engine. |
+| [`@e2e-dev/smol`](https://www.npmjs.com/package/@e2e-dev/smol) | smol machines browsers for the web engine: each attempt branches a warm Chromium microVM. |
 | [`@e2e-dev/decision`](https://e2e.tester.army/docs/decision-models) | Decision-model executors for bounded semantic actions and assertions. |
 
</file context>
Suggested change
| [`@e2e-dev/smol`](https://www.npmjs.com/package/@e2e-dev/smol) | smol machines browsers for the web engine: each attempt branches a warm Chromium microVM. |
| [`@e2e-dev/smol`](https://www.npmjs.com/package/@e2e-dev/smol) | smol machines browsers for the web engine: by default, each attempt branches a warm Chromium microVM. |

`@e2e-dev/eas`: hosted iOS simulators and Android emulators for mobile targets.
</Card>
<Card title="smol machines" icon="/images/integrations/smol.png" href="/integrations/smol">
`@e2e-dev/smol`: Chromium microVMs on your computer, each test in a branch of a warm, signed-in browser, optionally with its own copy of the app and its database.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This description promises a signed-in browser and per-test branches for every configuration, but sign-in requires optional prepare and scope: 'worker' disables branching. Qualify both as optional/default behavior so users do not expect capabilities their configuration does not provide.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/integrations/index.mdx, line 21:

<comment>This description promises a signed-in browser and per-test branches for every configuration, but sign-in requires optional `prepare` and `scope: 'worker'` disables branching. Qualify both as optional/default behavior so users do not expect capabilities their configuration does not provide.</comment>

<file context>
@@ -17,6 +17,9 @@ integration decides where it comes from.
     `@e2e-dev/eas`: hosted iOS simulators and Android emulators for mobile targets.
   </Card>
+  <Card title="smol machines" icon="/images/integrations/smol.png" href="/integrations/smol">
+    `@e2e-dev/smol`: Chromium microVMs on your computer, each test in a branch of a warm, signed-in browser, optionally with its own copy of the app and its database.
+  </Card>
 </CardGroup>
</file context>
Suggested change
`@e2e-dev/smol`: Chromium microVMs on your computer, each test in a branch of a warm, signed-in browser, optionally with its own copy of the app and its database.
`@e2e-dev/smol`: Chromium microVMs on your computer, with per-attempt branches by default, optional signed-in browser preparation, and an optional copy of the app and its database.

every attempt on that slot starts:

```ts title="e2e.config.ts"
import { chromium } from 'playwright-core';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The prepare example imports playwright-core, but none of the install commands add it as a project dependency; pnpm users cannot import @e2e-dev/web's transitive dependency from their config. Add playwright-core at the web engine's pinned version to the documented install commands.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/integrations/smol.mdx, line 76:

<comment>The `prepare` example imports `playwright-core`, but none of the install commands add it as a project dependency; pnpm users cannot import `@e2e-dev/web`'s transitive dependency from their config. Add `playwright-core` at the web engine's pinned version to the documented install commands.</comment>

<file context>
@@ -0,0 +1,189 @@
+every attempt on that slot starts:
+
+```ts title="e2e.config.ts"
+import { chromium } from 'playwright-core';
+
+smol({
</file context>

attempt runs in a copy-on-write branch of that running browser, made in under
a second and deleted when the attempt ends. With `app`, the app under test
runs in the same machine, so it is sandboxed away from your computer and
every attempt gets its own copy of it and its data. Tests stay the same; the

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This guarantee also covers externally hosted state, but copy-on-write branches isolate only the machine; attempts using a shared remote database can still see each other's changes. Limit the guarantee to data stored on the machine and tell users to isolate external services separately.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/integrations/smol.mdx, line 15:

<comment>This guarantee also covers externally hosted state, but copy-on-write branches isolate only the machine; attempts using a shared remote database can still see each other's changes. Limit the guarantee to data stored on the machine and tell users to isolate external services separately.</comment>

<file context>
@@ -0,0 +1,189 @@
+attempt runs in a copy-on-write branch of that running browser, made in under
+a second and deleted when the attempt ends. With `app`, the app under test
+runs in the same machine, so it is sandboxed away from your computer and
+every attempt gets its own copy of it and its data. Tests stay the same; the
+config changes only in the target's `browser` and `app.url`.
+
</file context>

...(app.setup === undefined ? [] : [app.setup]),
detached(`sh -c ${shellQuote(app.start)}`, APP_LOG),
// Any HTTP answer counts, as for app.command's readyUrl: wget exits 8 on a 4xx or 5xx.
`ready=; for i in $(seq ${APP_READY_SECONDS * 4}); do rc=0; wget -q -O /dev/null -T 1 http://127.0.0.1:${app.port}/ >/dev/null 2>&1 || rc=$?; if [ $rc -eq 0 ] || [ $rc -eq 8 ]; then ready=1; break; fi; sleep 0.25; done`,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Alpine's BusyBox wget does not use GNU Wget's exit status 8 for HTTP errors, so a healthy app returning 404 at / is treated as unready and boot fails after 180 seconds. Use a probe that accepts HTTP error responses with the image's available tools, or install a client with that behavior.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/smol/src/provider.ts, line 225:

<comment>Alpine's BusyBox `wget` does not use GNU Wget's exit status 8 for HTTP errors, so a healthy app returning 404 at `/` is treated as unready and boot fails after 180 seconds. Use a probe that accepts HTTP error responses with the image's available tools, or install a client with that behavior.</comment>

<file context>
@@ -0,0 +1,360 @@
+    ...(app.setup === undefined ? [] : [app.setup]),
+    detached(`sh -c ${shellQuote(app.start)}`, APP_LOG),
+    // Any HTTP answer counts, as for app.command's readyUrl: wget exits 8 on a 4xx or 5xx.
+    `ready=; for i in $(seq ${APP_READY_SECONDS * 4}); do rc=0; wget -q -O /dev/null -T 1 http://127.0.0.1:${app.port}/ >/dev/null 2>&1 || rc=$?; if [ $rc -eq 0 ] || [ $rc -eq 8 ]; then ready=1; break; fi; sleep 0.25; done`,
+    `[ -n "$ready" ] || { echo "the app did not answer on port ${app.port} within ${APP_READY_SECONDS}s:" >&2; tail -20 ${APP_LOG} >&2; exit 1; }`,
+  ].join('\n');
</file context>

## Cleanup

Machines are named after the run and the target. A worker that dies leaves
its browsers behind, and the runner deletes them when the run finishes, after

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: Attempt-scope machines are nonpersistent and the smol engine deletes them when their worker exits; only worker-scope machines are deliberately left for the runner's sweep. Qualify this cleanup claim as worker-scope behavior.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/integrations/smol.mdx, line 188:

<comment>Attempt-scope machines are nonpersistent and the smol engine deletes them when their worker exits; only worker-scope machines are deliberately left for the runner's sweep. Qualify this cleanup claim as worker-scope behavior.</comment>

<file context>
@@ -0,0 +1,189 @@
+## Cleanup
+
+Machines are named after the run and the target. A worker that dies leaves
+its browsers behind, and the runner deletes them when the run finishes, after
+the last worker exits. `smol machine ls` lists anything left over.
</file context>

/**
* `@e2e-dev/smol` public surface: `smol()`, a browser provider that runs
* Chromium in smol machines microVMs for `@e2e-dev/web`, branching a warm
* browser for every test attempt.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This summary says every attempt gets a branch, but scope: 'worker' keeps one browser per worker without branching. Qualify branching as the default behavior.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/smol/src/index.ts, line 4:

<comment>This summary says every attempt gets a branch, but `scope: 'worker'` keeps one browser per worker without branching. Qualify branching as the default behavior.</comment>

<file context>
@@ -0,0 +1,8 @@
+/**
+ * `@e2e-dev/smol` public surface: `smol()`, a browser provider that runs
+ * Chromium in smol machines microVMs for `@e2e-dev/web`, branching a warm
+ * browser for every test attempt.
+ */
+
</file context>
Suggested change
* browser for every test attempt.
* browser per test attempt by default.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant