Muyue 97a25295fc feat(browser-test): persistent token + auto-reconnect + screenshot + smarter strategy (v0.7.9) ...

Four user-reported issues with the AI-driven browser test feature:

1. WS dies on every page reload / navigation (token was single-use,
   5 min TTL → AI loses session permanently).
2. AI can't see new pages opened by its actions (same root cause).
3. No screenshot capability — AI cannot capture page state visually.
4. AI burns ~150 tool calls for what's 5 human actions, mostly
   list_clickables loops.

Fixes:

- Token sliding TTL (60 min): ConsumeToken no longer deletes the
  token; it refreshes its expiration on each successful WS connect.
  Same token survives reload / re-paste / navigation as long as
  there's no 60-min idle gap.

- Snippet auto-reconnect: WS onclose schedules reconnect with
  500ms × attempt backoff (max ~2.5s). Handles transient drops,
  server restarts, and WS hiccups without user intervention. Full
  navigation kills the JS context and is unrecoverable from JS — but
  the user just re-pastes the snippet, same token works.

- New 'screenshot' action: snippet captures via SVG foreignObject +
  canvas → base64 PNG → sent back over the existing WS reply
  channel. Server decodes and writes to ~/.muyue/screenshots/
  <filename>.png (sanitized name, timestamp default). Filename
  characters limited to a safe charset to prevent path escape.
  Best-effort: external CSS / cross-origin images / iframes won't
  inline.

- Studio system prompt rewritten <browser_test_strategy>:
  - Explicit rule: don't list_clickables after every click
  - Action cost table (cheap vs expensive)
  - When to re-list (URL change, dialog, click-not-found only)
  - Standard final report format ✓ / ✗ / ⚠ / 📸

Also bundles v0.7.8 (cherry-picked): unsafe.Pointer(uintptr(hPC))
instead of unsafe.Pointer(&hPC) in UpdateProcThreadAttribute, so
PROC_THREAD_ATTRIBUTE_PSEUDOCONSOLE is correctly applied and the
spawned shell attaches to the embedded xterm.js instead of opening
a separate external console window (regression fix from v0.7.6).

- internal/api/browsertest.go: token sliding, screenshot save,
  param schema, snippet rewrite, helpers
- internal/agent/prompts/studio_system.md: strategy rewrite
- internal/version/version.go: 0.7.7 → 0.7.9
- CHANGELOG.md: v0.7.9 entry covering all fixes

2026-04-27 16:50:04 +02:00

26 KiB

Raw Blame History

View Raw