feat: AI-driven browser tests — Tests tab + browser_test agent tool
Some checks failed
PR Check / check (pull_request) Failing after 33s

New feature: give Studio's AI control of any browser tab to test buttons,
read the console, and report which buttons work / fail.

Backend (internal/api/browser_test.go, ~480 LOC):
- WebSocket endpoint /api/ws/browser-test, auth by single-use 5-min token
- BrowserTestStore: session map (capped at 16, LRU evict), token store
- REST: /api/test/snippet (issues token + JS snippet), /api/test/sessions,
  /api/test/console/{id}
- Agent tool 'browser_test' wired into the registry, with actions:
  list_clickables / click / eval / console / current_url / type / wait /
  summary. click returns the console_delta produced during the click.
- Embedded JS runner: opens WS, hooks console + window.onerror +
  unhandledrejection, dispatches dispatcher commands, replies with
  correlation IDs, watches for URL changes.

Frontend:
- New Tests tab (web/src/components/Tests.jsx): snippet copy + connected
  sessions list + live console viewer
- App.jsx: 5th tab + Ctrl+4 shortcut (Config moves to Ctrl+5)
- api/client.js: getTestSnippet / getTestSessions / getTestConsole

Studio prompt:
- internal/agent/prompts/studio_system.md: added browser_test entry to the
  tools table + <browser_test_strategy> section explaining the recommended
  loop (summary → list_clickables → click → check console_delta → report)

Versioning:
- v0.6.0 → v0.7.0
- CHANGELOG.md: full entry under v0.7.0
This commit is contained in:
Muyue
2026-04-27 11:02:05 +02:00
parent 6a7b4d8001
commit c820d55710
8 changed files with 914 additions and 3 deletions

View File

@@ -4,6 +4,26 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
## v0.7.0
### Nouvelle fonctionnalité majeure : Tests pilotés par l'IA
- **Onglet Tests** dédié dans l'UI : génère un snippet JS à coller dans n'importe quelle page web ouverte (Chrome, Firefox, Edge, dev local ou distant).
- **Session WebSocket** authentifiée par token à usage unique (5 min TTL) — la page connectée transmet ses messages console en temps réel et expose une RPC pour cliquer / évaluer / inspecter.
- **Outil agent `browser_test`** disponible pour Studio, avec actions :
- `list_clickables` : énumère tous les éléments cliquables visibles avec un index stable
- `click` : clic par sélecteur CSS ou par index — retourne le **delta console** émis pendant le clic
- `eval` : évalue une expression JS et retourne sa valeur sérialisée
- `console` / `summary` : lit le buffer console (200 dernières entrées)
- `current_url` : URL et titre courants
- `type` : remplit un champ input/textarea (utilise le setter natif pour compatibilité React)
- `wait` : pause asynchrone (max 5s)
- **Stratégie BMAD** intégrée au prompt système Studio : boucle `summary → list_clickables → click → vérifier console_delta → rapport final ✓/✗/⚠`.
- **Multi-sessions** : jusqu'à 16 onglets connectés simultanément ; éviction LRU au-delà.
- **Sécurité** : token consommé à la première connexion ; CheckOrigin libre côté snippet (gating par token uniquement) ; CORS API REST inchangé.
- **Backend** : `internal/api/browser_test.go` (nouveau, ~480 lignes) + 4 routes (`/api/test/snippet`, `/api/test/sessions`, `/api/test/console/{id}`, `/api/ws/browser-test`).
- **Frontend** : `web/src/components/Tests.jsx` (nouveau) + nouvel onglet ⌃4.
## v0.6.0
### Audit & corrections (sécurité, concurrence, stabilité)

View File

@@ -54,6 +54,7 @@ Muyue gère :
|-------|-------|
| **terminal** | Exécuter des commandes shell (builds, tests, git, etc.) |
| **crush_run** | Déléguer une tâche complexe à Crush (édition de fichiers, refactoring, debug) — préfère cet outil pour les tâches multi-fichiers ou l'écriture de code |
| **claude_run** | Déléguer une tâche complexe à Claude Code CLI |
| **read_file** | Lire le contenu d'un fichier |
| **list_files** | Lister les fichiers d'un répertoire |
| **search_files** | Chercher des fichiers par motif (glob) |
@@ -62,6 +63,27 @@ Muyue gère :
| **set_provider** | Configurer un fournisseur IA |
| **manage_ssh** | Gérer les connexions SSH |
| **web_fetch** | Récupérer le contenu d'une URL |
| **browser_test** | Piloter un onglet de navigateur de l'utilisateur (clic, eval, lecture console) — voir `<browser_test_strategy>` ci-dessous |
<browser_test_strategy>
Quand l'utilisateur demande de **tester** une UI / une page (ses boutons, ses formulaires, son comportement), utilise `browser_test`. La page cible doit déjà être connectée via le snippet de l'onglet "Tests" — sinon, l'outil te le dira et tu demandes à l'utilisateur de coller le snippet.
Boucle recommandée :
1. `browser_test` action `summary` — voir l'URL, le titre et les dernières erreurs console déjà présentes.
2. `browser_test` action `list_clickables` — récupérer la liste indexée des boutons / liens / inputs cliquables.
3. Pour chaque cible : `browser_test` action `click` (avec `index` ou `selector`).
4. Immédiatement après chaque clic, **regarde le `console_delta` retourné** : c'est la liste des messages console émis pendant le clic. `level: "error"` = bouton cassé.
5. Vérifie aussi `current_url` retourné — un changement d'URL inattendu peut signaler un bug.
6. Si l'élément ouvre un dialog ou modifie le DOM, refais `list_clickables` pour découvrir les nouveaux éléments.
7. Pour les inputs : utilise `type` avant `click` sur le bouton de soumission.
8. À la fin, fournis un **rapport** structuré : ✓ boutons OK / ✗ boutons cassés (avec le message d'erreur exact) / ⚠ boutons disabled ou non trouvés.
Astuces :
- Préfère cliquer **par `index`** que par sélecteur — le sélecteur change avec le DOM, l'index reste stable jusqu'au prochain `list_clickables`.
- Entre deux actions sensibles, `wait` 200-500 ms si la page a des transitions / fetches asynchrones.
- N'utilise jamais `eval` pour cliquer si `click` suffit.
</browser_test_strategy>
<tool_strategy>
- **Recherche avant action** — Utilise `search_files`, `grep_content`, `read_file` avant de supposer quoi que ce soit sur l'état du système

View File

@@ -0,0 +1,612 @@
package api
// Browser-test feature: an out-of-process page (the user's target tab)
// connects to Muyue via WebSocket using a short-lived token, and exposes a
// thin RPC: Studio's AI can list clickable elements, click them, evaluate JS,
// read the recent console buffer, and observe what changes after each action.
//
// Threat model: an injected snippet runs in the user's chosen page only, with
// the same origin as that page; the WS endpoint is bound to localhost and
// gated by a 5-minute token issued by the local Muyue server.
import (
"context"
"crypto/rand"
"encoding/hex"
"encoding/json"
"fmt"
"net/http"
"strings"
"sync"
"time"
"github.com/gorilla/websocket"
"github.com/muyue/muyue/internal/agent"
)
const (
browserTestTokenTTL = 5 * time.Minute
browserTestCommandTTL = 30 * time.Second
browserTestConsoleMax = 200
browserTestSessionsMax = 16
)
// BrowserTestSession represents one connected browser tab.
type BrowserTestSession struct {
ID string
URL string
Title string
conn *websocket.Conn
mu sync.Mutex
console []ConsoleEntry
pending map[string]chan json.RawMessage
pendingMu sync.Mutex
connectedAt time.Time
writeMu sync.Mutex
}
// ConsoleEntry is a captured console message from the connected page.
type ConsoleEntry struct {
Level string `json:"level"` // log, info, warn, error, debug
Message string `json:"message"`
Time string `json:"time"`
}
// BrowserTestStore manages active sessions + pending one-shot connect tokens.
type BrowserTestStore struct {
mu sync.RWMutex
sessions map[string]*BrowserTestSession
tokens map[string]time.Time
tokensMu sync.Mutex
}
func NewBrowserTestStore() *BrowserTestStore {
return &BrowserTestStore{
sessions: map[string]*BrowserTestSession{},
tokens: map[string]time.Time{},
}
}
// IssueToken creates a single-use token used by the snippet to authenticate.
func (s *BrowserTestStore) IssueToken() string {
buf := make([]byte, 16)
if _, err := rand.Read(buf); err != nil {
return fmt.Sprintf("fallback-%d", time.Now().UnixNano())
}
tok := hex.EncodeToString(buf)
s.tokensMu.Lock()
now := time.Now()
for k, v := range s.tokens {
if now.Sub(v) > browserTestTokenTTL {
delete(s.tokens, k)
}
}
s.tokens[tok] = now
s.tokensMu.Unlock()
return tok
}
// ConsumeToken validates and removes a token in one step.
func (s *BrowserTestStore) ConsumeToken(tok string) bool {
s.tokensMu.Lock()
defer s.tokensMu.Unlock()
t, ok := s.tokens[tok]
if !ok {
return false
}
delete(s.tokens, tok)
return time.Since(t) <= browserTestTokenTTL
}
// Register inserts a new session, evicting the oldest if at capacity.
func (s *BrowserTestStore) Register(session *BrowserTestSession) {
s.mu.Lock()
defer s.mu.Unlock()
if len(s.sessions) >= browserTestSessionsMax {
var oldestID string
var oldest time.Time
for id, sess := range s.sessions {
if oldestID == "" || sess.connectedAt.Before(oldest) {
oldestID = id
oldest = sess.connectedAt
}
}
if old, ok := s.sessions[oldestID]; ok {
old.conn.Close()
delete(s.sessions, oldestID)
}
}
s.sessions[session.ID] = session
}
func (s *BrowserTestStore) Remove(id string) {
s.mu.Lock()
defer s.mu.Unlock()
if sess, ok := s.sessions[id]; ok {
sess.conn.Close()
delete(s.sessions, id)
}
}
func (s *BrowserTestStore) Get(id string) *BrowserTestSession {
s.mu.RLock()
defer s.mu.RUnlock()
return s.sessions[id]
}
// Pick returns the requested session by ID, or the most-recently-connected
// session if id is empty. Returns nil if no session matches.
func (s *BrowserTestStore) Pick(id string) *BrowserTestSession {
s.mu.RLock()
defer s.mu.RUnlock()
if id != "" {
return s.sessions[id]
}
var picked *BrowserTestSession
for _, sess := range s.sessions {
if picked == nil || sess.connectedAt.After(picked.connectedAt) {
picked = sess
}
}
return picked
}
func (s *BrowserTestStore) List() []map[string]interface{} {
s.mu.RLock()
defer s.mu.RUnlock()
out := make([]map[string]interface{}, 0, len(s.sessions))
for _, sess := range s.sessions {
out = append(out, map[string]interface{}{
"id": sess.ID,
"url": sess.URL,
"title": sess.Title,
"connected_at": sess.connectedAt.Format(time.RFC3339),
})
}
return out
}
// Send issues an RPC command to the browser session and waits up to TTL for
// the matching reply. Returns the raw payload or an error.
func (sess *BrowserTestSession) Send(action string, params map[string]interface{}) (json.RawMessage, error) {
cid := newCorrelationID()
ch := make(chan json.RawMessage, 1)
sess.pendingMu.Lock()
sess.pending[cid] = ch
sess.pendingMu.Unlock()
defer func() {
sess.pendingMu.Lock()
delete(sess.pending, cid)
sess.pendingMu.Unlock()
}()
cmd := map[string]interface{}{
"id": cid,
"action": action,
"params": params,
}
sess.writeMu.Lock()
err := sess.conn.WriteJSON(cmd)
sess.writeMu.Unlock()
if err != nil {
return nil, fmt.Errorf("write: %w", err)
}
select {
case payload := <-ch:
return payload, nil
case <-time.After(browserTestCommandTTL):
return nil, fmt.Errorf("browser session did not reply within %s", browserTestCommandTTL)
}
}
// AppendConsole records a console line, trimming to the buffer cap.
func (sess *BrowserTestSession) AppendConsole(level, message string) {
sess.mu.Lock()
defer sess.mu.Unlock()
sess.console = append(sess.console, ConsoleEntry{
Level: level,
Message: message,
Time: time.Now().Format(time.RFC3339),
})
if len(sess.console) > browserTestConsoleMax {
sess.console = sess.console[len(sess.console)-browserTestConsoleMax:]
}
}
// SnapshotConsole returns a copy of the current console buffer.
func (sess *BrowserTestSession) SnapshotConsole() []ConsoleEntry {
sess.mu.Lock()
defer sess.mu.Unlock()
out := make([]ConsoleEntry, len(sess.console))
copy(out, sess.console)
return out
}
func newCorrelationID() string {
buf := make([]byte, 8)
rand.Read(buf)
return hex.EncodeToString(buf)
}
// HTTP handlers --------------------------------------------------------------
func (s *Server) handleBrowserTestSnippet(w http.ResponseWriter, r *http.Request) {
if r.Method != "GET" {
writeError(w, "GET only", http.StatusMethodNotAllowed)
return
}
tok := s.browserTestStore.IssueToken()
host := r.Host
if host == "" {
host = "127.0.0.1"
}
scheme := "ws"
if r.TLS != nil {
scheme = "wss"
}
wsURL := fmt.Sprintf("%s://%s/api/ws/browser-test?token=%s", scheme, host, tok)
snippet := buildBrowserTestSnippet(wsURL)
writeJSON(w, map[string]interface{}{
"token": tok,
"ws_url": wsURL,
"snippet": snippet,
"expires_in": int(browserTestTokenTTL / time.Second),
})
}
func (s *Server) handleBrowserTestSessions(w http.ResponseWriter, r *http.Request) {
if r.Method != "GET" {
writeError(w, "GET only", http.StatusMethodNotAllowed)
return
}
writeJSON(w, map[string]interface{}{
"sessions": s.browserTestStore.List(),
})
}
func (s *Server) handleBrowserTestConsole(w http.ResponseWriter, r *http.Request) {
if r.Method != "GET" {
writeError(w, "GET only", http.StatusMethodNotAllowed)
return
}
id := strings.TrimPrefix(r.URL.Path, "/api/test/console/")
sess := s.browserTestStore.Pick(id)
if sess == nil {
writeError(w, "no active browser test session", http.StatusNotFound)
return
}
writeJSON(w, map[string]interface{}{
"session_id": sess.ID,
"url": sess.URL,
"console": sess.SnapshotConsole(),
})
}
// browserTestUpgrader accepts any origin: the connection is gated by a
// short-lived token issued to the local UI, not by Origin checking.
var browserTestUpgrader = websocket.Upgrader{
CheckOrigin: func(r *http.Request) bool { return true },
}
func (s *Server) handleBrowserTestWS(w http.ResponseWriter, r *http.Request) {
tok := r.URL.Query().Get("token")
if tok == "" || !s.browserTestStore.ConsumeToken(tok) {
writeError(w, "invalid or expired token", http.StatusUnauthorized)
return
}
conn, err := browserTestUpgrader.Upgrade(w, r, nil)
if err != nil {
return
}
conn.SetReadLimit(2 << 20)
// Read the hello message: page sends {"type":"hello","url":"...","title":"..."}.
conn.SetReadDeadline(time.Now().Add(10 * time.Second))
var hello struct {
Type string `json:"type"`
URL string `json:"url"`
Title string `json:"title"`
}
if err := conn.ReadJSON(&hello); err != nil || hello.Type != "hello" {
conn.WriteJSON(map[string]string{"type": "error", "message": "expected hello"})
conn.Close()
return
}
conn.SetReadDeadline(time.Time{})
id := newCorrelationID()
sess := &BrowserTestSession{
ID: id,
URL: hello.URL,
Title: hello.Title,
conn: conn,
pending: map[string]chan json.RawMessage{},
connectedAt: time.Now(),
}
s.browserTestStore.Register(sess)
defer s.browserTestStore.Remove(id)
// Acknowledge with the assigned session ID.
sess.writeMu.Lock()
conn.WriteJSON(map[string]string{"type": "registered", "session_id": id})
sess.writeMu.Unlock()
for {
_, raw, err := conn.ReadMessage()
if err != nil {
return
}
var msg struct {
Type string `json:"type"`
ID string `json:"id,omitempty"`
Level string `json:"level,omitempty"`
Text string `json:"text,omitempty"`
URL string `json:"url,omitempty"`
Data json.RawMessage `json:"data,omitempty"`
}
if err := json.Unmarshal(raw, &msg); err != nil {
continue
}
switch msg.Type {
case "console":
sess.AppendConsole(msg.Level, msg.Text)
case "url_change":
sess.mu.Lock()
sess.URL = msg.URL
sess.mu.Unlock()
case "reply":
sess.pendingMu.Lock()
ch, ok := sess.pending[msg.ID]
sess.pendingMu.Unlock()
if ok {
select {
case ch <- msg.Data:
default:
}
}
case "ping":
sess.writeMu.Lock()
conn.WriteJSON(map[string]string{"type": "pong"})
sess.writeMu.Unlock()
}
}
}
// Agent tool -----------------------------------------------------------------
// BrowserTestParams is the schema exposed to the AI for the browser_test tool.
type BrowserTestParams struct {
Action string `json:"action" description:"One of: list_clickables, click, eval, console, current_url, wait, type, summary"`
SessionID string `json:"session_id,omitempty" description:"Browser session id (optional, defaults to most recent)"`
Selector string `json:"selector,omitempty" description:"CSS selector for click/type actions"`
Index int `json:"index,omitempty" description:"Alternative to selector: index into the last list_clickables result (0-based)"`
Expr string `json:"expr,omitempty" description:"JS expression to evaluate (eval action only)"`
Text string `json:"text,omitempty" description:"Text to type (type action only)"`
WaitMs int `json:"wait_ms,omitempty" description:"Milliseconds to wait (wait action only, max 5000)"`
Tail int `json:"tail,omitempty" description:"Console action: how many recent lines to return (default 50, max 200)"`
}
// RegisterBrowserTestTool wires the agent tool against a session store.
func RegisterBrowserTestTool(reg *agent.Registry, store *BrowserTestStore) error {
tool, err := agent.NewTool("browser_test",
"Drive the user's connected browser tab for end-to-end testing. Available actions: list_clickables (returns indexed clickable elements), click (by selector or index), eval (run a JS expression and return result), console (read recent console output, ideal to spot errors after a click), current_url, wait (sleep ms before next check), type (set value on an input), summary (URL+title+last console entries). Always start with list_clickables; click; then console to verify no errors.",
func(ctx context.Context, p BrowserTestParams) (agent.ToolResponse, error) {
sess := store.Pick(p.SessionID)
if sess == nil {
return agent.TextErrorResponse("no active browser session — ask the user to paste the snippet from the Tests tab in their target page"), nil
}
action := strings.ToLower(strings.TrimSpace(p.Action))
switch action {
case "":
return agent.TextErrorResponse("action is required"), nil
case "list_clickables", "click", "eval", "current_url", "type":
case "console", "summary", "wait":
default:
return agent.TextErrorResponse("unknown action: " + p.Action), nil
}
if action == "console" {
tail := p.Tail
if tail <= 0 {
tail = 50
}
if tail > browserTestConsoleMax {
tail = browserTestConsoleMax
}
entries := sess.SnapshotConsole()
if len(entries) > tail {
entries = entries[len(entries)-tail:]
}
out, _ := json.MarshalIndent(map[string]interface{}{
"session_id": sess.ID,
"console": entries,
}, "", " ")
return agent.TextResponse(string(out)), nil
}
if action == "summary" {
entries := sess.SnapshotConsole()
if len(entries) > 20 {
entries = entries[len(entries)-20:]
}
out, _ := json.MarshalIndent(map[string]interface{}{
"session_id": sess.ID,
"url": sess.URL,
"title": sess.Title,
"recent_console": entries,
}, "", " ")
return agent.TextResponse(string(out)), nil
}
if action == "wait" {
ms := p.WaitMs
if ms <= 0 {
ms = 200
}
if ms > 5000 {
ms = 5000
}
select {
case <-ctx.Done():
return agent.TextErrorResponse("cancelled"), nil
case <-time.After(time.Duration(ms) * time.Millisecond):
}
return agent.TextResponse(fmt.Sprintf("waited %dms", ms)), nil
}
// Capture console snapshot length before so we can return only the delta
// after the action — useful so the AI can spot errors caused by the click.
pre := len(sess.SnapshotConsole())
params := map[string]interface{}{}
if p.Selector != "" {
params["selector"] = p.Selector
}
if p.Index > 0 || (action == "click" && p.Selector == "") {
params["index"] = p.Index
}
if p.Expr != "" {
params["expr"] = p.Expr
}
if p.Text != "" {
params["text"] = p.Text
}
payload, err := sess.Send(action, params)
if err != nil {
return agent.TextErrorResponse(err.Error()), nil
}
// Console delta: messages logged during this command.
post := sess.SnapshotConsole()
var delta []ConsoleEntry
if len(post) > pre {
delta = post[pre:]
}
result := map[string]interface{}{
"action": action,
"reply": json.RawMessage(payload),
"console_delta": delta,
"current_url": sess.URL,
}
out, _ := json.MarshalIndent(result, "", " ")
return agent.TextResponse(string(out)), nil
})
if err != nil {
return err
}
return reg.Register(tool)
}
// Snippet generator ----------------------------------------------------------
func buildBrowserTestSnippet(wsURL string) string {
// Note: this is the JS injected into the user's target page. It opens the
// WS, hooks console, and dispatches commands. Kept terse on purpose.
return `(function(){
if (window.__muyueTestRunner) { console.log('[Muyue] runner already attached'); return; }
var WS_URL = ` + jsString(wsURL) + `;
var ws = new WebSocket(WS_URL);
var lastList = [];
function send(obj){ try{ ws.send(JSON.stringify(obj)); }catch(e){} }
function reply(id, data){ send({type:'reply', id:id, data:data}); }
function safeText(el){
var t = (el.innerText || el.textContent || '').trim();
if (t.length > 80) t = t.slice(0,80)+'…';
return t;
}
function describe(el){
var sel = el.id ? '#'+el.id : el.tagName.toLowerCase();
if (!el.id && el.className && typeof el.className === 'string') {
sel += '.' + el.className.trim().split(/\s+/).slice(0,2).join('.');
}
var label = el.getAttribute('aria-label') || el.getAttribute('title') || el.getAttribute('name') || '';
return { tag: el.tagName.toLowerCase(), selector: sel, text: safeText(el), label: label, type: el.getAttribute('type')||'', disabled: !!el.disabled };
}
function list(){
var els = Array.from(document.querySelectorAll('button, a[href], input[type=submit], input[type=button], [role=button], [onclick]'));
lastList = els.filter(function(e){ var r=e.getBoundingClientRect(); return r.width>0 && r.height>0; });
return lastList.map(describe).map(function(d,i){ d.index = i; return d; });
}
function clickEl(el){
if (!el) return { ok:false, error:'element not found' };
if (el.disabled) return { ok:false, error:'element is disabled' };
try { el.scrollIntoView({block:'center'}); el.click(); return { ok:true }; }
catch(e){ return { ok:false, error:String(e) }; }
}
function dispatch(msg){
var p = msg.params || {};
switch(msg.action){
case 'list_clickables': return list();
case 'click': {
var el;
if (p.selector) el = document.querySelector(p.selector);
else if (typeof p.index === 'number') el = lastList[p.index];
return clickEl(el);
}
case 'eval': {
try { var r = (0,eval)(p.expr); return { ok:true, value: serialize(r) }; }
catch(e){ return { ok:false, error:String(e) }; }
}
case 'current_url': return { url: location.href, title: document.title };
case 'type': {
var el = p.selector ? document.querySelector(p.selector) : (lastList[p.index]);
if (!el) return { ok:false, error:'element not found' };
var proto = Object.getPrototypeOf(el);
var setter = Object.getOwnPropertyDescriptor(proto, 'value');
try { setter && setter.set ? setter.set.call(el, p.text||'') : (el.value = p.text||''); }
catch(e){ el.value = p.text||''; }
el.dispatchEvent(new Event('input', {bubbles:true}));
el.dispatchEvent(new Event('change', {bubbles:true}));
return { ok:true };
}
}
return { ok:false, error:'unknown action' };
}
function serialize(v){
if (v === undefined) return 'undefined';
try { return JSON.parse(JSON.stringify(v)); }
catch(e){ return String(v); }
}
['log','info','warn','error','debug'].forEach(function(lvl){
var orig = console[lvl];
console[lvl] = function(){
try {
var parts = Array.from(arguments).map(function(a){
if (typeof a === 'string') return a;
try { return JSON.stringify(a); } catch(e){ return String(a); }
});
send({type:'console', level: lvl, text: parts.join(' ')});
} catch(e){}
return orig.apply(console, arguments);
};
});
window.addEventListener('error', function(e){
send({type:'console', level:'error', text:'window.onerror: '+(e.message||e.error||'unknown')});
});
window.addEventListener('unhandledrejection', function(e){
send({type:'console', level:'error', text:'unhandledrejection: '+String(e.reason)});
});
var lastUrl = location.href;
setInterval(function(){
if (location.href !== lastUrl){ lastUrl = location.href; send({type:'url_change', url: lastUrl}); }
}, 500);
ws.onopen = function(){ send({type:'hello', url: location.href, title: document.title}); };
ws.onmessage = function(ev){
try { var msg = JSON.parse(ev.data); }
catch(e){ return; }
if (msg.type === 'registered') { console.log('[Muyue] connected — session', msg.session_id); return; }
if (msg.action) reply(msg.id, dispatch(msg));
};
ws.onclose = function(){ console.log('[Muyue] runner disconnected'); window.__muyueTestRunner = null; };
window.__muyueTestRunner = { ws: ws, list: list };
})();`
}
func jsString(s string) string {
b, _ := json.Marshal(s)
return string(b)
}

View File

@@ -27,6 +27,7 @@ type Server struct {
shellAgentRegistry *agent.Registry
shellAgentToolsJSON json.RawMessage
workflowEngine *workflow.Engine
browserTestStore *BrowserTestStore
activeCrushAgents atomic.Int32
activeClaudeAgents atomic.Int32
}
@@ -58,6 +59,11 @@ func NewServer(cfg *config.MuyueConfig) *Server {
s.shellConvStore = NewShellConvStore()
s.consumption = newConsumptionStore()
s.agentRegistry = agent.DefaultRegistry()
s.browserTestStore = NewBrowserTestStore()
if err := RegisterBrowserTestTool(s.agentRegistry, s.browserTestStore); err != nil {
// Tool registration only fails for duplicate names — non-fatal
_ = err
}
tools := s.agentRegistry.OpenAITools()
toolsJSON, _ := json.Marshal(tools)
s.agentToolsJSON = json.RawMessage(toolsJSON)
@@ -146,6 +152,11 @@ func (s *Server) routes() {
s.mux.HandleFunc("/api/recent-commands", s.handleRecentCommands)
s.mux.HandleFunc("/api/running-processes", s.handleRunningProcesses)
s.mux.HandleFunc("/api/system/metrics", s.handleSystemMetrics)
s.mux.HandleFunc("/api/test/snippet", s.handleBrowserTestSnippet)
s.mux.HandleFunc("/api/test/sessions", s.handleBrowserTestSessions)
s.mux.HandleFunc("/api/test/console/", s.handleBrowserTestConsole)
s.mux.HandleFunc("/api/ws/browser-test", s.handleBrowserTestWS)
}
func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {

View File

@@ -7,7 +7,7 @@ import (
const (
Name = "muyue"
Version = "0.6.0"
Version = "0.7.0"
Author = "La Légion de Muyue"
)

View File

@@ -44,6 +44,9 @@ const api = {
getRecentCommands: () => request('/recent-commands'),
getRunningProcesses: () => request('/running-processes'),
getSystemMetrics: () => request('/system/metrics'),
getTestSnippet: () => request('/test/snippet'),
getTestSessions: () => request('/test/sessions'),
getTestConsole: (sessionId) => request(`/test/console/${encodeURIComponent(sessionId || '')}`),
savePreferences: (prefs) => request('/preferences', { method: 'PUT', body: JSON.stringify(prefs) }),
saveProfile: (profile) => request('/config/profile', { method: 'PUT', body: JSON.stringify(profile) }),
saveProvider: (provider) => request('/config/provider', { method: 'PUT', body: JSON.stringify(provider) }),

View File

@@ -1,5 +1,5 @@
import { useState, useEffect, useCallback, useMemo, useRef } from 'react'
import { LayoutDashboard, Sparkles, Terminal, Settings } from 'lucide-react'
import { LayoutDashboard, Sparkles, Terminal, Settings, TestTube2 } from 'lucide-react'
import api from '../api/client'
import { getTheme, applyTheme } from '../themes'
import { useI18n } from '../i18n'
@@ -7,6 +7,7 @@ import Dashboard from './Dashboard'
import Studio from './Studio'
import Shell from './Shell'
import Config from './Config'
import Tests from './Tests'
import OnboardingWizard from './OnboardingWizard'
export default function App() {
@@ -24,6 +25,7 @@ export default function App() {
{ id: 'dash', label: t('tabs.dashboard'), icon: <LayoutDashboard size={15} /> },
{ id: 'studio', label: t('tabs.studio'), icon: <Sparkles size={15} /> },
{ id: 'shell', label: t('tabs.shell'), icon: <Terminal size={15} /> },
{ id: 'tests', label: 'Tests', icon: <TestTube2 size={15} /> },
{ id: 'config', label: t('tabs.config'), icon: <Settings size={15} /> },
], [t])
@@ -54,7 +56,8 @@ export default function App() {
Digit1: 'dash',
Digit2: 'studio',
Digit3: 'shell',
Digit4: 'config',
Digit4: 'tests',
Digit5: 'config',
}
if (map[e.code]) {
e.preventDefault()
@@ -92,6 +95,7 @@ export default function App() {
{ keys: layout.keys.enter, desc: t('statusbar.runCommand') },
{ keys: `${layout.keys.up}/${layout.keys.down}`, desc: t('statusbar.commandHistory') },
],
tests: [],
config: [],
}), [layout, t])
@@ -129,6 +133,7 @@ export default function App() {
<div className={activeTab === 'dash' ? '' : 'tab-hidden'}><Dashboard api={api} refreshRef={dashRefreshRef} /></div>
<div className={activeTab === 'studio' ? '' : 'tab-hidden'}><Studio api={api} /></div>
<div className={activeTab === 'shell' ? '' : 'tab-hidden'}><Shell api={api} isSudo={isSudo} /></div>
<div className={activeTab === 'tests' ? '' : 'tab-hidden'}><Tests api={api} /></div>
<div className={activeTab === 'config' ? '' : 'tab-hidden'}><Config api={api} /></div>
</main>

View File

@@ -0,0 +1,238 @@
import { useEffect, useRef, useState, useCallback } from 'react'
import { TestTube2, Copy, RefreshCw, CheckCircle2, AlertTriangle, Globe, Terminal as TerminalIcon } from 'lucide-react'
export default function Tests({ api }) {
const [snippet, setSnippet] = useState(null)
const [snippetError, setSnippetError] = useState('')
const [sessions, setSessions] = useState([])
const [console_, setConsole_] = useState([])
const [activeSessionId, setActiveSessionId] = useState('')
const [copied, setCopied] = useState(false)
const pollRef = useRef(null)
const refreshSnippet = useCallback(async () => {
try {
const data = await api.getTestSnippet()
setSnippet(data)
setSnippetError('')
} catch (err) {
setSnippetError(err.message || 'Failed to load snippet')
}
}, [api])
const refreshSessions = useCallback(async () => {
try {
const data = await api.getTestSessions()
const next = data.sessions || []
setSessions(next)
if (!activeSessionId && next.length > 0) {
setActiveSessionId(next[0].id)
} else if (activeSessionId && !next.find(s => s.id === activeSessionId)) {
setActiveSessionId(next.length > 0 ? next[0].id : '')
}
} catch {}
}, [api, activeSessionId])
const refreshConsole = useCallback(async () => {
if (!activeSessionId) {
setConsole_([])
return
}
try {
const data = await api.getTestConsole(activeSessionId)
setConsole_(data.console || [])
} catch {
setConsole_([])
}
}, [api, activeSessionId])
useEffect(() => {
refreshSnippet()
}, [refreshSnippet])
useEffect(() => {
refreshSessions()
refreshConsole()
pollRef.current = setInterval(() => {
refreshSessions()
refreshConsole()
}, 2000)
return () => clearInterval(pollRef.current)
}, [refreshSessions, refreshConsole])
const copySnippet = useCallback(async () => {
if (!snippet) return
try {
await navigator.clipboard.writeText(snippet.snippet)
setCopied(true)
setTimeout(() => setCopied(false), 1500)
} catch {}
}, [snippet])
const activeSession = sessions.find(s => s.id === activeSessionId) || null
return (
<div className="tests-layout" style={{ padding: '20px', display: 'grid', gridTemplateColumns: '1fr 1fr', gap: '20px', height: '100%', overflow: 'auto' }}>
<section className="tests-pane">
<header style={{ display: 'flex', alignItems: 'center', gap: 8, marginBottom: 12 }}>
<TestTube2 size={18} />
<h2 style={{ margin: 0, fontSize: '1.1em' }}>Tests pilotés par l'IA</h2>
</header>
<p style={{ marginTop: 0, opacity: 0.85, lineHeight: 1.5 }}>
Donnez à l'IA Studio le contrôle d'un onglet de votre navigateur pour tester chaque bouton et détecter les erreurs console.
</p>
<div style={{ borderTop: '1px solid var(--border, rgba(128,128,128,0.3))', paddingTop: 12, marginTop: 12 }}>
<h3 style={{ fontSize: '0.95em', margin: '0 0 8px' }}>1. Connexion</h3>
<ol style={{ paddingLeft: 18, lineHeight: 1.6 }}>
<li>Ouvrez la page à tester dans n'importe quel navigateur (Chrome, Firefox, Edge).</li>
<li>Ouvrez la console développeur (<kbd>F12</kbd>).</li>
<li>Collez ce snippet et appuyez sur <kbd>Entrée</kbd> :</li>
</ol>
{snippetError && (
<div style={{ background: 'rgba(220,80,80,0.1)', border: '1px solid rgba(220,80,80,0.3)', padding: 8, borderRadius: 4, marginBottom: 8 }}>
{snippetError}
</div>
)}
<div style={{ position: 'relative', marginBottom: 12 }}>
<pre style={{
background: 'var(--bg-secondary, rgba(0,0,0,0.3))',
padding: '10px 12px',
borderRadius: 4,
fontSize: '0.75em',
maxHeight: 180,
overflow: 'auto',
border: '1px solid var(--border, rgba(128,128,128,0.3))',
margin: 0,
}}>
{snippet?.snippet || 'Chargement…'}
</pre>
<button
onClick={copySnippet}
disabled={!snippet}
title="Copier"
style={{
position: 'absolute', top: 6, right: 6,
background: 'var(--bg-tertiary, rgba(255,255,255,0.08))',
border: '1px solid var(--border, rgba(128,128,128,0.3))',
color: 'inherit', padding: '4px 8px', borderRadius: 3,
cursor: 'pointer', fontSize: '0.75em',
display: 'flex', alignItems: 'center', gap: 4,
}}
>
<Copy size={11} /> {copied ? 'Copié !' : 'Copier'}
</button>
</div>
<button onClick={refreshSnippet} style={{ background: 'transparent', border: '1px solid var(--border, rgba(128,128,128,0.3))', color: 'inherit', padding: '4px 10px', borderRadius: 3, cursor: 'pointer', display: 'inline-flex', alignItems: 'center', gap: 4, fontSize: '0.85em' }}>
<RefreshCw size={12} /> Régénérer le token
</button>
<small style={{ display: 'block', opacity: 0.6, marginTop: 4 }}>
Le token expire après {snippet?.expires_in ? Math.round(snippet.expires_in / 60) : 5} minutes ou dès la première connexion.
</small>
</div>
<div style={{ borderTop: '1px solid var(--border, rgba(128,128,128,0.3))', paddingTop: 12, marginTop: 16 }}>
<h3 style={{ fontSize: '0.95em', margin: '0 0 8px' }}>2. Pilotage par l'IA</h3>
<p style={{ margin: '0 0 8px', lineHeight: 1.5 }}>
Une fois la session connectée, allez dans l'onglet <strong>Studio</strong> et demandez par exemple :
</p>
<pre style={{ background: 'var(--bg-secondary, rgba(0,0,0,0.3))', padding: 8, borderRadius: 4, fontSize: '0.85em', margin: 0 }}>
{`Teste tous les boutons de cette page,
clique sur chacun, et dis-moi
lesquels déclenchent une erreur console.`}
</pre>
<p style={{ margin: '8px 0 0', opacity: 0.75, fontSize: '0.85em' }}>
L'IA dispose de l'outil <code>browser_test</code> avec les actions <code>list_clickables</code>, <code>click</code>, <code>console</code>, <code>eval</code>, <code>type</code>, <code>current_url</code>, <code>wait</code>, <code>summary</code>.
</p>
</div>
</section>
<section className="tests-pane">
<header style={{ display: 'flex', alignItems: 'center', gap: 8, marginBottom: 12, justifyContent: 'space-between' }}>
<div style={{ display: 'flex', alignItems: 'center', gap: 8 }}>
<Globe size={16} />
<h2 style={{ margin: 0, fontSize: '1.1em' }}>Sessions connectées</h2>
</div>
<span style={{ display: 'inline-flex', alignItems: 'center', gap: 4, fontSize: '0.85em' }}>
{sessions.length > 0 ? <CheckCircle2 size={14} color="#3aaa61" /> : <span style={{ display: 'inline-block', width: 8, height: 8, borderRadius: '50%', background: '#888' }} />}
{sessions.length} session{sessions.length > 1 ? 's' : ''}
</span>
</header>
{sessions.length === 0 ? (
<div style={{ padding: 16, textAlign: 'center', opacity: 0.7, border: '1px dashed var(--border, rgba(128,128,128,0.3))', borderRadius: 4 }}>
<AlertTriangle size={20} style={{ opacity: 0.4 }} />
<div style={{ marginTop: 6 }}>Aucune session active.</div>
<small>Collez le snippet dans une page pour démarrer.</small>
</div>
) : (
<div style={{ display: 'flex', flexDirection: 'column', gap: 6, marginBottom: 12 }}>
{sessions.map(s => (
<button
key={s.id}
onClick={() => setActiveSessionId(s.id)}
style={{
textAlign: 'left',
background: s.id === activeSessionId ? 'var(--accent-bg, rgba(108,92,231,0.15))' : 'transparent',
border: '1px solid ' + (s.id === activeSessionId ? 'var(--accent, #6c5ce7)' : 'var(--border, rgba(128,128,128,0.3))'),
color: 'inherit',
padding: 8, borderRadius: 4, cursor: 'pointer',
}}
>
<div style={{ fontWeight: 500, fontSize: '0.9em', overflow: 'hidden', textOverflow: 'ellipsis', whiteSpace: 'nowrap' }}>
{s.title || s.url || s.id}
</div>
<div style={{ fontSize: '0.75em', opacity: 0.65, overflow: 'hidden', textOverflow: 'ellipsis', whiteSpace: 'nowrap' }}>
{s.url} · session {s.id.slice(0, 8)}
</div>
</button>
))}
</div>
)}
{activeSession && (
<div style={{ borderTop: '1px solid var(--border, rgba(128,128,128,0.3))', paddingTop: 12 }}>
<header style={{ display: 'flex', alignItems: 'center', gap: 6, marginBottom: 8 }}>
<TerminalIcon size={14} />
<h3 style={{ margin: 0, fontSize: '0.95em' }}>Console (live, dernières {console_.length})</h3>
</header>
<div style={{
background: 'var(--bg-secondary, rgba(0,0,0,0.3))',
padding: 8,
borderRadius: 4,
maxHeight: 380,
overflow: 'auto',
fontSize: '0.8em',
fontFamily: 'var(--font-mono, ui-monospace, monospace)',
border: '1px solid var(--border, rgba(128,128,128,0.3))',
}}>
{console_.length === 0 ? (
<div style={{ opacity: 0.5 }}>(aucun message console)</div>
) : (
console_.map((c, i) => (
<div key={i} style={{ color: levelColor(c.level), padding: '2px 0', borderBottom: '1px dashed rgba(128,128,128,0.15)' }}>
<span style={{ opacity: 0.55, fontSize: '0.85em' }}>[{c.time?.slice(11, 19)} {c.level}]</span> {c.message}
</div>
))
)}
</div>
</div>
)}
</section>
</div>
)
}
function levelColor(lvl) {
switch (lvl) {
case 'error': return '#ff6b6b'
case 'warn': return '#f5a623'
case 'info': return '#4dabf7'
case 'debug': return '#888'
default: return 'inherit'
}
}