Browser Control¶

MemoRizz 0.5 provides a provider-neutral browser_control capability. When it is enabled, a MemAgent receives one bounded natural-language browser tool. The first provider is Browser Use.

Browser automation can navigate, click, type, upload, submit, publish, buy, or delete. MemoRizz therefore marks every model-initiated browser call as nondeterministic, side-effecting, and durable-approval-required. There is no model-visible approved or confirm argument.

Install Browser Use separately¶

Browser Use currently targets Python 3.11+ and its package dependency line can conflict with MemoRizz's MCP 2.x runtime. Install it as an isolated tool instead of into the MemoRizz environment:

uv tool install --python 3.12 browser-use
browser-use install   # downloads Chromium

From Browser Use 0.13 the browser-use command itself drives your own running Chrome through a background daemon, so browser-use doctor checks that daemon and reports failures that do not affect MemoRizz. MemoRizz needs only the Python package and Chromium. To check the whole path, run a real task: /browser run Read the title at https://example.com in the CLI.

MemoRizz resolves the isolated Python interpreter from the browser-use entry point installed by uv tool/pipx, then uses the official Python Agent API. It writes a fixed wrapper to a private temporary directory, sends the model's task only as JSON data, and never executes model-authored Python or shell commands. The worker process and browser are bounded and closed on success, failure, and timeout. If a custom launcher has no absolute Python shebang, configure MEMORIZZ_BROWSER_USE_PYTHON_COMMAND explicitly.

Set the API key for the Browser Use LLM provider you select:

export OPENAI_API_KEY=<secret>
# Or ANTHROPIC_API_KEY, GOOGLE_API_KEY, or BROWSER_USE_API_KEY.

SDK setup¶

from memorizz.memagent.builders import MemAgentBuilder

agent = (
    MemAgentBuilder()
    .with_memory_provider(memory_provider)
    .with_llm_config({"provider": "openai", "model": "gpt-4.1-mini"})
    .with_browser_control(
        {
            "provider": "browseruse",
            "llm_provider": "openai",
            "model": "gpt-5.4-mini",
            "headless": True,
            "use_vision": "auto",
            "allowed_domains": ["example.com", "*.notion.so"],
            "prohibited_domains": ["admin.example.com"],
            "block_ip_addresses": True,
            "max_steps": 25,
            "task_timeout": 600,
        }
    )
    .build(validate=True, persist=True)
)

Equivalent direct construction and runtime changes are available:

from memorizz import MemAgent

agent = MemAgent(
    model=model,
    memory_provider=memory_provider,
    browser_control={"provider": "browseruse", "allowed_domains": ["example.com"]},
)

agent.with_browser_control({"provider": "browseruse"})
agent.with_browser_control(None)  # detach and remove the tool

agent.capability_report() reports configured/readiness state without exposing credentials. Persisted browser configuration contains policy and environment variable names only, never API-key values.

Approval lifecycle¶

If the model calls:

{
  "name": "browser_control",
  "arguments": {
    "task": "Open example.com and submit the support form",
    "max_steps": 8
  }
}

MemoRizz stores an ApprovalProposal containing the exact tool name, arguments, canonical argument hash, policy reason, agent/thread/checkpoint identity, creation/expiry times, and later the approver identity and audit timestamp. The provider is not called yet.

A trusted host decides and resumes the exact checkpoint:

pending = agent.list_approval_proposals(status="pending")
proposal_id = pending[0]["proposal_id"]

agent.approve(
    proposal_id,
    approver_id="operator@example.com",
    reason="Destination and form values reviewed",
)
resume = agent.resume_approval(proposal_id)
print(resume.tool_result)         # exact BrowserControlResult payload
print(resume.assistant_response)  # optional model continuation

# Deterministic hosts can omit the model continuation:
resume = agent.resume_approval(proposal_id, continue_model=False)

Alternatives are agent.reject(...) and agent.cancel_approval(...). Proposals expire, bind to the stored argument hash, and are consumed atomically once. Resume does not ask the model to reconstruct the task.

The local playground renders the proposal, arguments/hash, expiry, and Approve/Reject controls. The interactive CLI exposes the same lifecycle:

/approvals pending
/approvals approve PROPOSAL_ID operator@example.com reviewed
/approvals resume PROPOSAL_ID

CLI and environment¶

Browser control is explicit opt-in. Merely setting an LLM API key does not add the tool.

memorizz chat --browser-control
memorizz run --browser-control "Read the title at example.com"

Inside the REPL:

/browser status
/browser on
/browser off
/browser run Read the title at https://example.com

/browser run is a direct trusted-host action: the user at the terminal is the authorizing host. Model-initiated calls still create a durable proposal.

Environment configuration:

Variable	Default	Purpose
`MEMORIZZ_BROWSER_CONTROL_PROVIDER`	disabled	Set `browseruse` to opt in globally
`MEMORIZZ_BROWSER_USE_COMMAND`	`browser-use`	Entry point for the isolated Browser Use installation
`MEMORIZZ_BROWSER_USE_PYTHON_COMMAND`	discovered from entry point	Optional isolated Python executable override
`MEMORIZZ_BROWSER_USE_LLM_PROVIDER`	`openai`	`openai`, `anthropic`, `google`, or `browseruse`
`MEMORIZZ_BROWSER_USE_MODEL`	see below	Browser-agent model
`MEMORIZZ_BROWSER_USE_HEADLESS`	`true`	Hide/show the local browser window
`MEMORIZZ_BROWSER_USE_VISION`	`auto`	`auto` sends a screenshot only when the browser agent asks for one; `true` sends one every step; `false` never
`MEMORIZZ_BROWSER_USE_ALLOWED_DOMAINS`	empty	Comma-separated navigation allowlist
`MEMORIZZ_BROWSER_USE_PROHIBITED_DOMAINS`	empty	Comma-separated denylist
`MEMORIZZ_BROWSER_USE_BLOCK_IP_ADDRESSES`	`true`	Block direct IP navigation
`MEMORIZZ_BROWSER_USE_MAX_STEPS`	`25`	Host maximum, clamped to 1–100
`MEMORIZZ_BROWSER_USE_TASK_TIMEOUT`	`600`	Wall timeout, clamped to 10–3600 seconds
`MEMORIZZ_BROWSER_USE_CLOUD`	`false`	Use Browser Use cloud browser
`MEMORIZZ_BROWSER_USE_CDP_URL_ENV`	`BU_CDP_URL`	Name of the allowlisted CDP URL variable

The UI Settings page exposes the common fields, while each agent editor and playground panel controls whether the capability is attached.

Speed and cost¶

Each task runs in a fresh worker with a fresh headless Chromium, so a call spends about two seconds importing Browser Use and launching the browser before the browser agent's first step. MemoRizz keeps that boundary, so no task inherits cookies, tabs or sign-ins from an earlier approved task, and trims the rest:

The browser agent defaults to a fast, cheap model: gpt-5.4-mini for OpenAI, claude-haiku-5-5 for Anthropic, gemini-flash-latest for Google, and Browser Use's own default for browseruse. Browser Use's OpenAI, Anthropic and Google wrappers have no default model of their own. Set MEMORIZZ_BROWSER_USE_MODEL=gpt-5.6-luna for the lowest cost; in the runs below it cost about a fifth as much and took about a fifth longer.
Vision defaults to auto. Every step carries the page's text and element tree; a screenshot goes along only when the browser agent asks for one.
Browser Use's judge is off. It replays each finished run, with up to ten screenshots, in one more LLM call whose verdict is only logged.
Browser Use's telemetry is off unless you set ANONYMIZED_TELEMETRY yourself. It uploads each task, the URLs visited, the actions taken and the final answer.

Measured on a Mac with Browser Use 0.13.7, over three tasks (read example.com, list the top three Hacker News stories, search Wikipedia and read the first sentence of an article), two runs each:

Setup	Mean time per task	Mean prompt tokens	Mean cost
Earlier defaults with `gpt-4.1-mini`: screenshot every step, judge on	31.4 s	98,100	$0.030
Current defaults: `gpt-5.4-mini`, vision `auto`, no judge	8.6 s	29,400	$0.013
`gpt-5.6-luna`, vision `auto`, no judge	10.1 s	23,200	$0.002

Result contract¶

The tool returns structured JSON:

{
  "success": true,
  "task": "Read the title at https://example.com",
  "output": "Example Domain",
  "urls": ["https://example.com/"],
  "actions": ["navigate", "done"],
  "errors": [],
  "steps": 2,
  "duration_seconds": 4.9,
  "metadata": {
    "provider": "browseruse",
    "execution_boundary": "isolated_browser_use_python",
    "model": "gpt-5.4-mini",
    "max_steps": 8,
    "headless": true,
    "use_vision": "auto",
    "block_ip_addresses": true,
    "timings": {
      "import_seconds": 0.6,
      "browser_start_seconds": 1.0,
      "startup_seconds": 1.8,
      "run_seconds": 2.8,
      "steps_seconds": 1.6,
      "browser_stop_seconds": 0.0
    },
    "usage": {
      "prompt_tokens": 14100,
      "cached_tokens": 0,
      "completion_tokens": 188,
      "llm_calls": 1,
      "cost_usd": 0.011421
    }
  }
}

duration_seconds is the whole call. timings shows where it went: startup_seconds covers importing Browser Use, launching Chromium and building the agent, and steps_seconds is the browser agent's own steps. usage is the browser agent's token use, with cost_usd at list rates for OpenAI and Anthropic models MemoRizz has a rate card for. output is capped at 12,000 characters (max_output_chars) so one page dump cannot fill the chat model's context; a cut output sets output_truncated. Only the last five step errors are kept, each shortened, with errors_omitted counting the rest.

Timeouts and provider failures remain structured failures rather than partial untyped text. The isolated worker process group is terminated on timeout, Browser Use receives an environment allowlist, and the wrapper always stops its browser in a finally block.

Security guidance¶

Prefer a narrow allowed_domains list. An empty list is unrestricted; wildcard * is rejected.
Treat page text as untrusted prompt-injection content. Approval is an action gate, not a guarantee that page instructions are safe.
Use separate browser profiles/accounts with least privilege; do not expose a personal administrator profile to a multi-tenant agent.
Keep direct IP blocking enabled and deny sensitive domains explicitly.
Leave ANONYMIZED_TELEMETRY unset (MemoRizz then sends false) unless you accept Browser Use uploading task text, visited URLs and results.
Review the complete task and destination before approval, especially for forms, messages, purchases, uploads, account changes, and destructive work.
Browser Use is isolated from MemoRizz at the Python dependency boundary and runs a private fixed worker per task. This is lifecycle isolation, not an OS security sandbox; the browser retains the network and account authority you configure.

Custom providers¶

Equivalent providers implement BrowserControlProvider.run_task(...), return a BrowserControlResult, expose secret-free get_config(), and register an explicit name. The MemoRizz manager preserves the same single governed tool and approval policy regardless of provider.

from memorizz.browser_control import (
    BrowserControlProvider,
    BrowserControlResult,
    register_provider,
)

class InternalBrowserProvider(BrowserControlProvider):
    provider_name = "internal"

    def run_task(self, task, *, max_steps=None, timeout=None):
        return BrowserControlResult(success=True, task=task, output="done")

register_provider("internal", InternalBrowserProvider)