vibeguard/DESIGN.md

Toxic Content Filter

Overview

Toxic Content Filter is a browser extension that automatically identifies and hides toxic posts and comments on supported social-media websites.

The extension performs all inference locally. Social-media content is never sent to an external classification service.

The initial classifier produces a simple binary classification:

• Toxic
• Non-toxic

The model should also expose its confidence/probability, allowing the user to configure the threshold at which content is filtered.

Goals

The extension should:

• Detect toxic social-media posts and comments as they appear.
• Hide or collapse content whose toxicity score exceeds a configurable threshold.
• Perform inference entirely on the user's computer.
• Support multiple simultaneously open social-media tabs without loading multiple copies of the model.
• Avoid interfering with normal page responsiveness.
• Support both Firefox and Chromium-based browsers.
• Allow new websites to be supported through site-specific post parsers.

Classification Model

The initial implementation will use a small transformer model trained specifically for toxicity classification.

A candidate is Citizen Lab's multilingual DistilBERT toxicity classifier, which produces:

toxic
not_toxic

along with classification probabilities.

The model should be converted to ONNX and quantized as aggressively as practical while retaining acceptable classification accuracy.

The deployed extension does not require Python or a native companion application.

Conceptually:

DistilBERT toxicity model
          |
          v
        ONNX
          |
          v
Transformers.js / ONNX Runtime Web
          |
          v
       WASM CPU

CPU/WASM is used for browser compatibility and to avoid unnecessary GPU utilization.

CPU WASM inference uses ONNX Runtime's browser-selected defaults.

Site Integration

The extension supports websites through declarative, user-configurable site definitions rather than site-specific code. A definition supplies URL patterns, page markers, post selectors, text selectors, and stable-ID sources. The content script is loaded broadly but exits without examining content unless a definition matches.

Each active definition is executed by the same generic discovery engine.

Conceptually:

Site definition
    |
    +-- Generic discovery engine

A definition is responsible for declaring:

• Where posts and comments appear.
• Which rendered text should be extracted.
• Which attributes or links provide stable identifiers.

A normalized post might resemble:

{
    id,
    text,
    element
}

Dynamic websites should be monitored with MutationObserver so newly loaded content can be discovered during infinite scrolling and navigation.

Filtering

The general processing pipeline is:

Social-media DOM
       |
       v
Site-specific parser
       |
       v
Shared inference queue
       |
       v
Toxicity classifier
       |
       v
Toxicity probability
       |
       v
User-configured threshold
       |
       +---- below ----> display normally
       |
       +---- above ----> hide/collapse

Rather than permanently removing filtered DOM elements, the extension should preferably collapse them.

An optional placeholder may allow the user to reveal incorrectly classified content:

Content hidden as toxic (91%)    [Show]

The exact presentation should be configurable and may include an option to hide filtered content completely.

Shared Inference Architecture

The ML model must not be instantiated independently inside each social-media tab.

All tabs share a single inference backend:

Facebook tab ----\
Twitter tab ------\
Reddit tab --------> inference queue --> classifier
Twitter tab ------/

This provides:

• One loaded model.
• One tokenizer.
• One ONNX runtime.
• One set of inference buffers.
• Centralized scheduling and batching.

Content scripts should remain lightweight and communicate with the shared backend through browser extension messaging APIs.

Firefox Architecture

For Firefox with Manifest V2:

Content scripts
      |
      v
Persistent background page
      |
      v
Inference Web Worker
      |
      v
ONNX Runtime
      |
      v
Toxicity model

The persistent background context owns the inference worker and therefore the single loaded model.

Chromium Manifest V3 Architecture

Manifest V3 still permits content scripts to inspect and modify page DOMs. Its restrictions on network-request interception that affected extensions such as uBlock Origin do not prevent this extension's core functionality.

Chromium MV3 service workers are ephemeral and therefore should not own the loaded ML model.

Instead:

Content scripts
      |
      v
MV3 service worker
      |
      v
Offscreen document
      |
      v
Inference Web Worker
      |
      v
ONNX Runtime
      |
      v
Toxicity model

The service worker acts primarily as a coordinator/router.

An offscreen extension document provides a longer-lived context and creates the inference Web Worker. Chromium's Offscreen API explicitly provides a WORKERS reason for offscreen documents that need to spawn workers.

The inference worker owns the model and queue.

Inference Queue

Every supported tab submits discovered content to one global inference queue.

Requests should be batched:

Twitter:    7 posts --\
Reddit:    12 posts ---+--> queue --> batch --> classifier
Facebook:   4 posts --/

Batch size should be determined experimentally based on latency, memory consumption, and throughput.

The queue should support prioritization. A possible initial policy is:

0 - Visible content in active tab
1 - Nearby/upcoming content in active tab
2 - Content in background tabs
3 - Preloaded/speculative content

Interactive responsiveness is more important than maximizing raw classifier throughput.

The queue should also be bounded. Stale requests should be discarded when:

• Their originating tab closes.
• Navigation invalidates the associated content.
• The associated DOM element disappears.
• Newer work makes speculative classification unnecessary.

Result Cache

The inference backend should maintain a shared cache:

hash(normalizedText) -> toxicity probability

This prevents unnecessary inference when:

• A framework destroys and recreates a DOM element.
• Identical posts appear multiple times.
• Content appears in multiple tabs.
• Previously classified content is encountered again.

An in-memory cache should be implemented initially. Persistent caching across browser sessions can be considered later.

Configuration

Initial user-facing configuration should remain deliberately simple.

At minimum:

Enable filtering:       Yes
Toxicity threshold:     0.80
Filtered content:       Collapse

Site-specific enable/disable controls should also be available.

More sophisticated filtering options should only be added if they prove useful.

Privacy

Privacy is an important architectural requirement.

Post text must remain local to the browser. The extension should not require an account, remote API, or cloud inference service.

The basic data path is therefore:

Website
   |
   v
Extension
   |
   v
Local model
   |
   X
 Internet

No social-media content needs to leave the user's computer for classification.

Model Distribution

The initial model may be several hundred megabytes before optimization.

The project should investigate:

• ONNX graph optimization.
• INT8/Q8 quantization.
• More aggressive quantization where supported.
• Smaller toxicity models if classification quality remains acceptable.

The model may either ship with the extension or be downloaded once after installation and cached locally.

Downloading the model separately may substantially reduce extension package size.

Future Work

Possible later improvements include:

• Additional supported websites.
• Improved batching and scheduling.
• Persistent classification cache.
• Per-site thresholds.
• User feedback on incorrect classifications.
• Locally collected examples for personalization.
• Fine-tuning or distillation of a smaller toxicity model specifically for browser filtering.
• Additional languages.

The initial implementation should avoid these features unless necessary to establish the basic architecture.

Core Design Principle

The project should maintain three distinct responsibilities:

Site parser
    |
    | discovers and normalizes content
    v
Shared inference backend
    |
    | produces toxicity probability
    v
Filtering policy
    |
    | applies user threshold
    v
Page presentation

Site definitions and their discovery engine should not contain ML logic. The inference backend should not understand individual websites. The classifier should only determine toxicity probability, while filtering policy determines whether that probability warrants hiding content.

This separation should allow website support, ML implementation, and user-facing filtering behavior to evolve independently.