Replace toxicity model with multi-label classifier

This commit is contained in:
Jordan Wages 2026-08-26 17:00:49 -05:00
commit 1568c787f8
24 changed files with 2581129 additions and 119717 deletions

View file

@ -6,12 +6,7 @@ Toxic Content Filter is a browser extension that automatically identifies and hi
The extension performs all inference locally. Social-media content is never sent to an external classification service.
The initial classifier produces a simple binary classification:
* **Toxic**
* **Non-toxic**
The model should also expose its confidence/probability, allowing the user to configure the threshold at which content is filtered.
The classifier produces independent toxicity-category probabilities. Users enable the categories they want filtered and configure one shared threshold; content is filtered when any enabled category reaches it.
## Goals
@ -27,16 +22,7 @@ The extension should:
## Classification Model
The initial implementation will use a small transformer model trained specifically for toxicity classification.
A candidate is Citizen Lab's multilingual DistilBERT toxicity classifier, which produces:
```text
toxic
not_toxic
```
along with classification probabilities.
The implementation uses `wagesj45/multilabel-toxic-comment-classifier`, a multilingual ModernBERT classifier distributed under Apache-2.0. It produces probability scores for toxicity, severe toxicity, obscene, threat, insult, identity attack, and sexual explicit content.
The model should be converted to ONNX and quantized as aggressively as practical while retaining acceptable classification accuracy.
@ -45,7 +31,7 @@ The deployed extension does not require Python or a native companion application
Conceptually:
```text
DistilBERT toxicity model
ModernBERT multi-label toxicity model
|
v
ONNX