Supporting v140+

After testing in Betterbird v140, updating manifest.

AGENTS.md

@@ -0,0 +1,76 @@
+# AGENTS Guidelines for Sortana
+
+This file provides guidelines for codex agents contributing to the Sortana project. It describes the repository structure, the expected coding conventions, and testing commands. If you make changes to Sortana, make sure to update this document and the README.md if applicable.
+
+## Repository Overview
+
+- `background.js`: Handles startup tasks and coordinates message passing within the extension.
+- `modules/`: Contains reusable JavaScript modules such as `AiClassifier.js`,
+  `defaultParams.js` and `themeUtils.js`.
+- `options/`: The options page HTML, JavaScript and bundled Bulma CSS (v1.0.3).
+- `details.html` and `details.js`: View AI reasoning and clear cache for a message.
+- `resources/`: Images and other static files.
+- `prompt_templates/`: Prompt template files for the AI service.
+- `build-xpi.ps1`: PowerShell script to package the extension.
+
+## Coding Style
+
+- Use **modern JavaScript** (ES6 or later). Prefer `const` and `let` over `var`.
+- Keep functions small and focused. Aim for clear naming and concise comments when necessary.
+- Use template literals for string interpolation.
+- Avoid trailing whitespace and ensure files end with a newline.
+
+## Commit Guidelines
+
+- Group related changes together. Each commit should have a clear purpose.
+- Use descriptive commit messages in the imperative mood (e.g., "Add filter editor patch").
+- Run `git status` before committing to verify only intended files are staged.
+
+## Testing
+
+There are currently no automated tests for this project. If you add tests in the future, specify the commands to run them here. For now, verification must happen manually in Thunderbird. Do **not** run the `ps1` build script or the SVG processing script.
+
+## Documentation
+
+Additional documentation exists outside this repository.
+
+- Development guide: [Webextention-API for Thunderbird](https://webextension-api.thunderbird.net/en/stable/)
+  - [Messages API](https://webextension-api.thunderbird.net/en/stable/messages.html)
+  - [Message Tags API](https://webextension-api.thunderbird.net/en/stable/messages.tags.html)
+  - [messageDisplayAction API](https://webextension-api.thunderbird.net/en/stable/messageDisplayAction.html)
+  - [Storage API](https://webextension-api.thunderbird.net/en/stable/storage.html)
+- Thunderbird Add-on Store Policies
+  - [Third Party Library Usage](https://extensionworkshop.com/documentation/publish/third-party-library-usage/)
+- Third Party Libraries
+  - [Bulma.css v1.0.3](https://github.com/jgthms/bulma/blob/1.0.3/css/bulma.css)
+- Issue tracker: [Thunderbird tracker on Bugzilla](https://bugzilla.mozilla.org/describecomponents.cgi?product=Thunderbird)
+
+
+### Message Structure Notes
+
+Messages retrieved with `messenger.messages.getFull` are returned as
+nested objects. The root contains `headers` and a `parts` array. Each part may
+itself contain `parts` for multipart messages or a `body` string. Attachments are
+indicated via the `content-disposition` header.
+
+When constructing the text sent to the AI service, parse the full message
+recursively. Include key headers such as `from`, `to`, `subject`, and others, and
+record attachment summaries rather than raw binary data. Inline or attached
+base64 data should be replaced with placeholders showing the byte size. The
+final string should have the headers, a brief attachment section, then the plain
+text extracted from all text parts.
+
+### Cache Strategy
+
+`aiCache` persists classification results. Each key is the SHA‑256 hex of
+`"<message Message-ID>|<criterion>"` and maps to an object with `matched` and `reason`
+properties. Any legacy `aiReasonCache` data is merged into `aiCache` the first
+time the add-on loads after an update.
+
+### Icon Set Usage
+
+Toolbar and menu icons reside under `resources/img` and are provided in 16, 32
+and 64 pixel variants. When changing these icons, pass a dictionary mapping the
+sizes to the paths in `browserAction.setIcon` or `messageDisplayAction.setIcon`.
+Use `resources/svg2img.ps1` to regenerate PNGs from the SVG sources.
+

README.md

@@ -9,28 +9,43 @@ message meets a specified criterion.
 
 ## Features
 
-- **AI classification rule** – adds the "AI classification" term with
-  `matches` and `doesn't match` operators.
 - **Configurable endpoint** – set the classification service URL on the options page.
 - **Prompt templates** – choose between several model formats or provide your own custom template.
-- **Filter editor integration** – patches Thunderbird's filter editor to accept
-  text criteria for AI classification.
-- **Result caching** – avoids duplicate requests for already-evaluated messages.
+- **Custom system prompts** – tailor the instructions sent to the model for more precise results.
+- **Persistent result caching** – classification results and reasoning are saved to disk so messages aren't re-evaluated across restarts.
+- **Advanced parameters** – tune generation settings like temperature, top‑p and more from the options page.
+- **Markdown conversion** – optionally convert HTML bodies to Markdown before sending them to the AI service.
+- **Debug logging** – optional colorized logs help troubleshoot interactions with the AI service.
+- **Debug tab** – view the last request payload and a diff between the unaltered message text and the final prompt.
+- **Light/Dark themes** – automatically match Thunderbird's appearance with optional manual override.
+- **Automatic rules** – create rules that tag, move, copy, forward, reply, delete, archive, mark read/unread or flag/unflag messages based on AI classification. Rules can optionally apply only to unread messages and can ignore messages outside a chosen age range.
+- **Rule ordering** – drag rules to prioritize them and optionally stop processing after a match.
+- **Rule enable/disable** – temporarily turn a rule off without removing it.
+- **Account & folder filters** – limit rules to specific accounts or folders.
+- **Context menu** – apply AI rules from the message list or the message display action button.
+- **Status icons** – toolbar icons show when classification is in progress and briefly display success states. If a failure occurs the icon turns red until you dismiss the notification.
+- **Error notification** – failed classification displays a notification with a button to clear the error and reset the icon.
+- **View reasoning** – inspect why rules matched via the Details popup.
+- **Cache management** – clear cached results from the context menu or options page.
+- **Queue & timing stats** – monitor processing time on the Maintenance tab.
 - **Packaging script** – `build-xpi.ps1` builds an XPI ready for installation.
+- **Maintenance tab** – view rule counts, cache entries and clear cached results from the options page.
+
+### Cache Storage
+
+Classification results are stored under the `aiCache` key in extension storage.
+Each entry maps a SHA‑256 hash of `"<message Message-ID>|<criterion>"` to an object
+containing `matched` and `reason` fields. Older installations with a separate
+`aiReasonCache` will be migrated automatically on startup.
 
 ## Architecture Overview
 
-The extension relies on both WebExtension scripts and Thunderbird's experiment
-APIs:
+Sortana is implemented entirely with standard WebExtension scripts—no custom experiment code is required:
 
-- `background.js` loads the saved endpoint, registers the UI patching script,
-  and listens for test messages.
-- `experiment/api.js` exposes the `aiFilter` namespace. It loads
-  `modules/ExpressionSearchFilter.jsm` which implements the custom filter term
-  and performs the HTTP request.
-- `experiment/DomContentScript/` registers content scripts for Thunderbird
-  windows; `content/filterEditor.js` modifies the filter editor UI.
-- `options/` contains the HTML and JavaScript for the options page.
+- `background.js` loads saved settings, manages the classification queue and listens for new messages.
+- `modules/AiClassifier.js` implements the classification logic and cache handling.
+- `options/` contains the HTML and JavaScript for configuring the endpoint and rules.
+- `details.html` / `details.js` present cached reasoning for a message.
 - `_locales/` holds localized strings used throughout the UI.
 
 ### Key Files
@@ -38,34 +53,101 @@ APIs:
 | Path                                    | Purpose                                        |
 | --------------------------------------- | ---------------------------------------------- |
 | `manifest.json`                         | Extension manifest and entry points. |
-| `background.js`                         | Startup tasks and message handling.            |
-| `modules/ExpressionSearchFilter.jsm`    | Custom filter term and AI request logic.       |
-| `experiment/api.js`                     | Bridges WebExtension code with privileged APIs.|
-| `content/filterEditor.js`               | Patches the filter editor interface.           |
-| `options/options.html` and `options.js` | Endpoint configuration UI.                     |
+| `background.js`                         | Startup tasks and classification queue management. |
+| `modules/AiClassifier.js`               | Core classification logic and cache handling. |
+| `options/options.html` and `options.js` | Endpoint and rule configuration UI. |
+| `details.html` and `details.js`         | View stored reasoning for a message. |
+| `logger.js`                             | Colorized logging with optional debug mode. |
 
 ## Building
 
 1. Ensure PowerShell is available (for Windows) or adapt the script for other
    environments.
-2. Run `powershell ./build-xpi.ps1` from the repository root. The script reads
-the version from `manifest.json` and creates an XPI in the `release` folder.
-3. Install the generated XPI in Thunderbird via the Add-ons Manager. During
+2. The Bulma stylesheet (v1.0.3) is already included as `options/bulma.css`.
+3. Run `powershell ./build-xpi.ps1` from the repository root. The script reads
+   the version from `manifest.json` and creates an XPI in the `release` folder.
+4. Install the generated XPI in Thunderbird via the Add-ons Manager. During
    development you can also load the directory as a temporary add-on.
+5. To regenerate PNG icons from the SVG sources, run `resources/svg2img.ps1`.
 
 ## Usage
 
 1. Open the add-on's options and set the URL of your classification service.
-2. Create or edit a filter in Thunderbird and choose the **AI classification**
-   term. Enter the desired criterion (for example, a short description of the
-   messages you want to match).
-3. When the filter runs, the add-on sends the message text to the service and
-   checks the JSON response for a match.
+ 2. Use the **Classification Rules** section to add a criterion and optional
+   actions such as tagging, moving, copying, forwarding, replying,
+   deleting or archiving a message when it matches. Drag rules to
+   reorder them, check *Only apply to unread messages* to skip read mail,
+   set optional minimum or maximum message age limits, select the accounts or
+ folders a rule should apply to. Use the
+ slashed-circle/check button to disable or re-enable a rule. The small
+ circle buttons for optional conditions show a filled dot when active, and
+ check *Stop after match* to halt further processing. Forward and reply actions
+   open a compose window using the account that received the message.
+3. Save your settings. New mail will be evaluated automatically using the
+   configured rules.
+4. If the toolbar icon shows a red X, click the notification's **Dismiss** button to clear the error.
+
+### Example Filters
+
+Here are some useful and fun example criteria you can use in your filters. Filters should be able to be answered as either `true` or `false`.
+
+- **"Does this message require my attention, response, or decision soon?"**  
+  Identify emails with deadlines, requests, or actionable items.
+
+- **"Is this message spam, phishing, or irrelevant bulk mail?"**  
+  Catch low-value or deceptive messages and sweep them into the junk folder.
+
+- **"Is this email promotional, advertising a product or service, or part of a mass mailing?"**  
+  Great for filtering out newsletters, deals, and marketing campaigns.
+
+- **"Is this a personal message from a friend or family member?"**  
+  Keep emotionally meaningful or social emails from getting lost.
+
+- **"Is this message a receipt, invoice, or shipping notification?"**  
+  Automatically tag or file transactional messages for easy lookup.
+
+- **"Does this message relate to one of my current work projects?"**  
+  Focus your inbox around what matters right now.
+
+- **"Would I roll my eyes reading this email?"**  
+  For when you're ready to filter based on vibes.
+
+You can define as many filters as you'd like, each using a different prompt and
+triggering tags, moves, copies, deletes, archives, read/unread changes or flag updates based on the model's classification.
+
+## Required Permissions
+
+Sortana requests the following Thunderbird permissions:
+
+- `storage` – store configuration and cached classification results.
+- `messagesRead` – read message contents for classification.
+- `messagesMove` – move or copy messages when a rule specifies a target folder.
+ - `messagesUpdate` – change message properties such as tags, junk status, read/unread state and flags.
+- `messagesTagsList` – retrieve existing message tags for rule actions.
+- `accountsRead` – list accounts and folders for move or copy actions.
+- `menus` – add context menu commands.
+- `tabs` – open new tabs and query the active tab.
+- `notifications` – display error notifications.
+- `compose` – create reply and forward compose windows for matching rules.
+
+## Thunderbird Add-on Store Disclosures
+
+The [Third Party Library Usage](https://extensionworkshop.com/documentation/publish/third-party-library-usage/) policy
+requires disclosure of third party libraries that are included in the add-on. Even though
+the disclosure is only required for add-on review, they'll be listed here as well. Sortana
+uses the following third party libraries:
+
+- [Bulma.css v1.0.3](https://github.com/jgthms/bulma/blob/1.0.3/css/bulma.css)
+  - MIT License
+- [turndown v7.2.0](https://github.com/mixmark-io/turndown/tree/v7.2.0)
+  - MIT License
+- [diff](https://github.com/google/diff-match-patch/blob/62f2e689f498f9c92dbc588c58750addec9b1654/javascript/diff_match_patch_uncompressed.js)
+  -  Apache-2.0 license
 
 ## License
 
 This project is licensed under the terms of the GNU General Public License
-version 3. See `LICENSE` for the full text.
+version 3. See `LICENSE` for the full text. Third party libraries are licensed seperately.
 
 ## Acknowledgments
 
@@ -75,3 +157,5 @@ Sortana builds upon knowledge gained from open-source projects. In particular,
 how Thunderbird's WebExtension and experiment APIs can be extended. Their code
 provided invaluable guidance during development.
 
+- Icons from [cc0-icons.jonh.eu](https://cc0-icons.jonh.eu/) are used under the CC0 license.
+

_locales/en-US/messages.json

@@ -14,5 +14,24 @@
   "template.mistral": { "message": "Mistral" },
   "template.custom": { "message": "Custom" },
   "options.save": { "message": "Save" },
-  "options.debugLogging": { "message": "Enable debug logging" }
+  "options.debugLogging": { "message": "Enable debug logging" },
+  "options.htmlToMarkdown": { "message": "Convert HTML body to Markdown" },
+  "options.stripUrlParams": { "message": "Remove URL tracking parameters" },
+  "options.altTextImages": { "message": "Replace images with alt text" },
+  "options.collapseWhitespace": { "message": "Collapse long whitespace" },
+  "options.tokenReduction": { "message": "Aggressive token reduction" }
+  ,"action.read": { "message": "read" }
+  ,"action.flag": { "message": "flag" }
+    ,"action.copy": { "message": "copy" }
+    ,"action.delete": { "message": "delete" }
+  ,"action.archive": { "message": "archive" }
+  ,"action.forward": { "message": "forward" }
+  ,"action.reply": { "message": "reply" }
+  ,"param.markRead": { "message": "mark read" }
+  ,"param.markUnread": { "message": "mark unread" }
+  ,"param.flag": { "message": "flag" }
+  ,"param.unflag": { "message": "unflag" }
+  ,"param.address": { "message": "address" }
+  ,"param.replyAll": { "message": "reply all" }
+  ,"param.replySender": { "message": "reply sender" }
 }

ai-filter.sln

@@ -5,29 +5,31 @@ VisualStudioVersion = 17.12.35707.178
 MinimumVisualStudioVersion = 10.0.40219.1
 Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution Items", "{BCC6E6D2-343B-4C48-854D-5FE3BBC3CB70}"
 	ProjectSection(SolutionItems) = preProject
+		.gitignore = .gitignore
+		AGENTS.md = AGENTS.md
 		background.js = background.js
 		build-xpi.ps1 = build-xpi.ps1
+		details.html = details.html
+		details.js = details.js
+		LICENSE = LICENSE
+		logger.js = logger.js
 		manifest.json = manifest.json
+		README.md = README.md
 	EndProjectSection
 	ProjectSection(FolderGlobals) = preProject
 		Q_5_4Users_4Jordan_4Documents_4Gitea_4thunderbird-ai-filter_4src_4manifest_1json__JsonSchema = 
 	EndProjectSection
 EndProject
-Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "experiment", "experiment", "{F2C8C786-FA23-4B63-934C-8CAA39D9BF95}"
-	ProjectSection(SolutionItems) = preProject
-		experiment\api.js = experiment\api.js
-		experiment\schema.json = experiment\schema.json
-	EndProjectSection
-EndProject
 Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "options", "options", "{7372FCA6-B0BC-4968-A748-7ABB17A7929A}"
 	ProjectSection(SolutionItems) = preProject
+		options\bulma.css = options\bulma.css
 		options\options.html = options\options.html
 		options\options.js = options\options.js
 	EndProjectSection
 EndProject
 Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "modules", "modules", "{75ED3C1E-D3C7-4546-9F2E-AC85859DDF4B}"
 	ProjectSection(SolutionItems) = preProject
-		modules\ExpressionSearchFilter.jsm = modules\ExpressionSearchFilter.jsm
+		modules\AiClassifier.js = modules\AiClassifier.js
 	EndProjectSection
 EndProject
 Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "_locales", "_locales", "{D446E5C6-BDDE-4091-BD1A-EC57170003CF}"
@@ -37,17 +39,6 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "en-US", "en-US", "{8BEA7793
 		_locales\en-US\messages.json = _locales\en-US\messages.json
 	EndProjectSection
 EndProject
-Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "DomContentScript", "DomContentScript", "{B334FFB0-4BD2-496E-BDC4-786620E019DA}"
-	ProjectSection(SolutionItems) = preProject
-		experiment\DomContentScript\implementation.js = experiment\DomContentScript\implementation.js
-		experiment\DomContentScript\schema.json = experiment\DomContentScript\schema.json
-	EndProjectSection
-EndProject
-Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "content", "content", "{028FDA4B-AC3E-4A0E-9291-978E213F9B78}"
-	ProjectSection(SolutionItems) = preProject
-		content\filterEditor.js = content\filterEditor.js
-	EndProjectSection
-EndProject
 Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "prompt_templates", "prompt_templates", "{86516D53-50D4-4FE2-9D8A-977A8F5EBDBD}"
 	ProjectSection(SolutionItems) = preProject
 		prompt_templates\mistral.txt = prompt_templates\mistral.txt
@@ -56,11 +47,87 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "prompt_templates", "prompt_
 	EndProjectSection
 EndProject
 Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "resources", "resources", "{68A87938-5C2B-49F5-8AAA-8A34FBBFD854}"
+	ProjectSection(SolutionItems) = preProject
+		resources\svg2img.ps1 = resources\svg2img.ps1
+	EndProjectSection
 EndProject
 Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "img", "img", "{F266602F-1755-4A95-A11B-6C90C701C5BF}"
 	ProjectSection(SolutionItems) = preProject
+		resources\img\average-16.png = resources\img\average-16.png
+		resources\img\average-32.png = resources\img\average-32.png
+		resources\img\average-64.png = resources\img\average-64.png
+		resources\img\check-16.png = resources\img\check-16.png
+		resources\img\check-32.png = resources\img\check-32.png
+		resources\img\check-64.png = resources\img\check-64.png
+		resources\img\circle-16.png = resources\img\circle-16.png
+		resources\img\circle-32.png = resources\img\circle-32.png
+		resources\img\circle-64.png = resources\img\circle-64.png
+		resources\img\circledots-16.png = resources\img\circledots-16.png
+		resources\img\circledots-32.png = resources\img\circledots-32.png
+		resources\img\circledots-64.png = resources\img\circledots-64.png
+		resources\img\clipboarddata-16.png = resources\img\clipboarddata-16.png
+		resources\img\clipboarddata-32.png = resources\img\clipboarddata-32.png
+		resources\img\clipboarddata-64.png = resources\img\clipboarddata-64.png
+		resources\img\download-16.png = resources\img\download-16.png
+		resources\img\download-32.png = resources\img\download-32.png
+		resources\img\download-64.png = resources\img\download-64.png
+		resources\img\eye-16.png = resources\img\eye-16.png
+		resources\img\eye-32.png = resources\img\eye-32.png
+		resources\img\eye-64.png = resources\img\eye-64.png
+		resources\img\flag-16.png = resources\img\flag-16.png
+		resources\img\flag-32.png = resources\img\flag-32.png
+		resources\img\flag-64.png = resources\img\flag-64.png
+		resources\img\full-logo-white.png = resources\img\full-logo-white.png
 		resources\img\full-logo.png = resources\img\full-logo.png
+		resources\img\gear-16.png = resources\img\gear-16.png
+		resources\img\gear-32.png = resources\img\gear-32.png
+		resources\img\gear-64.png = resources\img\gear-64.png
 		resources\img\logo.png = resources\img\logo.png
+		resources\img\logo128.png = resources\img\logo128.png
+		resources\img\logo16.png = resources\img\logo16.png
+		resources\img\logo32.png = resources\img\logo32.png
+		resources\img\logo48.png = resources\img\logo48.png
+		resources\img\logo64.png = resources\img\logo64.png
+		resources\img\logo96.png = resources\img\logo96.png
+		resources\img\reply-16.png = resources\img\reply-16.png
+		resources\img\reply-32.png = resources\img\reply-32.png
+		resources\img\reply-64.png = resources\img\reply-64.png
+		resources\img\settings-16.png = resources\img\settings-16.png
+		resources\img\settings-32.png = resources\img\settings-32.png
+		resources\img\settings-64.png = resources\img\settings-64.png
+		resources\img\trash-16.png = resources\img\trash-16.png
+		resources\img\trash-32.png = resources\img\trash-32.png
+		resources\img\trash-64.png = resources\img\trash-64.png
+		resources\img\upload-16.png = resources\img\upload-16.png
+		resources\img\upload-32.png = resources\img\upload-32.png
+		resources\img\upload-64.png = resources\img\upload-64.png
+		resources\img\x-16.png = resources\img\x-16.png
+		resources\img\x-32.png = resources\img\x-32.png
+		resources\img\x-64.png = resources\img\x-64.png
+	EndProjectSection
+EndProject
+Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "js", "js", "{21D2A42C-3F85-465C-9141-C106AFD92B68}"
+	ProjectSection(SolutionItems) = preProject
+		resources\js\diff_match_patch_uncompressed.js = resources\js\diff_match_patch_uncompressed.js
+		resources\js\turndown.js = resources\js\turndown.js
+	EndProjectSection
+EndProject
+Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "svg", "svg", "{D4E9C905-4884-488E-B763-5BD39049C1B1}"
+	ProjectSection(SolutionItems) = preProject
+		resources\svg\average.svg = resources\svg\average.svg
+		resources\svg\check.svg = resources\svg\check.svg
+		resources\svg\circle.svg = resources\svg\circle.svg
+		resources\svg\circledots.svg = resources\svg\circledots.svg
+		resources\svg\clipboarddata.svg = resources\svg\clipboarddata.svg
+		resources\svg\download.svg = resources\svg\download.svg
+		resources\svg\eye.svg = resources\svg\eye.svg
+		resources\svg\flag.svg = resources\svg\flag.svg
+		resources\svg\gear.svg = resources\svg\gear.svg
+		resources\svg\reply.svg = resources\svg\reply.svg
+		resources\svg\settings.svg = resources\svg\settings.svg
+		resources\svg\trash.svg = resources\svg\trash.svg
+		resources\svg\upload.svg = resources\svg\upload.svg
+		resources\svg\x.svg = resources\svg\x.svg
 	EndProjectSection
 EndProject
 Global
@@ -68,15 +135,14 @@ Global
 		HideSolutionNode = FALSE
 	EndGlobalSection
 	GlobalSection(NestedProjects) = preSolution
-		{F2C8C786-FA23-4B63-934C-8CAA39D9BF95} = {BCC6E6D2-343B-4C48-854D-5FE3BBC3CB70}
 		{7372FCA6-B0BC-4968-A748-7ABB17A7929A} = {BCC6E6D2-343B-4C48-854D-5FE3BBC3CB70}
 		{75ED3C1E-D3C7-4546-9F2E-AC85859DDF4B} = {BCC6E6D2-343B-4C48-854D-5FE3BBC3CB70}
 		{D446E5C6-BDDE-4091-BD1A-EC57170003CF} = {BCC6E6D2-343B-4C48-854D-5FE3BBC3CB70}
 		{8BEA7793-3336-40ED-AB96-7FFB09FEB0F6} = {D446E5C6-BDDE-4091-BD1A-EC57170003CF}
-		{B334FFB0-4BD2-496E-BDC4-786620E019DA} = {F2C8C786-FA23-4B63-934C-8CAA39D9BF95}
-		{028FDA4B-AC3E-4A0E-9291-978E213F9B78} = {BCC6E6D2-343B-4C48-854D-5FE3BBC3CB70}
 		{86516D53-50D4-4FE2-9D8A-977A8F5EBDBD} = {BCC6E6D2-343B-4C48-854D-5FE3BBC3CB70}
 		{68A87938-5C2B-49F5-8AAA-8A34FBBFD854} = {BCC6E6D2-343B-4C48-854D-5FE3BBC3CB70}
 		{F266602F-1755-4A95-A11B-6C90C701C5BF} = {68A87938-5C2B-49F5-8AAA-8A34FBBFD854}
+		{21D2A42C-3F85-465C-9141-C106AFD92B68} = {68A87938-5C2B-49F5-8AAA-8A34FBBFD854}
+		{D4E9C905-4884-488E-B763-5BD39049C1B1} = {68A87938-5C2B-49F5-8AAA-8A34FBBFD854}
 	EndGlobalSection
 EndGlobal

background.js

@@ -10,63 +10,765 @@
 
 "use strict";
 
+const storage = (globalThis.messenger ?? browser).storage;
+
 let logger;
-// Startup
+let AiClassifier;
+let aiRules = [];
+let queue = Promise.resolve();
+let queuedCount = 0;
+let processing = false;
+let iconTimer = null;
+let timingStats = { count: 0, mean: 0, m2: 0, total: 0, last: -1 };
+let currentStart = 0;
+let logGetTiming = true;
+let htmlToMarkdown = false;
+let stripUrlParams = false;
+let altTextImages = false;
+let collapseWhitespace = false;
+let tokenReduction = false;
+let maxTokens = 4096;
+let TurndownService = null;
+let userTheme = 'auto';
+let currentTheme = 'light';
+let detectSystemTheme;
+let errorPending = false;
+let showDebugTab = false;
+const ERROR_NOTIFICATION_ID = 'sortana-error';
+
+function normalizeRules(rules) {
+    return Array.isArray(rules) ? rules.map(r => {
+        if (r.actions) {
+            if (!Array.isArray(r.accounts)) r.accounts = [];
+            if (!Array.isArray(r.folders)) r.folders = [];
+            r.enabled = r.enabled !== false;
+            return r;
+        }
+        const actions = [];
+        if (r.tag) actions.push({ type: 'tag', tagKey: r.tag });
+        if (r.moveTo) actions.push({ type: 'move', folder: r.moveTo });
+        if (r.copyTarget || r.copyTo) actions.push({ type: 'copy', copyTarget: r.copyTarget || r.copyTo });
+        const rule = { criterion: r.criterion, actions };
+        if (r.stopProcessing) rule.stopProcessing = true;
+        if (r.unreadOnly) rule.unreadOnly = true;
+        if (typeof r.minAgeDays === 'number') rule.minAgeDays = r.minAgeDays;
+        if (typeof r.maxAgeDays === 'number') rule.maxAgeDays = r.maxAgeDays;
+        if (Array.isArray(r.accounts)) rule.accounts = r.accounts;
+        if (Array.isArray(r.folders)) rule.folders = r.folders;
+        rule.enabled = r.enabled !== false;
+        return rule;
+    }) : [];
+}
+
+function iconPaths(name) {
+    return {
+        16: `resources/img/${name}-${currentTheme}-16.png`,
+        32: `resources/img/${name}-${currentTheme}-32.png`,
+        64: `resources/img/${name}-${currentTheme}-64.png`
+    };
+}
+
+
+const ICONS = {
+    logo: () => 'resources/img/logo.png',
+    circledots: () => iconPaths('circledots'),
+    circle: () => iconPaths('circle'),
+    average: () => iconPaths('average'),
+    error: () => iconPaths('x')
+};
+
+function setIcon(path) {
+    if (browser.browserAction) {
+        browser.browserAction.setIcon({ path });
+    }
+    if (browser.messageDisplayAction) {
+        browser.messageDisplayAction.setIcon({ path });
+    }
+}
+
+function updateActionIcon() {
+    let path = ICONS.logo();
+    if (errorPending) {
+        path = ICONS.error();
+    } else if (processing || queuedCount > 0) {
+        path = ICONS.circledots();
+    }
+    setIcon(path);
+}
+
+function showTransientIcon(factory, delay = 1500) {
+    if (errorPending) {
+        return;
+    }
+    clearTimeout(iconTimer);
+    const path = typeof factory === 'function' ? factory() : factory;
+    setIcon(path);
+    iconTimer = setTimeout(updateActionIcon, delay);
+}
+
+async function clearError() {
+    errorPending = false;
+    await storage.local.set({ errorPending: false });
+    await browser.notifications.clear(ERROR_NOTIFICATION_ID);
+    updateActionIcon();
+}
+
+function refreshMenuIcons() {
+    browser.menus.update('apply-ai-rules-list', { icons: iconPaths('eye') });
+    browser.menus.update('apply-ai-rules-display', { icons: iconPaths('eye') });
+    browser.menus.update('clear-ai-cache-list', { icons: iconPaths('trash') });
+    browser.menus.update('clear-ai-cache-display', { icons: iconPaths('trash') });
+    browser.menus.update('view-ai-reason-list', { icons: iconPaths('clipboarddata') });
+    browser.menus.update('view-ai-reason-display', { icons: iconPaths('clipboarddata') });
+}
+
+
+function byteSize(str) {
+    return new TextEncoder().encode(str || "").length;
+}
+
+function replaceInlineBase64(text) {
+    return text.replace(/(?:data:[^;]+;base64,)?[A-Za-z0-9+/=\r\n]{100,}/g,
+        m => tokenReduction ? '__BASE64__' : `[base64: ${byteSize(m)} bytes]`);
+}
+
+function sanitizeString(text) {
+    let t = String(text);
+    if (tokenReduction) {
+        t = t.replace(/<!--.*?-->/gs, '')
+            .replace(/url\([^\)]*\)/gi, 'url(__IMG__)');
+    }
+    if (stripUrlParams) {
+        t = t.replace(/https?:\/\/[^\s)]+/g, m => {
+            const idx = m.indexOf('?');
+            return idx >= 0 ? m.slice(0, idx) : m;
+        });
+    }
+    if (collapseWhitespace) {
+        t = t.replace(/[\u200B-\u200D\u2060\s]{2,}/g, ' ').replace(/\n{3,}/g, '\n\n');
+    }
+    return t;
+}
+
+function collectText(part, bodyParts, attachments) {
+    if (part.parts && part.parts.length) {
+        for (const p of part.parts) collectText(p, bodyParts, attachments);
+        return;
+    }
+    const ct = (part.contentType || "text/plain").toLowerCase();
+    const cd = (part.headers?.["content-disposition"]?.[0] || "").toLowerCase();
+    const body = String(part.body || "");
+    if (cd.includes("attachment") || !ct.startsWith("text/")) {
+        const nameMatch = /filename\s*=\s*"?([^";]+)/i.exec(cd) || /name\s*=\s*"?([^";]+)/i.exec(part.headers?.["content-type"]?.[0] || "");
+        const name = nameMatch ? nameMatch[1] : "";
+        attachments.push(`${name} (${ct}, ${part.size || byteSize(body)} bytes)`);
+    } else if (ct.startsWith("text/html")) {
+        const doc = new DOMParser().parseFromString(body, 'text/html');
+        if (tokenReduction) {
+            doc.querySelectorAll('script,style').forEach(el => el.remove());
+            const walker = doc.createTreeWalker(doc, NodeFilter.SHOW_COMMENT);
+            let node;
+            while ((node = walker.nextNode())) {
+                node.parentNode.removeChild(node);
+            }
+            doc.querySelectorAll('*').forEach(el => {
+                for (const attr of Array.from(el.attributes)) {
+                    if (!['href','src','alt'].includes(attr.name)) {
+                        el.removeAttribute(attr.name);
+                    }
+                }
+            });
+        }
+        doc.querySelectorAll('img').forEach(img => {
+            const alt = img.getAttribute('alt') || '';
+            const text = altTextImages ? alt : '__IMG__';
+            img.replaceWith(doc.createTextNode(text));
+        });
+        if (stripUrlParams) {
+            doc.querySelectorAll('[href]').forEach(a => {
+                const href = a.getAttribute('href');
+                if (href) a.setAttribute('href', href.split('?')[0]);
+            });
+            doc.querySelectorAll('[src]').forEach(e => {
+                const src = e.getAttribute('src');
+                if (src) e.setAttribute('src', src.split('?')[0]);
+            });
+        }
+        if (htmlToMarkdown && TurndownService) {
+            try {
+                const td = new TurndownService();
+                const md = sanitizeString(td.turndown(doc.body.innerHTML || body));
+                bodyParts.push(replaceInlineBase64(`[HTML Body converted to Markdown]\n${md}`));
+            } catch (e) {
+                bodyParts.push(replaceInlineBase64(sanitizeString(doc.body.textContent || "")));
+            }
+        } else {
+            bodyParts.push(replaceInlineBase64(sanitizeString(doc.body.textContent || "")));
+        }
+    } else {
+        bodyParts.push(replaceInlineBase64(sanitizeString(body)));
+    }
+}
+
+function collectRawText(part, bodyParts, attachments) {
+    if (part.parts && part.parts.length) {
+        for (const p of part.parts) collectRawText(p, bodyParts, attachments);
+        return;
+    }
+    const ct = (part.contentType || "text/plain").toLowerCase();
+    const cd = (part.headers?.["content-disposition"]?.[0] || "").toLowerCase();
+    const body = String(part.body || "");
+    if (cd.includes("attachment") || !ct.startsWith("text/")) {
+        const nameMatch = /filename\s*=\s*"?([^";]+)/i.exec(cd) || /name\s*=\s*"?([^";]+)/i.exec(part.headers?.["content-type"]?.[0] || "");
+        const name = nameMatch ? nameMatch[1] : "";
+        attachments.push(`${name} (${ct}, ${part.size || byteSize(body)} bytes)`);
+    } else if (ct.startsWith("text/html")) {
+        const doc = new DOMParser().parseFromString(body, 'text/html');
+        bodyParts.push(doc.body.textContent || "");
+    } else {
+        bodyParts.push(body);
+    }
+}
+
+function buildEmailText(full, applyTransforms = true) {
+    const bodyParts = [];
+    const attachments = [];
+    const collect = applyTransforms ? collectText : collectRawText;
+    collect(full, bodyParts, attachments);
+    const headers = Object.entries(full.headers || {})
+        .map(([k, v]) => `${k}: ${v.join(' ')}`)
+        .join('\n');
+    const attachInfo = `Attachments: ${attachments.length}` +
+        (attachments.length ? "\n" + attachments.map(a => ` - ${a}`).join('\n') : "");
+    let combined = `${headers}\n${attachInfo}\n\n${bodyParts.join('\n')}`.trim();
+    if (applyTransforms && tokenReduction) {
+        const seen = new Set();
+        combined = combined.split('\n').filter(l => {
+            if (seen.has(l)) return false;
+            seen.add(l);
+            return true;
+        }).join('\n');
+    }
+    return applyTransforms ? sanitizeString(combined) : combined;
+}
+
+function updateTimingStats(elapsed) {
+    const t = timingStats;
+    t.count += 1;
+    t.total += elapsed;
+    t.last = elapsed;
+    const delta = elapsed - t.mean;
+    t.mean += delta / t.count;
+    t.m2 += delta * (elapsed - t.mean);
+}
+
+async function getAllMessageIds(list) {
+    const ids = [];
+    if (!list) {
+        return ids;
+    }
+    let page = list;
+    ids.push(...(page.messages || []).map(m => m.id));
+    while (page.id) {
+        page = await messenger.messages.continueList(page.id);
+        ids.push(...(page.messages || []).map(m => m.id));
+    }
+    return ids;
+}
+
+async function processMessage(id) {
+    processing = true;
+    currentStart = Date.now();
+    queuedCount--;
+    updateActionIcon();
+    try {
+        const full = await messenger.messages.getFull(id);
+        const originalText = buildEmailText(full, false);
+        let text = buildEmailText(full);
+        if (tokenReduction && maxTokens > 0) {
+            const limit = Math.floor(maxTokens * 0.9);
+            if (text.length > limit) {
+                text = text.slice(0, limit);
+            }
+        }
+        if (showDebugTab) {
+            await storage.local.set({ lastFullText: originalText, lastPromptText: text });
+        }
+        let hdr;
+        let currentTags = [];
+        let alreadyRead = false;
+        let identityId = null;
+        try {
+            hdr = await messenger.messages.get(id);
+            currentTags = Array.isArray(hdr.tags) ? [...hdr.tags] : [];
+            alreadyRead = hdr.read === true;
+            const ids = await messenger.identities.list(hdr.folder.accountId);
+            identityId = ids[0]?.id || null;
+        } catch (e) {
+            currentTags = [];
+            alreadyRead = false;
+            identityId = null;
+        }
+
+        for (const rule of aiRules) {
+            if (rule.enabled === false) {
+                continue;
+            }
+            if (hdr && Array.isArray(rule.accounts) && rule.accounts.length &&
+                !rule.accounts.includes(hdr.folder.accountId)) {
+                continue;
+            }
+            if (hdr && Array.isArray(rule.folders) && rule.folders.length &&
+                !rule.folders.includes(hdr.folder.path)) {
+                continue;
+            }
+            if (rule.unreadOnly && alreadyRead) {
+                continue;
+            }
+            if (hdr && (typeof rule.minAgeDays === 'number' || typeof rule.maxAgeDays === 'number')) {
+                const msgTime = new Date(hdr.date).getTime();
+                if (!isNaN(msgTime)) {
+                    const ageDays = (Date.now() - msgTime) / 86400000;
+                    if (typeof rule.minAgeDays === 'number' && ageDays < rule.minAgeDays) {
+                        continue;
+                    }
+                    if (typeof rule.maxAgeDays === 'number' && ageDays > rule.maxAgeDays) {
+                        continue;
+                    }
+                }
+            }
+            const cacheKey = await AiClassifier.buildCacheKey(id, rule.criterion);
+            const matched = await AiClassifier.classifyText(text, rule.criterion, cacheKey);
+            if (matched) {
+                for (const act of (rule.actions || [])) {
+                    if (act.type === 'tag' && act.tagKey) {
+                        if (!currentTags.includes(act.tagKey)) {
+                            currentTags.push(act.tagKey);
+                            await messenger.messages.update(id, { tags: currentTags });
+                        }
+                    } else if (act.type === 'move' && act.folder) {
+                        await messenger.messages.move([id], act.folder);
+                    } else if (act.type === 'copy' && act.copyTarget) {
+                        await messenger.messages.copy([id], act.copyTarget);
+                    } else if (act.type === 'junk') {
+                        await messenger.messages.update(id, { junk: !!act.junk });
+                    } else if (act.type === 'read') {
+                        await messenger.messages.update(id, { read: !!act.read });
+                    } else if (act.type === 'flag') {
+                        await messenger.messages.update(id, { flagged: !!act.flagged });
+                    } else if (act.type === 'delete') {
+                        await messenger.messages.delete([id]);
+                    } else if (act.type === 'archive') {
+                        await messenger.messages.archive([id]);
+                    } else if (act.type === 'forward' && act.address && identityId) {
+                        await browser.compose.beginForward(id, { to: [act.address], identityId });
+                    } else if (act.type === 'reply' && act.replyType && identityId) {
+                        await browser.compose.beginReply(id, { replyType: act.replyType, identityId });
+                    }
+                }
+                if (rule.stopProcessing) {
+                    break;
+                }
+            }
+        }
+        processing = false;
+        const elapsed = Date.now() - currentStart;
+        currentStart = 0;
+        updateTimingStats(elapsed);
+        await storage.local.set({ classifyStats: timingStats });
+        showTransientIcon(ICONS.circle);
+    } catch (e) {
+        processing = false;
+        const elapsed = Date.now() - currentStart;
+        currentStart = 0;
+        updateTimingStats(elapsed);
+        await storage.local.set({ classifyStats: timingStats, errorPending: true });
+        errorPending = true;
+        logger.aiLog("failed to apply AI rules", { level: 'error' }, e);
+        setIcon(ICONS.error());
+        browser.notifications.create(ERROR_NOTIFICATION_ID, {
+            type: 'basic',
+            iconUrl: browser.runtime.getURL('resources/img/logo.png'),
+            title: 'Sortana Error',
+            message: 'Failed to apply AI rules',
+            buttons: [{ title: 'Dismiss' }]
+        });
+    }
+}
+async function applyAiRules(idsInput) {
+    const ids = Array.isArray(idsInput) ? idsInput : [idsInput];
+    if (!ids.length) return queue;
+
+    if (!aiRules.length) {
+        const { aiRules: stored } = await storage.local.get("aiRules");
+        aiRules = normalizeRules(stored);
+    }
+
+    for (const msg of ids) {
+        const id = msg?.id ?? msg;
+        queuedCount++;
+        updateActionIcon();
+        queue = queue.then(() => processMessage(id));
+    }
+
+    return queue;
+}
+
+async function clearCacheForMessages(idsInput) {
+    const ids = Array.isArray(idsInput) ? idsInput : [idsInput];
+    if (!ids.length) return;
+
+    if (!aiRules.length) {
+        const { aiRules: stored } = await storage.local.get("aiRules");
+        aiRules = normalizeRules(stored);
+    }
+
+    const keys = [];
+    for (const msg of ids) {
+        const id = msg?.id ?? msg;
+        for (const rule of aiRules) {
+            const key = await AiClassifier.buildCacheKey(id, rule.criterion);
+            keys.push(key);
+        }
+    }
+    if (keys.length) {
+        await AiClassifier.removeCacheEntries(keys);
+        showTransientIcon(ICONS.circle);
+    }
+}
+
 (async () => {
     logger = await import(browser.runtime.getURL("logger.js"));
-    logger.aiLog("background.js loaded – ready to classify", {debug: true});
+    ({ detectSystemTheme } = await import(browser.runtime.getURL('modules/themeUtils.js')));
     try {
-        const store = await browser.storage.local.get(["endpoint", "templateName", "customTemplate", "customSystemPrompt", "aiParams", "debugLogging"]);
-        logger.setDebug(store.debugLogging);
-        await browser.aiFilter.initConfig(store);
-        logger.aiLog("configuration loaded", {debug: true}, store);
-        try {
-            await browser.DomContentScript.registerWindow(
-                "chrome://messenger/content/FilterEditor.xhtml",
-                "resource://aifilter/content/filterEditor.js"
-            );
-            logger.aiLog("registered FilterEditor content script", {debug: true});
+        AiClassifier = await import(browser.runtime.getURL("modules/AiClassifier.js"));
+        logger.aiLog("AiClassifier imported", { debug: true });
+        const td = await import(browser.runtime.getURL("resources/js/turndown.js"));
+        TurndownService = td.default || td.TurndownService;
     } catch (e) {
-            logger.aiLog("failed to register content script", {level: 'error'}, e);
+        console.error("failed to import AiClassifier", e);
+        return;
+    }
+
+    try {
+        const store = await storage.local.get(["endpoint", "templateName", "customTemplate", "customSystemPrompt", "aiParams", "debugLogging", "htmlToMarkdown", "stripUrlParams", "altTextImages", "collapseWhitespace", "tokenReduction", "aiRules", "theme", "errorPending", "showDebugTab"]);
+        logger.setDebug(store.debugLogging);
+        await AiClassifier.setConfig(store);
+        userTheme = store.theme || 'auto';
+        currentTheme = userTheme === 'auto' ? await detectSystemTheme() : userTheme;
+        await AiClassifier.init();
+        htmlToMarkdown = store.htmlToMarkdown === true;
+        stripUrlParams = store.stripUrlParams === true;
+        altTextImages = store.altTextImages === true;
+        collapseWhitespace = store.collapseWhitespace === true;
+        tokenReduction = store.tokenReduction === true;
+        if (store.aiParams && typeof store.aiParams.max_tokens !== 'undefined') {
+            maxTokens = parseInt(store.aiParams.max_tokens) || maxTokens;
+        }
+        errorPending = store.errorPending === true;
+        showDebugTab = store.showDebugTab === true;
+        const savedStats = await storage.local.get('classifyStats');
+        if (savedStats.classifyStats && typeof savedStats.classifyStats === 'object') {
+            Object.assign(timingStats, savedStats.classifyStats);
+        }
+        if (typeof timingStats.last !== 'number') {
+            timingStats.last = -1;
+        }
+        aiRules = normalizeRules(store.aiRules);
+        logger.aiLog("configuration loaded", { debug: true }, store);
+        storage.onChanged.addListener(async changes => {
+            if (changes.aiRules) {
+                const newRules = changes.aiRules.newValue || [];
+                aiRules = normalizeRules(newRules);
+                logger.aiLog("aiRules updated from storage change", { debug: true }, aiRules);
+            }
+            if (changes.endpoint || changes.templateName || changes.customTemplate || changes.customSystemPrompt || changes.aiParams || changes.debugLogging) {
+                const config = {};
+                if (changes.endpoint) config.endpoint = changes.endpoint.newValue;
+                if (changes.templateName) config.templateName = changes.templateName.newValue;
+                if (changes.customTemplate) config.customTemplate = changes.customTemplate.newValue;
+                if (changes.customSystemPrompt) config.customSystemPrompt = changes.customSystemPrompt.newValue;
+                if (changes.aiParams) {
+                    config.aiParams = changes.aiParams.newValue;
+                    if (changes.aiParams.newValue && typeof changes.aiParams.newValue.max_tokens !== 'undefined') {
+                        maxTokens = parseInt(changes.aiParams.newValue.max_tokens) || maxTokens;
+                    }
+                }
+                if (changes.debugLogging) {
+                    config.debugLogging = changes.debugLogging.newValue === true;
+                    logger.setDebug(config.debugLogging);
+                }
+                await AiClassifier.setConfig(config);
+                logger.aiLog("AiClassifier config updated from storage change", { debug: true }, config);
+            }
+            if (changes.htmlToMarkdown) {
+                htmlToMarkdown = changes.htmlToMarkdown.newValue === true;
+                logger.aiLog("htmlToMarkdown updated from storage change", { debug: true }, htmlToMarkdown);
+            }
+            if (changes.stripUrlParams) {
+                stripUrlParams = changes.stripUrlParams.newValue === true;
+                logger.aiLog("stripUrlParams updated from storage change", { debug: true }, stripUrlParams);
+            }
+            if (changes.altTextImages) {
+                altTextImages = changes.altTextImages.newValue === true;
+                logger.aiLog("altTextImages updated from storage change", { debug: true }, altTextImages);
+            }
+            if (changes.collapseWhitespace) {
+                collapseWhitespace = changes.collapseWhitespace.newValue === true;
+                logger.aiLog("collapseWhitespace updated from storage change", { debug: true }, collapseWhitespace);
+            }
+            if (changes.tokenReduction) {
+                tokenReduction = changes.tokenReduction.newValue === true;
+                logger.aiLog("tokenReduction updated from storage change", { debug: true }, tokenReduction);
+            }
+            if (changes.showDebugTab) {
+                showDebugTab = changes.showDebugTab.newValue === true;
+            }
+            if (changes.errorPending) {
+                errorPending = changes.errorPending.newValue === true;
+                updateActionIcon();
+            }
+            if (changes.theme) {
+                userTheme = changes.theme.newValue || 'auto';
+                currentTheme = userTheme === 'auto' ? await detectSystemTheme() : userTheme;
+                updateActionIcon();
+                refreshMenuIcons();
+            }
+        });
+
+        if (browser.theme?.onUpdated) {
+            browser.theme.onUpdated.addListener(async () => {
+                if (userTheme === 'auto') {
+                    const theme = await detectSystemTheme();
+                    if (theme !== currentTheme) {
+                        currentTheme = theme;
+                        updateActionIcon();
+                        refreshMenuIcons();
+                    }
+                }
+            });
         }
     } catch (err) {
-        logger.aiLog("failed to load config", {level: 'error'}, err);
+        logger.aiLog("failed to load config", { level: 'error' }, err);
     }
-})();
 
-// Listen for messages from UI/devtools
-browser.runtime.onMessage.addListener((msg) => {
-    logger.aiLog("onMessage received", {debug: true}, msg);
+    logger.aiLog("background.js loaded – ready to classify", { debug: true });
+    updateActionIcon();
+    if (browser.messageDisplayAction) {
+        browser.messageDisplayAction.setTitle({ title: "Details" });
+        if (browser.messageDisplayAction.setLabel) {
+            browser.messageDisplayAction.setLabel({ label: "Details" });
+        }
 
-    if (msg?.type === "aiFilter:test") {
+    }
+
+    browser.menus.create({
+        id: "apply-ai-rules-list",
+        title: "Apply AI Rules",
+        contexts: ["message_list"],
+        icons: iconPaths('eye')
+    });
+    browser.menus.create({
+        id: "apply-ai-rules-display",
+        title: "Apply AI Rules",
+        contexts: ["message_display_action"],
+        icons: iconPaths('eye')
+    });
+    browser.menus.create({
+        id: "clear-ai-cache-list",
+        title: "Clear AI Cache",
+        contexts: ["message_list"],
+        icons: iconPaths('trash')
+    });
+    browser.menus.create({
+        id: "clear-ai-cache-display",
+        title: "Clear AI Cache",
+        contexts: ["message_display_action"],
+        icons: iconPaths('trash')
+    });
+    browser.menus.create({
+        id: "view-ai-reason-list",
+        title: "View Reasoning",
+        contexts: ["message_list"],
+        icons: iconPaths('clipboarddata')
+    });
+    browser.menus.create({
+        id: "view-ai-reason-display",
+        title: "View Reasoning",
+        contexts: ["message_display_action"],
+        icons: iconPaths('clipboarddata')
+    });
+    refreshMenuIcons();
+
+    browser.menus.onClicked.addListener(async (info, tab) => {
+        if (info.menuItemId === "apply-ai-rules-list" || info.menuItemId === "apply-ai-rules-display") {
+            let ids = info.messageId ? [info.messageId] : [];
+            if (info.selectedMessages) {
+                ids = await getAllMessageIds(info.selectedMessages);
+            }
+            await applyAiRules(ids);
+        } else if (info.menuItemId === "clear-ai-cache-list" || info.menuItemId === "clear-ai-cache-display") {
+            let ids = info.messageId ? [info.messageId] : [];
+            if (info.selectedMessages) {
+                ids = await getAllMessageIds(info.selectedMessages);
+            }
+            await clearCacheForMessages(ids);
+        } else if (info.menuItemId === "view-ai-reason-list" || info.menuItemId === "view-ai-reason-display") {
+            const [header] = await browser.messageDisplay.getDisplayedMessages(tab.id);
+            if (!header) { return; }
+
+            const url = `${browser.runtime.getURL("details.html")}?mid=${header.id}`;
+
+            await browser.tabs.create({ url });
+        }
+    });
+
+    // Listen for messages from UI/devtools
+    browser.runtime.onMessage.addListener(async (msg) => {
+        if ((msg?.type === "sortana:getTiming" && logGetTiming) || (msg?.type !== "sortana:getTiming")) {
+            logGetTiming = false;
+            logger.aiLog("onMessage received", { debug: true }, msg);
+        }
+
+        if (msg?.type === "sortana:test") {
             const { text = "", criterion = "" } = msg;
-        logger.aiLog("aiFilter:test – text", {debug: true}, text);
-        logger.aiLog("aiFilter:test – criterion", {debug: true}, criterion);
+            logger.aiLog("sortana:test – text", { debug: true }, text);
+            logger.aiLog("sortana:test – criterion", { debug: true }, criterion);
 
             try {
-            logger.aiLog("Calling browser.aiFilter.classify()", {debug: true});
-            const result = browser.aiFilter.classify(text, criterion);
-            logger.aiLog("classify() returned", {debug: true}, result);
+                logger.aiLog("Calling AiClassifier.classifyText()", { debug: true });
+                const result = await AiClassifier.classifyText(text, criterion);
+                logger.aiLog("classify() returned", { debug: true }, result);
                 return { match: result };
             }
             catch (err) {
-            logger.aiLog("Error in classify()", {level: 'error'}, err);
+                logger.aiLog("Error in classify()", { level: 'error' }, err);
                 // rethrow so the caller sees the failure
                 throw err;
             }
+        } else if (msg?.type === "sortana:clearCacheForDisplayed") {
+            try {
+                const msgs = await browser.messageDisplay.getDisplayedMessages();
+                const ids = msgs.map(m => m.id);
+                await clearCacheForMessages(ids);
+            } catch (e) {
+                logger.aiLog("failed to clear cache from message script", { level: 'error' }, e);
             }
-    else {
-        logger.aiLog("Unknown message type, ignoring", {level: 'warn'}, msg?.type);
+        } else if (msg?.type === "sortana:getReasons") {
+            try {
+                const id = msg.id;
+                const hdr = await messenger.messages.get(id);
+                const subject = hdr?.subject || "";
+                if (!aiRules.length) {
+                    const { aiRules: stored } = await storage.local.get("aiRules");
+                    aiRules = normalizeRules(stored);
                 }
-});
+                const reasons = [];
+                for (const rule of aiRules) {
+                    const key = await AiClassifier.buildCacheKey(id, rule.criterion);
+                    const reason = AiClassifier.getReason(key);
+                    if (reason) {
+                        reasons.push({ criterion: rule.criterion, reason });
+                    }
+                }
+                return { subject, reasons };
+            } catch (e) {
+                logger.aiLog("failed to collect reasons", { level: 'error' }, e);
+                return { subject: '', reasons: [] };
+            }
+        } else if (msg?.type === "sortana:getDetails") {
+            try {
+                const id = msg.id;
+                const hdr = await messenger.messages.get(id);
+                const subject = hdr?.subject || "";
+                if (!aiRules.length) {
+                    const { aiRules: stored } = await storage.local.get("aiRules");
+                    aiRules = normalizeRules(stored);
+                }
+                const results = [];
+                for (const rule of aiRules) {
+                    const key = await AiClassifier.buildCacheKey(id, rule.criterion);
+                    const matched = AiClassifier.getCachedResult(key);
+                    const reason = AiClassifier.getReason(key);
+                    if (matched !== null || reason) {
+                        results.push({ criterion: rule.criterion, matched, reason });
+                    }
+                }
+                return { subject, results };
+            } catch (e) {
+                logger.aiLog("failed to collect details", { level: 'error' }, e);
+                return { subject: '', results: [] };
+            }
+        } else if (msg?.type === "sortana:getDisplayedMessages") {
+            try {
+                const [tab] = await browser.tabs.query({ active: true, currentWindow: true });
+                const messages = await browser.messageDisplay.getDisplayedMessages(tab?.id);
+                const ids = messages.map(hdr => hdr.id);
 
-// Catch any unhandled rejections
-window.addEventListener("unhandledrejection", ev => {
-    logger.aiLog("Unhandled promise rejection", {level: 'error'}, ev.reason);
-});
+                return { ids };
+            } catch (e) {
+                logger.aiLog("failed to get displayed messages", { level: 'error' }, e);
+                return { messages: [] };
+            }
+        } else if (msg?.type === "sortana:clearCacheForMessage") {
+            try {
+                await clearCacheForMessages([msg.id]);
+                return { ok: true };
+            } catch (e) {
+                logger.aiLog("failed to clear cache for message", { level: 'error' }, e);
+                return { ok: false };
+            }
+        } else if (msg?.type === "sortana:getQueueCount") {
+            return { count: queuedCount + (processing ? 1 : 0) };
+        } else if (msg?.type === "sortana:getTiming") {
+            const t = timingStats;
+            const std = t.count > 1 ? Math.sqrt(t.m2 / (t.count - 1)) : 0;
+            return {
+                count: queuedCount + (processing ? 1 : 0),
+                current: currentStart ? Date.now() - currentStart : -1,
+                last: t.last,
+                runs: t.count,
+                average: t.mean,
+                total: t.total,
+                stddev: std
+            };
+        } else {
+            logger.aiLog("Unknown message type, ignoring", { level: 'warn' }, msg?.type);
+        }
+    });
 
-browser.runtime.onInstalled.addListener(async ({ reason }) => {
+    // Automatically classify new messages
+    if (typeof messenger !== "undefined" && messenger.messages?.onNewMailReceived) {
+        messenger.messages.onNewMailReceived.addListener(async (folder, messages) => {
+            logger.aiLog("onNewMailReceived", { debug: true }, messages);
+            const ids = (messages?.messages || messages || []).map(m => m.id ?? m);
+            await applyAiRules(ids);
+        });
+    } else {
+        logger.aiLog("messenger.messages API unavailable, skipping new mail listener", { level: 'warn' });
+    }
+
+    // Catch any unhandled rejections
+    window.addEventListener("unhandledrejection", ev => {
+        logger.aiLog("Unhandled promise rejection", { level: 'error' }, ev.reason);
+    });
+
+    browser.notifications.onClicked.addListener(id => {
+        if (id === ERROR_NOTIFICATION_ID) {
+            clearError();
+        }
+    });
+
+    browser.notifications.onButtonClicked.addListener((id) => {
+        if (id === ERROR_NOTIFICATION_ID) {
+            clearError();
+        }
+    });
+
+    browser.runtime.onInstalled.addListener(async ({ reason }) => {
         if (reason === "install") {
             await browser.runtime.openOptionsPage();
         }
-});
+    });
+
+})();

build-xpi.ps1

@@ -26,7 +26,7 @@ if (-not $version) {
 }
 
 # 4) Define output names & clean up
-$xpiName = "ai-filter-$version.xpi"
+$xpiName = "sortana-$version.xpi"
 $zipPath = Join-Path $ReleaseDir "ai-filter-$version.zip"
 $xpiPath = Join-Path $ReleaseDir $xpiName
 

content/filterEditor.js

@@ -1,50 +0,0 @@
-(function() {
-  function patch(container) {
-    if (!container || container.getAttribute("ai-filter-patched") === "true") {
-      return;
-    }
-    while (container.firstChild) {
-      container.firstChild.remove();
-    }
-    let frag = window.MozXULElement.parseXULToFragment(
-      `<html:input class="search-value-textbox flexinput ai-filter-textbox" inherits="disabled"
-        onchange="this.parentNode.setAttribute('value', this.value); this.parentNode.value=this.value;">
-      </html:input>`
-    );
-    container.appendChild(frag);
-    if (container.hasAttribute("value")) {
-      container.firstChild.value = container.getAttribute("value");
-    }
-    container.classList.add("flexelementcontainer");
-    container.setAttribute("ai-filter-patched", "true");
-  }
-
-  function check(node) {
-    if (!(node instanceof Element)) {
-      return;
-    }
-    if (
-      node.classList.contains("search-value-custom") &&
-      node.getAttribute("searchAttribute") === "aifilter#classification"
-    ) {
-      patch(node);
-    }
-    node
-      .querySelectorAll('.search-value-custom[searchAttribute="aifilter#classification"]')
-      .forEach(patch);
-  }
-
-  const observer = new MutationObserver(mutations => {
-    for (let mutation of mutations) {
-      if (mutation.type === "childList") {
-        mutation.addedNodes.forEach(check);
-      } else if (mutation.type === "attributes") {
-        check(mutation.target);
-      }
-    }
-  });
-
-  const termList = document.getElementById("searchTermList") || document;
-  observer.observe(termList, { childList: true, attributes: true, subtree: true });
-  check(termList);
-})();

details.html

@@ -0,0 +1,20 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8">
+  <title>AI Details</title>
+  <link rel="stylesheet" href="options/bulma.css">
+</head>
+<body>
+  <section class="section">
+    <div class="container">
+      <h1 class="title" id="subject"></h1>
+      <div id="rules"></div>
+      <div class="buttons mt-4">
+        <button class="button is-danger" id="clear">Clear Cache</button>
+      </div>
+    </div>
+  </section>
+  <script type="module" src="details.js"></script>
+</body>
+</html>

details.js

@@ -0,0 +1,75 @@
+const aiLog = (await import(browser.runtime.getURL("logger.js"))).aiLog;
+const storage = (globalThis.messenger ?? browser).storage;
+const { detectSystemTheme } = await import(browser.runtime.getURL('modules/themeUtils.js'));
+const { theme } = await storage.local.get('theme');
+const mode = (theme || 'auto') === 'auto'
+  ? await detectSystemTheme()
+  : theme;
+document.documentElement.dataset.theme = mode;
+
+const qMid = parseInt(new URLSearchParams(location.search).get("mid"), 10);
+if (!isNaN(qMid)) {
+  loadMessage(qMid);
+} else {
+  const { ids } = await browser.runtime.sendMessage({
+    type: "sortana:getDisplayedMessages",
+  });
+  if (ids && ids[0]) {
+    loadMessage(ids[0]);
+  } else {
+      const tabs = await browser.tabs.query({ active: true, currentWindow: true });
+      const tabId = tabs[0]?.id;
+      const msgs = tabId ? await browser.messageDisplay.getDisplayedMessages(tabId) : [];
+      let id = msgs[0]?.id;
+      if (id) {
+          loadMessage(id);
+      }
+      else {
+          aiLog("Details popup: no displayed message found");
+      }
+  }
+}
+
+async function loadMessage(id) {
+  const storage = (globalThis.messenger ?? browser).storage;
+  const logMod = await import(browser.runtime.getURL('logger.js'));
+  const { debugLogging } = await storage.local.get('debugLogging');
+  logMod.setDebug(debugLogging === true);
+  const log = logMod.aiLog;
+
+  log('details page loaded', { debug: true });
+  try {
+    log('requesting message details', {}, id);
+    const { subject, results } = await browser.runtime.sendMessage({ type: 'sortana:getDetails', id });
+    log('received details', { debug: true }, { subject, results });
+    document.getElementById('subject').textContent = subject;
+    const container = document.getElementById('rules');
+    for (const r of results) {
+      log('rendering rule result', { debug: true }, r);
+      const article = document.createElement('article');
+      const color = r.matched === true ? 'is-success' : 'is-danger';
+      article.className = `message ${color} mb-4`;
+      const header = document.createElement('div');
+      header.className = 'message-header';
+      header.innerHTML = `<p>${r.criterion}</p>`;
+      const body = document.createElement('div');
+      body.className = 'message-body';
+      const status = document.createElement('p');
+      status.textContent = r.matched ? 'Matched' : 'Did not match';
+      const pre = document.createElement('pre');
+      pre.textContent = r.reason || '';
+      body.appendChild(status);
+      body.appendChild(pre);
+      article.appendChild(header);
+      article.appendChild(body);
+      container.appendChild(article);
+    }
+    document.getElementById('clear').addEventListener('click', async () => {
+      log('clearing cache for message', {}, id);
+      await browser.runtime.sendMessage({ type: 'sortana:clearCacheForMessage', id });
+      window.close();
+    });
+  } catch (e) {
+    log('failed to load details', { level: 'error' }, e);
+  }
+}

experiment/DomContentScript/implementation.js

@@ -1,80 +0,0 @@
-
-
-var { AppConstants } = ChromeUtils.importESModule("resource://gre/modules/AppConstants.sys.mjs");
-var DomContent_ESM = parseInt(AppConstants.MOZ_APP_VERSION, 10) >= 128;
-
-var { ExtensionCommon } = ChromeUtils.importESModule(
-  "resource://gre/modules/ExtensionCommon.sys.mjs"
-);
-
-var { ExtensionUtils } = DomContent_ESM
-  ? ChromeUtils.importESModule("resource://gre/modules/ExtensionUtils.sys.mjs")
-  : ChromeUtils.import("resource://gre/modules/ExtensionUtils.jsm");
-
-var { ExtensionError } = ExtensionUtils;
-
-var registeredWindows = new Map();
-
-var DomContentScript = class extends ExtensionCommon.ExtensionAPI {
-  constructor(extension) {
-    super(extension);
-    
-    this._windowListener = {
-      // nsIWindowMediatorListener functions
-      onOpenWindow(appWindow) {
-        // A new window has opened.
-        let domWindow = appWindow.docShell.domWindow;
-
-        /**
-         * Set up listeners to run the callbacks on the given window.
-         *
-         * @param aWindow {nsIDOMWindow}  The window to set up.
-         * @param aID {String} Optional.  ID of the new caller that has registered right now.
-         */
-        domWindow.addEventListener(
-          "DOMContentLoaded",
-          function() {
-            // do stuff
-            let windowChromeURL = domWindow.document.location.href;
-            if (registeredWindows.has(windowChromeURL)) {
-              let jsPath = registeredWindows.get(windowChromeURL);
-              Services.scriptloader.loadSubScript(jsPath, domWindow, "UTF-8");
-            }
-          },
-          { once: true }
-        );
-      },
-
-      onCloseWindow(appWindow) {
-        // One of the windows has closed.
-        let domWindow = appWindow.docShell.domWindow; // we don't need to do anything (script only loads once)
-      },
-    };
-   
-    Services.wm.addListener(this._windowListener);
-   
-  }
-  
-
-
-
-  
-  onShutdown(isAppShutdown) {
-    if (isAppShutdown) {
-      return; // the application gets unloaded anyway
-    }
-    Services.wm.removeListener(this._windowListener);
-  } 
-  
-  getAPI(context) {
-    /** API IMPLEMENTATION **/
-    return {
-      DomContentScript: {
-        // only returns something, if a user pref value is set
-        registerWindow: async function (windowUrl,jsPath) {
-          registeredWindows.set(windowUrl,jsPath);
-        }
-      },
-    };
-  }
-};

experiment/DomContentScript/schema.json

@@ -1,25 +0,0 @@
-[
-  {
-    "namespace": "DomContentScript",
-    "functions": [
-      {
-        "name": "registerWindow",
-        "type": "function",
-        "async": true,
-        "description": "Register a script for onDOMContentLoaded",
-        "parameters": [
-          {
-            "name": "windowUrl",
-            "type": "string",
-            "description": "chrome URL of the window "
-          },
-          {
-            "name": "jsPath",
-            "type": "string",
-            "description": "chrome URL of the script"
-          }
-        ]
-      }
-    ]
-  }
-]

experiment/api.js

@@ -1,100 +0,0 @@
-var { ExtensionCommon } = ChromeUtils.importESModule("resource://gre/modules/ExtensionCommon.sys.mjs");
-var { Services } = globalThis || ChromeUtils.importESModule("resource://gre/modules/Services.sys.mjs");
-var { MailServices } = ChromeUtils.importESModule("resource:///modules/MailServices.sys.mjs");
-
-var aiLog = (...args) => console.log("[ai-filter][api]", ...args);
-var setDebug = () => {};
-
-console.log("[ai-filter][api] Experiment API module loading");
-
-var resProto = Cc["@mozilla.org/network/protocol;1?name=resource"]
-    .getService(Ci.nsISubstitutingProtocolHandler);
-
-function registerResourceUrl(extension, namespace) {
-    aiLog(`[api] registerResourceUrl called for namespace="${namespace}"`, {debug: true});
-    if (resProto.hasSubstitution(namespace)) {
-        aiLog(`[api] namespace="${namespace}" already registered, skipping`, {debug: true});
-        return;
-    }
-    let uri = Services.io.newURI(".", null, extension.rootURI);
-    aiLog(`[api] setting substitution for "${namespace}" → ${uri.spec}`, {debug: true});
-    resProto.setSubstitutionWithFlags(namespace, uri, resProto.ALLOW_CONTENT_ACCESS);
-}
-
-var gTerm;
-var AIFilterMod;
-
-var aiFilter = class extends ExtensionCommon.ExtensionAPI {
-    async onStartup() {
-        let { extension } = this;
-
-        registerResourceUrl(extension, "aifilter");
-
-        // Import logger using the resource URL we just registered
-        let loggerMod = ChromeUtils.import("resource://aifilter/modules/logger.jsm");
-        aiLog = loggerMod.aiLog;
-        setDebug = loggerMod.setDebug;
-        aiLog("[api] onStartup()", {debug: true});
-
-
-        try {
-            aiLog("[api] importing ExpressionSearchFilter.jsm", {debug: true});
-            AIFilterMod = ChromeUtils.import("resource://aifilter/modules/ExpressionSearchFilter.jsm");
-            aiLog("[api] ExpressionSearchFilter.jsm import succeeded", {debug: true});
-        }
-        catch (err) {
-            aiLog("[api] failed to import ExpressionSearchFilter.jsm", {level: 'error'}, err);
-        }
-    }
-
-    onShutdown(isAppShutdown) {
-        aiLog("[api] onShutdown()", {debug: true}, isAppShutdown);
-        if (!isAppShutdown && resProto.hasSubstitution("aifilter")) {
-            aiLog("[api] removing substitution for namespace='aifilter'", {debug: true});
-            resProto.setSubstitution("aifilter", null);
-        }
-    }
-
-    getAPI(context) {
-        aiLog("[api] getAPI()", {debug: true});
-        return {
-            aiFilter: {
-                initConfig: async (config) => {
-                    try {
-                        if (AIFilterMod?.AIFilter?.setConfig) {
-                            AIFilterMod.AIFilter.setConfig(config);
-                            if (typeof config.debugLogging === "boolean") {
-                                setDebug(config.debugLogging);
-                            }
-                            aiLog("[api] configuration applied", {debug: true}, config);
-                        }
-                    } catch (err) {
-                        aiLog("[api] failed to apply config", {level: 'error'}, err);
-                    }
-                },
-                classify: (msg) => {
-                    aiLog("[api] classify() called with msg", {debug: true}, msg);
-                    try {
-                        if (!gTerm) {
-                            aiLog("[api] instantiating new ClassificationTerm", {debug: true});
-                            let mod = AIFilterMod || ChromeUtils.import("resource://aifilter/modules/ExpressionSearchFilter.jsm");
-                            gTerm = new mod.ClassificationTerm();
-                        }
-                        aiLog("[api] calling gTerm.match()", {debug: true});
-                        let matchResult = gTerm.match(
-                            msg.msgHdr,
-                            msg.value,
-                            Ci.nsMsgSearchOp.Contains
-                        );
-                        aiLog("[api] gTerm.match() returned", {debug: true}, matchResult);
-                        return matchResult;
-                    }
-                    catch (err) {
-                        aiLog("[api] error in classify()", {level: 'error'}, err);
-                        throw err;
-                    }
-                }
-            }
-        };
-    }
-};

experiment/schema.json

@@ -1,25 +0,0 @@
-[
-  {
-    "namespace": "aiFilter",
-    "functions": [
-      {
-        "name": "initConfig",
-        "type": "function",
-        "async": true,
-        "parameters": [
-          { "name": "config", "type": "any" }
-        ]
-      },
-      {
-        "name": "classify",
-        "type": "function",
-        "parameters": [
-          {
-            "name": "msg",
-            "type": "any"
-          }
-        ]
-      }
-    ]
-  }
-]

manifest.json

@@ -1,40 +1,49 @@
 {
   "manifest_version": 2,
-  "name": "AI Filter",
-  "version": "1.1",
+  "name": "Sortana",
+  "version": "2.2.0",
   "default_locale": "en-US",
-  "applications": { "gecko": { "id": "ai-filter@example" } },
+  "applications": {
+    "gecko": {
+      "id": "ai-filter@jordanwages",
+      "strict_min_version": "128.0",
+      "strict_max_version": "140.*"
+    }
+  },
   "icons": {
-    "16": "resources/img/logo.png",
-    "32": "resources/img/logo.png",
-    "48": "resources/img/logo.png",
-    "64": "resources/img/logo.png",
-    "96": "resources/img/logo.png",
-    "128": "resources/img/logo.png"
+    "16": "resources/img/logo16.png",
+    "32": "resources/img/logo32.png",
+    "48": "resources/img/logo48.png",
+    "64": "resources/img/logo64.png",
+    "96": "resources/img/logo96.png",
+    "128": "resources/img/logo128.png"
+  },
+  "browser_action": {
+    "default_icon": "resources/img/logo32.png"
+  },
+  "message_display_action": {
+    "default_icon": "resources/img/logo.png",
+    "default_title": "Details",
+    "default_label": "Details",
+    "default_popup": "details.html"
   },
   "background": { "scripts": [ "background.js" ] },
-  "experiment_apis": {
-    "aiFilter": {
-      "schema": "experiment/schema.json",
-      "parent": {
-        "scopes": [ "addon_parent" ],
-        "paths": [ [ "aiFilter" ] ],
-        "script": "experiment/api.js",
-        "events": [ "startup" ]
-      }
-    },
-    "DomContentScript": {
-      "schema": "experiment/DomContentScript/schema.json",
-      "parent": {
-        "scopes": [ "addon_parent" ],
-        "paths": [ [ "DomContentScript" ] ],
-        "script": "experiment/DomContentScript/implementation.js"
-      }
-    }
-  },
   "options_ui": {
     "page": "options/options.html",
-    "open_in_tab": false
+    "open_in_tab": true
   },
-  "permissions": [ "storage" ]
+    "permissions": [
+      "storage",
+      "messagesRead",
+      "messagesMove",
+      "messagesUpdate",
+      "messagesTagsList",
+      "accountsRead",
+      "menus",
+      "notifications",
+      "scripting",
+      "tabs",
+      "theme",
+      "compose"
+    ]
 }

modules/AiClassifier.js

@@ -0,0 +1,347 @@
+"use strict";
+import { aiLog, setDebug } from "../logger.js";
+import { DEFAULT_AI_PARAMS } from "./defaultParams.js";
+
+const storage = (globalThis.messenger ?? globalThis.browser).storage;
+
+let Services;
+try {
+  if (typeof globalThis !== "undefined" && globalThis.Services) {
+    Services = globalThis.Services;
+  } else if (typeof ChromeUtils !== "undefined" && ChromeUtils.importESModule) {
+    ({ Services } = ChromeUtils.importESModule("resource://gre/modules/Services.sys.mjs"));
+  }
+} catch (e) {
+  Services = undefined;
+}
+
+const SYSTEM_PREFIX = `You are an email-classification assistant.
+Read the email below and the classification criterion provided by the user.
+`;
+
+const DEFAULT_CUSTOM_SYSTEM_PROMPT = "Determine whether the email satisfies the user's criterion.";
+
+const SYSTEM_SUFFIX = `
+Return ONLY a JSON object on a single line of the form:
+{"match": true} - if the email satisfies the criterion
+{"match": false} - otherwise
+
+Do not add any other keys, text, or formatting.`;
+
+let gEndpoint = "http://127.0.0.1:5000/v1/classify";
+let gTemplateName = "openai";
+let gCustomTemplate = "";
+let gCustomSystemPrompt = DEFAULT_CUSTOM_SYSTEM_PROMPT;
+let gTemplateText = "";
+
+let gAiParams = Object.assign({}, DEFAULT_AI_PARAMS);
+
+let gCache = new Map();
+let gCacheLoaded = false;
+
+function sha256HexSync(str) {
+  try {
+    const hasher = Cc["@mozilla.org/security/hash;1"].createInstance(Ci.nsICryptoHash);
+    hasher.init(Ci.nsICryptoHash.SHA256);
+    const data = new TextEncoder().encode(str);
+    hasher.update(data, data.length);
+    const binary = hasher.finish(false);
+    return Array.from(binary, c => ("0" + c.charCodeAt(0).toString(16)).slice(-2)).join("");
+  } catch (e) {
+    aiLog(`sha256HexSync failed`, { level: 'error' }, e);
+    return "";
+  }
+}
+
+async function sha256Hex(str) {
+  if (typeof crypto?.subtle?.digest === "function") {
+    const buf = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(str));
+    return Array.from(new Uint8Array(buf), b => b.toString(16).padStart(2, "0")).join("");
+  }
+  return sha256HexSync(str);
+}
+
+async function resolveHeaderId(id) {
+  if (typeof id === "number" && typeof messenger?.messages?.get === "function") {
+    try {
+      const hdr = await messenger.messages.get(id);
+      if (hdr?.headerMessageId) {
+        return hdr.headerMessageId;
+      }
+    } catch (e) {
+      aiLog(`Failed to resolve headerMessageId for ${id}`, { level: 'warn' }, e);
+    }
+  }
+  return String(id);
+}
+
+async function buildCacheKey(id, criterion) {
+  const resolvedId = await resolveHeaderId(id);
+  if (Services) {
+    return sha256HexSync(`${resolvedId}|${criterion}`);
+  }
+  return sha256Hex(`${resolvedId}|${criterion}`);
+}
+
+async function loadCache() {
+  if (gCacheLoaded) {
+    return;
+  }
+  aiLog(`[AiClassifier] Loading cache`, {debug: true});
+  try {
+    const { aiCache, aiReasonCache } = await storage.local.get(["aiCache", "aiReasonCache"]);
+    if (aiCache) {
+      for (let [k, v] of Object.entries(aiCache)) {
+        if (v && typeof v === "object") {
+          gCache.set(k, { matched: v.matched ?? null, reason: v.reason || "" });
+        } else {
+          gCache.set(k, { matched: v, reason: "" });
+        }
+      }
+      aiLog(`[AiClassifier] Loaded ${gCache.size} cache entries`, {debug: true});
+    } else {
+      aiLog(`[AiClassifier] Cache is empty`, {debug: true});
+    }
+    if (aiReasonCache) {
+      aiLog(`[AiClassifier] Migrating ${Object.keys(aiReasonCache).length} reason entries`, {debug: true});
+      for (let [k, reason] of Object.entries(aiReasonCache)) {
+        let entry = gCache.get(k) || { matched: null, reason: "" };
+        entry.reason = reason;
+        gCache.set(k, entry);
+      }
+      await storage.local.remove("aiReasonCache");
+      await storage.local.set({ aiCache: Object.fromEntries(gCache) });
+    }
+  } catch (e) {
+    aiLog(`Failed to load cache`, {level: 'error'}, e);
+  }
+  gCacheLoaded = true;
+}
+
+
+async function saveCache(updatedKey, updatedValue) {
+  if (typeof updatedKey !== "undefined") {
+    aiLog(`[AiClassifier] ⮡ Persisting entry '${updatedKey}' → ${updatedValue}`, {debug: true});
+  }
+  try {
+    await storage.local.set({ aiCache: Object.fromEntries(gCache) });
+  } catch (e) {
+    aiLog(`Failed to save cache`, {level: 'error'}, e);
+  }
+}
+
+
+async function loadTemplate(name) {
+  try {
+    const url = typeof browser !== "undefined" && browser.runtime?.getURL
+      ? browser.runtime.getURL(`prompt_templates/${name}.txt`)
+      : `resource://aifilter/prompt_templates/${name}.txt`;
+    const res = await fetch(url);
+    if (res.ok) {
+      return await res.text();
+    }
+  } catch (e) {
+    aiLog(`Failed to load template '${name}':`, {level: 'error'}, e);
+  }
+  return "";
+}
+
+function loadTemplateSync(name) {
+  if (!Services?.tm?.spinEventLoopUntil) {
+    throw new Error("loadTemplateSync requires Services");
+  }
+  let text = "";
+  let done = false;
+  loadTemplate(name).then(t => { text = t; }).catch(() => {}).finally(() => { done = true; });
+  Services.tm.spinEventLoopUntil(() => done);
+  return text;
+}
+
+async function setConfig(config = {}) {
+  if (config.endpoint) {
+    gEndpoint = config.endpoint;
+  }
+  if (config.templateName) {
+    gTemplateName = config.templateName;
+  }
+  if (typeof config.customTemplate === "string") {
+    gCustomTemplate = config.customTemplate;
+  }
+  if (typeof config.customSystemPrompt === "string") {
+    gCustomSystemPrompt = config.customSystemPrompt;
+  }
+  if (config.aiParams && typeof config.aiParams === "object") {
+    for (let [k, v] of Object.entries(config.aiParams)) {
+      if (k in gAiParams && typeof v !== "undefined") {
+        gAiParams[k] = v;
+      }
+    }
+  }
+  if (typeof config.debugLogging === "boolean") {
+    setDebug(config.debugLogging);
+  }
+  if (gTemplateName === "custom") {
+    gTemplateText = gCustomTemplate;
+  } else if (Services?.tm?.spinEventLoopUntil) {
+    gTemplateText = loadTemplateSync(gTemplateName);
+  } else {
+    gTemplateText = await loadTemplate(gTemplateName);
+  }
+  aiLog(`[AiClassifier] Endpoint set to ${gEndpoint}`, {debug: true});
+  aiLog(`[AiClassifier] Template set to ${gTemplateName}`, {debug: true});
+}
+
+function buildSystemPrompt() {
+  return SYSTEM_PREFIX + (gCustomSystemPrompt || DEFAULT_CUSTOM_SYSTEM_PROMPT) + SYSTEM_SUFFIX;
+}
+
+function buildPrompt(body, criterion) {
+  aiLog(`[AiClassifier] Building prompt with criterion: "${criterion}"`, {debug: true});
+  const data = {
+    system: buildSystemPrompt(),
+    email: body,
+    query: criterion,
+  };
+  let template = gTemplateText || loadTemplateSync(gTemplateName);
+  return template.replace(/{{\s*(\w+)\s*}}/g, (m, key) => data[key] || "");
+}
+
+function getCachedResult(cacheKey) {
+  if (!gCacheLoaded) {
+    return null;
+  }
+  if (cacheKey && gCache.has(cacheKey)) {
+    aiLog(`[AiClassifier] Cache hit for key: ${cacheKey}`, {debug: true});
+    const entry = gCache.get(cacheKey);
+    return entry?.matched ?? null;
+  }
+  return null;
+}
+
+function getReason(cacheKey) {
+  if (!gCacheLoaded) {
+    return null;
+  }
+  const entry = gCache.get(cacheKey);
+  return cacheKey && entry ? entry.reason || null : null;
+}
+
+function buildPayload(text, criterion) {
+  let payloadObj = Object.assign({
+    prompt: buildPrompt(text, criterion)
+  }, gAiParams);
+  return JSON.stringify(payloadObj);
+}
+
+function parseMatch(result) {
+  const rawText = result.choices?.[0]?.text || "";
+  const thinkText = rawText.match(/<think>[\s\S]*?<\/think>/gi)?.join('') || '';
+  aiLog('[AiClassifier] ⮡ Reasoning:', {debug: true}, thinkText);
+  const cleanedText = rawText.replace(/<think>[\s\S]*?<\/think>/gi, "").trim();
+  aiLog('[AiClassifier] ⮡ Cleaned Response Text:', {debug: true}, cleanedText);
+  const obj = JSON.parse(cleanedText);
+  const matched = obj.matched === true || obj.match === true;
+  return { matched, reason: thinkText };
+}
+
+function cacheEntry(cacheKey, matched, reason) {
+  if (!cacheKey) {
+    return;
+  }
+  aiLog(`[AiClassifier] Caching entry '${cacheKey}'`, {debug: true});
+  const entry = gCache.get(cacheKey) || { matched: null, reason: "" };
+  if (typeof matched === "boolean") {
+    entry.matched = matched;
+  }
+  if (typeof reason === "string") {
+    entry.reason = reason;
+  }
+  gCache.set(cacheKey, entry);
+  saveCache(cacheKey, entry);
+}
+
+async function removeCacheEntries(keys = []) {
+  if (!Array.isArray(keys)) {
+    keys = [keys];
+  }
+  if (!gCacheLoaded) {
+    await loadCache();
+  }
+  let removed = false;
+  for (let key of keys) {
+    if (gCache.delete(key)) {
+      removed = true;
+      aiLog(`[AiClassifier] Removed cache entry '${key}'`, {debug: true});
+    }
+  }
+  if (removed) {
+    await saveCache();
+  }
+}
+
+async function clearCache() {
+  if (!gCacheLoaded) {
+    await loadCache();
+  }
+  if (gCache.size > 0) {
+    gCache.clear();
+    await saveCache();
+    aiLog(`[AiClassifier] Cleared cache`, {debug: true});
+  }
+}
+
+async function getCacheSize() {
+  if (!gCacheLoaded) {
+    await loadCache();
+  }
+  return gCache.size;
+}
+
+
+async function classifyText(text, criterion, cacheKey = null) {
+  if (!gCacheLoaded) {
+    await loadCache();
+  }
+  const cached = getCachedResult(cacheKey);
+  if (cached !== null) {
+    return cached;
+  }
+
+  const payload = buildPayload(text, criterion);
+  try {
+    await storage.local.set({ lastPayload: JSON.parse(payload) });
+  } catch (e) {
+    aiLog('failed to save last payload', { level: 'warn' }, e);
+  }
+
+  aiLog(`[AiClassifier] Sending classification request to ${gEndpoint}`, {debug: true});
+  aiLog(`[AiClassifier] Classification request payload:`, { debug: true }, payload);
+
+  try {
+    const response = await fetch(gEndpoint, {
+      method: "POST",
+      headers: { "Content-Type": "application/json" },
+      body: payload,
+    });
+
+    if (!response.ok) {
+      aiLog(`HTTP status ${response.status}`, {level: 'warn'});
+      return false;
+    }
+
+    const result = await response.json();
+    aiLog(`[AiClassifier] Received response:`, {debug: true}, result);
+    const parsed = parseMatch(result);
+    cacheEntry(cacheKey, parsed.matched, parsed.reason);
+    return parsed.matched;
+  } catch (e) {
+    aiLog(`HTTP request failed`, {level: 'error'}, e);
+    return false;
+  }
+}
+
+async function init() {
+  await loadCache();
+}
+
+export { classifyText, setConfig, removeCacheEntries, clearCache, getReason, getCachedResult, buildCacheKey, getCacheSize, init };

modules/ExpressionSearchFilter.jsm

@@ -1,290 +0,0 @@
-"use strict";
-var { ExtensionParent } = ChromeUtils.importESModule("resource://gre/modules/ExtensionParent.sys.mjs");
-var { MailServices }    = ChromeUtils.importESModule("resource:///modules/MailServices.sys.mjs");
-var { Services }        = globalThis || ChromeUtils.importESModule("resource://gre/modules/Services.sys.mjs");
-var { NetUtil }         = ChromeUtils.importESModule("resource://gre/modules/NetUtil.sys.mjs");
-var { MimeParser }      = ChromeUtils.importESModule("resource:///modules/mimeParser.sys.mjs");
-var { FileUtils }       = ChromeUtils.importESModule("resource://gre/modules/FileUtils.sys.mjs");
-var { aiLog, setDebug } = ChromeUtils.import("resource://aifilter/modules/logger.jsm");
-
-function sha256Hex(str) {
-  const hasher = Cc["@mozilla.org/security/hash;1"].createInstance(Ci.nsICryptoHash);
-  hasher.init(Ci.nsICryptoHash.SHA256);
-  const data = new TextEncoder().encode(str);
-  hasher.update(data, data.length);
-  const binary = hasher.finish(false);
-  return Array.from(binary, c => ("0" + c.charCodeAt(0).toString(16)).slice(-2)).join("");
-}
-
-var EXPORTED_SYMBOLS = ["AIFilter", "ClassificationTerm"];
-
-const SYSTEM_PREFIX = `You are an email-classification assistant.
-Read the email below and the classification criterion provided by the user.
-`;
-
-const DEFAULT_CUSTOM_SYSTEM_PROMPT = "Determine whether the email satisfies the user's criterion.";
-
-const SYSTEM_SUFFIX = `
-Return ONLY a JSON object on a single line of the form:
-{"match": true} - if the email satisfies the criterion
-{"match": false} - otherwise
-
-Do not add any other keys, text, or formatting.`;
-
-class CustomerTermBase {
-  constructor(nameId, operators) {
-    this.extension = ExtensionParent.GlobalManager.getExtension("ai-filter@example");
-    this.id = "aifilter#" + nameId;
-    this.name = this.extension.localeData.localizeMessage(nameId);
-    this.operators = operators;
-    this.cache = new Map();
-    this._cacheFile = Services.dirsvc.get("ProfD", Ci.nsIFile);
-    this._cacheFile.append("aifilter_cache.json");
-    this._loadCache();
-
-    aiLog(`[ExpressionSearchFilter] Initialized term base "${this.id}"`, {debug: true});
-  }
-
-  _loadCache() {
-    aiLog(`[ExpressionSearchFilter] Loading cache from ${this._cacheFile.path}` , {debug: true});
-    try {
-      if (this._cacheFile.exists()) {
-        let stream = Cc["@mozilla.org/network/file-input-stream;1"].createInstance(Ci.nsIFileInputStream);
-        stream.init(this._cacheFile, -1, 0, 0);
-        let data = NetUtil.readInputStreamToString(stream, stream.available());
-        stream.close();
-        aiLog(`[ExpressionSearchFilter] Cache file contents: ${data}`, {debug: true});
-        let obj = JSON.parse(data);
-        for (let [k, v] of Object.entries(obj)) {
-          aiLog(`[ExpressionSearchFilter] ⮡ Loaded entry '${k}' → ${v}`, {debug: true});
-          this.cache.set(k, v);
-        }
-        aiLog(`[ExpressionSearchFilter] Loaded ${this.cache.size} cache entries`, {debug: true});
-      } else {
-        aiLog(`[ExpressionSearchFilter] Cache file does not exist`, {debug: true});
-      }
-    } catch (e) {
-      aiLog(`Failed to load cache`, {level: 'error'}, e);
-    }
-  }
-
-  _saveCache(updatedKey, updatedValue) {
-    aiLog(`[ExpressionSearchFilter] Saving cache to ${this._cacheFile.path}`, {debug: true});
-    if (typeof updatedKey !== "undefined") {
-      aiLog(`[ExpressionSearchFilter] ⮡ Persisting entry '${updatedKey}' → ${updatedValue}`, {debug: true});
-    }
-    try {
-      let obj = Object.fromEntries(this.cache);
-      let data = JSON.stringify(obj);
-      let stream = Cc["@mozilla.org/network/file-output-stream;1"].createInstance(Ci.nsIFileOutputStream);
-      stream.init(this._cacheFile,
-                  FileUtils.MODE_WRONLY | FileUtils.MODE_CREATE | FileUtils.MODE_TRUNCATE,
-                  FileUtils.PERMS_FILE,
-                  0);
-      stream.write(data, data.length);
-      stream.close();
-    } catch (e) {
-      aiLog(`Failed to save cache`, {level: 'error'}, e);
-    }
-  }
-
-  getEnabled() {
-    aiLog(`[ExpressionSearchFilter] getEnabled() called on "${this.id}"`, {debug: true});
-    return true;
-  }
-
-  getAvailable() {
-    aiLog(`[ExpressionSearchFilter] getAvailable() called on "${this.id}"`, {debug: true});
-    return true;
-  }
-
-  getAvailableOperators() {
-    aiLog(`[ExpressionSearchFilter] getAvailableOperators() called on "${this.id}"`, {debug: true});
-    return this.operators;
-  }
-
-  getAvailableValues() {
-    aiLog(`[ExpressionSearchFilter] getAvailableValues() called on "${this.id}"`, {debug: true});
-    return null;
-  }
-
-  get attrib() {
-    aiLog(`[ExpressionSearchFilter] attrib getter called for "${this.id}"`, {debug: true});
-
-    //return Ci.nsMsgSearchAttrib.Custom;
-  }
-}
-
-function getPlainText(msgHdr) {
-  aiLog(`[ExpressionSearchFilter] Extracting plain text for message ID ${msgHdr.messageId}`, {debug: true});
-  let folder = msgHdr.folder;
-  if (!folder.getMsgInputStream) return "";
-  let reusable = {};
-  let stream = folder.getMsgInputStream(msgHdr, reusable);
-  let data = NetUtil.readInputStreamToString(stream, msgHdr.messageSize);
-  if (!reusable.value) stream.close();
-  let parser = Cc["@mozilla.org/parserutils;1"].getService(Ci.nsIParserUtils);
-  return parser.convertToPlainText(data,
-    Ci.nsIDocumentEncoder.OutputLFLineBreak |
-    Ci.nsIDocumentEncoder.OutputNoScriptContent |
-    Ci.nsIDocumentEncoder.OutputNoFramesContent |
-    Ci.nsIDocumentEncoder.OutputBodyOnly, 0);
-}
-
-let gEndpoint = "http://127.0.0.1:5000/v1/classify";
-let gTemplateName = "openai";
-let gCustomTemplate = "";
-let gCustomSystemPrompt = DEFAULT_CUSTOM_SYSTEM_PROMPT;
-let gTemplateText = "";
-
-let gAiParams = {
-  max_tokens: 4096,
-  temperature: 0.6,
-  top_p: 0.95,
-  seed: -1,
-  repetition_penalty: 1.0,
-  top_k: 20,
-  min_p: 0,
-  presence_penalty: 0,
-  frequency_penalty: 0,
-  typical_p: 1,
-  tfs: 1,
-};
-
-function loadTemplate(name) {
-  try {
-    let url = `resource://aifilter/prompt_templates/${name}.txt`;
-    let xhr = new XMLHttpRequest();
-    xhr.open("GET", url, false);
-    xhr.overrideMimeType("text/plain");
-    xhr.send();
-    if (xhr.status === 0 || xhr.status === 200) {
-      return xhr.responseText;
-    }
-  } catch (e) {
-    aiLog(`Failed to load template '${name}':`, {level: 'error'}, e);
-  }
-  return "";
-}
-
-function setConfig(config = {}) {
-    if (config.endpoint) {
-        gEndpoint = config.endpoint;
-    }
-    if (config.templateName) {
-        gTemplateName = config.templateName;
-    }
-    if (typeof config.customTemplate === "string") {
-        gCustomTemplate = config.customTemplate;
-    }
-    if (typeof config.customSystemPrompt === "string") {
-        gCustomSystemPrompt = config.customSystemPrompt;
-    }
-    if (config.aiParams && typeof config.aiParams === "object") {
-        for (let [k, v] of Object.entries(config.aiParams)) {
-            if (k in gAiParams && typeof v !== "undefined") {
-                gAiParams[k] = v;
-            }
-        }
-    }
-    if (typeof config.debugLogging === "boolean") {
-        setDebug(config.debugLogging);
-    }
-    gTemplateText = gTemplateName === "custom" ? gCustomTemplate : loadTemplate(gTemplateName);
-    aiLog(`[ExpressionSearchFilter] Endpoint set to ${gEndpoint}`, {debug: true});
-    aiLog(`[ExpressionSearchFilter] Template set to ${gTemplateName}`, {debug: true});
-}
-
-function buildSystemPrompt() {
-  return SYSTEM_PREFIX + (gCustomSystemPrompt || DEFAULT_CUSTOM_SYSTEM_PROMPT) + SYSTEM_SUFFIX;
-}
-
-function buildPrompt(body, criterion) {
-  aiLog(`[ExpressionSearchFilter] Building prompt with criterion: "${criterion}"`, {debug: true});
-  const data = {
-    system: buildSystemPrompt(),
-    email: body,
-    query: criterion,
-  };
-  let template = gTemplateText || loadTemplate(gTemplateName);
-  return template.replace(/{{\s*(\w+)\s*}}/g, (m, key) => data[key] || "");
-}
-
-class ClassificationTerm extends CustomerTermBase {
-  constructor() {
-    super("classification", [Ci.nsMsgSearchOp.Matches, Ci.nsMsgSearchOp.DoesntMatch]);
-    aiLog(`[ExpressionSearchFilter] ClassificationTerm constructed`, {debug: true});
-  }
-
-  needsBody() { return true; }
-
-  match(msgHdr, value, op) {
-    const opName = op === Ci.nsMsgSearchOp.Matches ? "matches" :
-                   op === Ci.nsMsgSearchOp.DoesntMatch ? "doesn't match" : `unknown (${op})`;
-    aiLog(`[ExpressionSearchFilter] Matching message ${msgHdr.messageId} using op "${opName}" and value "${value}"`, {debug: true});
-
-    let key = [msgHdr.messageId, op, value].map(sha256Hex).join("|");
-    if (this.cache.has(key)) {
-      aiLog(`[ExpressionSearchFilter] Cache hit for key: ${key}`, {debug: true});
-      return this.cache.get(key);
-    }
-
-    let body = getPlainText(msgHdr);
-    let payloadObj = Object.assign({
-      prompt: buildPrompt(body, value)
-    }, gAiParams);
-    let payload = JSON.stringify(payloadObj);
-
-
-    aiLog(`[ExpressionSearchFilter] Sending classification request to ${gEndpoint}`, {debug: true});
-
-    let matched = false;
-    try {
-      let xhr = new XMLHttpRequest();
-      xhr.open("POST", gEndpoint, false); // synchronous request
-      xhr.setRequestHeader("Content-Type", "application/json");
-      xhr.send(payload);
-
-      if (xhr.status < 200 || xhr.status >= 300) {
-        aiLog(`HTTP status ${xhr.status}`, {level: 'warn'});
-      } else {
-        const result = JSON.parse(xhr.responseText);
-        aiLog(`[ExpressionSearchFilter] Received response:`, {debug: true}, result);
-        const rawText = result.choices?.[0]?.text || "";
-        const thinkText = rawText.match(/<think>[\s\S]*?<\/think>/gi)?.join('') || '';
-        aiLog('[ExpressionSearchFilter] ⮡ Reasoning:', {debug: true}, thinkText);
-        const cleanedText = rawText.replace(/<think>[\s\S]*?<\/think>/gi, "").trim();
-        aiLog('[ExpressionSearchFilter] ⮡ Cleaned Response Text:', {debug: true}, cleanedText);
-        const obj = JSON.parse(cleanedText);
-        matched = obj.matched === true || obj.match === true;
-
-        aiLog(`[ExpressionSearchFilter] Caching entry '${key}' → ${matched}`, {debug: true});
-        this.cache.set(key, matched);
-        this._saveCache(key, matched);
-      }
-    } catch (e) {
-      aiLog(`HTTP request failed`, {level: 'error'}, e);
-    }
-
-    if (op === Ci.nsMsgSearchOp.DoesntMatch) {
-      matched = !matched;
-      aiLog(`[ExpressionSearchFilter] Operator is "doesn't match" → inverting to ${matched}`, {debug: true});
-    }
-
-    aiLog(`[ExpressionSearchFilter] Final match result: ${matched}`, {debug: true});
-    return matched;
-  }
-}
-
-(function register() {
-  aiLog(`[ExpressionSearchFilter] Registering custom filter term...`, {debug: true});
-  let term = new ClassificationTerm();
-  if (!MailServices.filters.getCustomTerm(term.id)) {
-    MailServices.filters.addCustomTerm(term);
-    aiLog(`[ExpressionSearchFilter] Registered term: ${term.id}`, {debug: true});
-  } else {
-    aiLog(`[ExpressionSearchFilter] Term already registered: ${term.id}`, {debug: true});
-  }
-})();
-
-var AIFilter = { setConfig };

modules/defaultParams.js

@@ -0,0 +1,16 @@
+"use strict";
+
+export const DEFAULT_AI_PARAMS = {
+  max_tokens: 4096,
+  temperature: 0.6,
+  top_p: 0.95,
+  seed: -1,
+  repetition_penalty: 1.0,
+  top_k: 20,
+  min_p: 0,
+  presence_penalty: 0,
+  frequency_penalty: 0,
+  typical_p: 1,
+  tfs: 1,
+};
+

modules/logger.jsm

@@ -1,26 +0,0 @@
-var EXPORTED_SYMBOLS = ['aiLog', 'setDebug'];
-let debugEnabled = false;
-
-function setDebug(value) {
-    debugEnabled = !!value;
-}
-
-function getCaller() {
-    try {
-        let stack = new Error().stack.split('\n');
-        if (stack.length >= 3) {
-            return stack[2].trim().replace(/^@?\s*\(?/,'').replace(/^at\s+/, '');
-        }
-    } catch (e) {}
-    return '';
-}
-
-function aiLog(message, opts = {}, ...args) {
-    const { level = 'log', debug = false } = opts;
-    if (debug && !debugEnabled) {
-        return;
-    }
-    const caller = getCaller();
-    const prefix = caller ? `[ai-filter][${caller}]` : '[ai-filter]';
-    console[level](`%c${prefix}`, 'color:#1c92d2;font-weight:bold', message, ...args);
-}

modules/themeUtils.js

@@ -0,0 +1,20 @@
+"use strict";
+
+export async function detectSystemTheme() {
+    try {
+        const t = await browser.theme.getCurrent();
+        const scheme = t?.properties?.color_scheme;
+        if (scheme === 'dark' || scheme === 'light') {
+            return scheme;
+        }
+        const color = t?.colors?.frame || t?.colors?.toolbar;
+        if (color && /^#/.test(color)) {
+            const r = parseInt(color.slice(1, 3), 16);
+            const g = parseInt(color.slice(3, 5), 16);
+            const b = parseInt(color.slice(5, 7), 16);
+            const lum = (0.299 * r + 0.587 * g + 0.114 * b) / 255;
+            return lum < 0.5 ? 'dark' : 'light';
+        }
+    } catch {}
+    return 'light';
+}

options/dataTransfer.js

@@ -0,0 +1,45 @@
+"use strict";
+const storage = (globalThis.messenger ?? browser).storage;
+const KEY_GROUPS = {
+    settings: [
+        'endpoint',
+        'templateName',
+        'customTemplate',
+        'customSystemPrompt',
+        'aiParams',
+        'debugLogging',
+        'htmlToMarkdown',
+        'stripUrlParams',
+        'altTextImages',
+        'collapseWhitespace'
+    ],
+    rules: ['aiRules'],
+    cache: ['aiCache']
+};
+
+function collectKeys(categories = Object.keys(KEY_GROUPS)) {
+    return categories.flatMap(cat => KEY_GROUPS[cat] || []);
+}
+
+export async function exportData(categories) {
+    const data = await storage.local.get(collectKeys(categories));
+    const blob = new Blob([JSON.stringify(data, null, 2)], { type: 'application/json' });
+    const url = URL.createObjectURL(blob);
+    const a = document.createElement('a');
+    a.href = url;
+    a.download = 'sortana-export.json';
+    document.body.appendChild(a);
+    a.click();
+    a.remove();
+    URL.revokeObjectURL(url);
+}
+
+export async function importData(file, categories) {
+    const text = await file.text();
+    const parsed = JSON.parse(text);
+    const data = {};
+    for (const key of collectKeys(categories)) {
+        if (key in parsed) data[key] = parsed[key];
+    }
+    await storage.local.set(data);
+}

options/options.html

@@ -4,201 +4,300 @@
     <meta charset="UTF-8">
     <meta name="viewport" content="width=device-width, initial-scale=1">
     <title>AI Filter Options</title>
+    <link rel="stylesheet" href="bulma.css">
     <style>
-        body {
-            font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
-            margin: 0;
-            padding: 0;
-            background-color: #f9f9f9;
-            color: #333;
+        :root {
+            --bulma-primary-h: 205deg;
+            --bulma-primary-s: 91%;
+            --bulma-primary-l: 57%;
+            --bulma-link-h: 205deg;
+            --bulma-link-s: 91%;
+            --bulma-link-l: 48%;
+            --bulma-info-h: 199deg;
+            --bulma-info-s: 44%;
+            --bulma-info-l: 82%;
+            --bulma-success-h: 145deg;
+            --bulma-success-s: 25%;
+            --bulma-success-l: 68%;
+            --bulma-warning-h: 44deg;
+            --bulma-warning-s: 86%;
+            --bulma-warning-l: 70%;
+            --bulma-danger-h: 0deg;
+            --bulma-danger-s: 49%;
+            --bulma-scheme-h: 200;
+            --bulma-scheme-s: 25%;
         }
 
-        header {
-            background-color: #ffffff;
-            padding: 20px 30px;
-            box-shadow: 0 2px 4px rgba(0,0,0,0.05);
-            display: flex;
-            align-items: center;
-            justify-content: center;
-        }
-
-            header img {
-                max-height: 40px;
-            }
-
-        main {
-            max-width: 700px;
-            margin: 30px auto;
-            padding: 0 20px;
-        }
-
-        .form-group {
-            margin-bottom: 20px;
-        }
-
-        label {
-            display: block;
-            font-weight: 600;
-            margin-bottom: 8px;
-            color: #222;
-        }
-
-        input[type="text"],
-        select,
-        textarea {
-            width: 100%;
-            padding: 10px 12px;
-            border: 1px solid #ccc;
-            border-radius: 4px;
-            font-size: 14px;
-            background-color: #fff;
-            transition: border-color 0.3s;
-        }
-
-            input[type="text"]:focus,
-            select:focus,
-            textarea:focus {
-                border-color: #007acc;
-                outline: none;
-            }
-
-        textarea {
-            resize: vertical;
-        }
-
-        .placeholder-text {
-            color: #888;
-            font-style: italic;
-        }
-
-        .button-group {
-            margin-top: 20px;
-            display: flex;
-            gap: 15px;
-            flex-wrap: wrap;
-        }
-
-        button {
-            padding: 10px 20px;
-            border: none;
-            border-radius: 4px;
-            font-size: 14px;
-            cursor: pointer;
-            transition: background-color 0.3s, transform 0.2s;
-        }
-
-            button#save {
-                background-color: #007acc;
-                color: white;
-            }
-
-                button#save:hover {
-                    background-color: #005fa3;
-                }
-
-            button#reset-system {
-                background-color: #f44336;
-                color: white;
-            }
-
-                button#reset-system:hover {
-                    background-color: #d32f2f;
-                }
-
-        @media (max-width: 600px) {
-            main {
-                margin: 20px 15px;
+        .tag {
+            --bulma-tag-h: 318;
         }
+        #diff-display {
+            white-space: pre-wrap;
+            font-family: monospace;
         }
     </style>
 </head>
 <body>
-    <header>
-        <img src="../resources/img/full-logo.png" alt="AI Filter Logo">
-    </header>
+    <section class="section">
+        <div class="container" id="options-container">
+            <figure class="has-text-centered mb-4">
+                <img data-icon="full-logo" src="../resources/img/full-logo.png" alt="AI Filter Logo" style="max-height:40px;">
+            </figure>
 
-    <main>
-        <div class="form-group">
-            <label for="endpoint">Endpoint:</label>
-            <input type="text" id="endpoint" placeholder="https://api.example.com">
+            <div class="level mb-4">
+                <div class="level-left">
+                    <div class="tabs" id="main-tabs">
+                        <ul>
+                            <li class="is-active" data-tab="settings"><a><span class="icon is-small"><img data-icon="settings" data-size="16" src="../resources/img/settings-light-16.png" alt=""></span><span>Settings</span></a></li>
+                            <li data-tab="rules"><a><span class="icon is-small"><img data-icon="clipboarddata" data-size="16" src="../resources/img/clipboarddata-light-16.png" alt=""></span><span>Rules</span></a></li>
+                            <li data-tab="maintenance"><a><span class="icon is-small"><img data-icon="gear" data-size="16" src="../resources/img/gear-light-16.png" alt=""></span><span>Maintenance</span></a></li>
+                            <li id="debug-tab-button" class="is-hidden" data-tab="debug"><a><span class="icon is-small"><img data-icon="average" data-size="16" src="../resources/img/average-light-16.png" alt=""></span><span>Debug</span></a></li>
+                        </ul>
+                    </div>
+                </div>
+                <div class="level-right">
+                    <button class="button is-primary" id="save" disabled>
+                        <span class="icon is-small"><img data-icon="flag" data-size="16" src="../resources/img/flag-light-16.png" alt=""></span>
+                        <span>Save</span>
+                    </button>
+                </div>
             </div>
 
-        <div class="form-group">
-            <label for="template">Prompt template:</label>
-            <select id="template">
+            <div id="settings-tab" class="tab-content">
+                <h2 class="title is-4">
+                    <span class="icon is-small"><img data-icon="settings" data-size="16" src="../resources/img/settings-light-16.png" alt=""></span>
+                    <span>Settings</span>
+                </h2>
+                <div class="field">
+                    <label class="label" for="endpoint">Endpoint</label>
+                    <div class="control">
+                        <input class="input" type="text" id="endpoint" placeholder="https://api.example.com">
+                    </div>
+                </div>
+
+                <div class="field">
+                    <label class="label" for="template">Prompt template</label>
+                    <div class="control">
+                        <div class="select is-fullwidth">
+                            <select id="template"></select>
+                        </div>
+                    </div>
+                </div>
+
+                <div id="custom-template-container" class="field is-hidden">
+                    <label class="label">Custom template</label>
+                    <div class="control">
+                        <textarea class="textarea" id="custom-template" rows="6" placeholder="Enter your custom template here..."></textarea>
+                    </div>
+                    <p class="help">Placeholders: {{system}}, {{email}}, {{query}}</p>
+                </div>
+
+                <div class="field">
+                    <label class="label" for="system-instructions">System instructions</label>
+                    <div class="control">
+                        <textarea class="textarea" id="system-instructions" rows="4" placeholder="Enter system instructions..."></textarea>
+                    </div>
+                </div>
+
+                <div class="field">
+                    <label class="label" for="theme-select">Theme</label>
+                    <div class="control">
+                        <div class="select">
+                            <select id="theme-select">
+                                <option value="auto">Match Thunderbird</option>
+                                <option value="light">Light</option>
+                                <option value="dark">Dark</option>
                             </select>
                         </div>
-
-        <div id="custom-template-container" class="form-group" style="display:none">
-            <label>Custom template</label>
-            <textarea id="custom-template" rows="6" cols="60" placeholder="Enter your custom template here..."></textarea>
-            <p class="placeholder-text">Placeholders: {{system}}, {{email}}, {{query}}</p>
+                    </div>
                 </div>
 
-        <div class="form-group">
-            <label for="system-instructions">System instructions:</label>
-            <textarea id="system-instructions" rows="4" cols="60" placeholder="Enter system instructions..."></textarea>
+                <div class="buttons">
+                    <button class="button is-danger" id="reset-system">
+                        <span class="icon is-small"><img data-icon="reply" data-size="16" src="../resources/img/reply-light-16.png" alt=""></span>
+                        <span>Reset to default</span>
+                    </button>
+                    <button class="button" id="toggle-advanced" type="button">
+                        <span class="icon is-small"><img data-icon="gear" data-size="16" src="../resources/img/gear-light-16.png" alt=""></span>
+                        <span>Advanced</span>
+                    </button>
                 </div>
 
-        <div class="button-group">
-            <button id="reset-system">Reset to default</button>
-            <button id="toggle-advanced" type="button">Advanced</button>
-            <button id="save">Save</button>
-        </div>
-
-        <div id="advanced-options" style="display:none">
-            <div class="form-group">
-                <label>
+                <div id="advanced-options" class="mt-4 is-hidden">
+                    <div class="field">
+                        <label class="checkbox">
                             <input type="checkbox" id="debug-logging"> Enable debug logging
                         </label>
                     </div>
-            <div class="form-group">
-                <label for="max_tokens">Max tokens:</label>
-                <input type="number" id="max_tokens">
+                    <div class="field">
+                        <label class="checkbox">
+                            <input type="checkbox" id="html-to-markdown"> Convert HTML body to Markdown
+                        </label>
                     </div>
-            <div class="form-group">
-                <label for="temperature">Temperature:</label>
-                <input type="number" step="0.01" id="temperature">
+                    <div class="field">
+                        <label class="checkbox">
+                            <input type="checkbox" id="strip-url-params"> Remove URL tracking parameters
+                        </label>
                     </div>
-            <div class="form-group">
-                <label for="top_p">Top P:</label>
-                <input type="number" step="0.01" id="top_p">
+                    <div class="field">
+                        <label class="checkbox">
+                            <input type="checkbox" id="alt-text-images"> Replace images with alt text
+                        </label>
                     </div>
-            <div class="form-group">
-                <label for="seed">Seed:</label>
-                <input type="number" id="seed">
+                    <div class="field">
+                        <label class="checkbox">
+                            <input type="checkbox" id="collapse-whitespace"> Collapse long whitespace
+                        </label>
                     </div>
-            <div class="form-group">
-                <label for="repetition_penalty">Repetition penalty:</label>
-                <input type="number" step="0.01" id="repetition_penalty">
+                    <div class="field">
+                        <label class="checkbox">
+                            <input type="checkbox" id="token-reduction"> Aggressive token reduction
+                        </label>
+                    </div>
+                    <div class="field">
+                        <label class="checkbox">
+                            <input type="checkbox" id="show-debug-tab"> Show debug information
+                        </label>
+                    </div>
+                    <div class="field">
+                        <label class="label" for="max_tokens">Max tokens</label>
+                        <div class="control">
+                            <input class="input" type="number" id="max_tokens">
+                        </div>
+                    </div>
+                    <div class="field">
+                        <label class="label" for="temperature">Temperature</label>
+                        <div class="control">
+                            <input class="input" type="number" step="0.01" id="temperature">
+                        </div>
+                    </div>
+                    <div class="field">
+                        <label class="label" for="top_p">Top P</label>
+                        <div class="control">
+                            <input class="input" type="number" step="0.01" id="top_p">
+                        </div>
+                    </div>
+                    <div class="field">
+                        <label class="label" for="seed">Seed</label>
+                        <div class="control">
+                            <input class="input" type="number" id="seed">
+                        </div>
+                    </div>
+                    <div class="field">
+                        <label class="label" for="repetition_penalty">Repetition penalty</label>
+                        <div class="control">
+                            <input class="input" type="number" step="0.01" id="repetition_penalty">
+                        </div>
+                    </div>
+                    <div class="field">
+                        <label class="label" for="top_k">Top K</label>
+                        <div class="control">
+                            <input class="input" type="number" id="top_k">
+                        </div>
+                    </div>
+                    <div class="field">
+                        <label class="label" for="min_p">Min P</label>
+                        <div class="control">
+                            <input class="input" type="number" step="0.01" id="min_p">
+                        </div>
+                    </div>
+                    <div class="field">
+                        <label class="label" for="presence_penalty">Presence penalty</label>
+                        <div class="control">
+                            <input class="input" type="number" step="0.01" id="presence_penalty">
+                        </div>
+                    </div>
+                    <div class="field">
+                        <label class="label" for="frequency_penalty">Frequency penalty</label>
+                        <div class="control">
+                            <input class="input" type="number" step="0.01" id="frequency_penalty">
+                        </div>
+                    </div>
+                    <div class="field">
+                        <label class="label" for="typical_p">Typical P</label>
+                        <div class="control">
+                            <input class="input" type="number" step="0.01" id="typical_p">
+                        </div>
+                    </div>
+                    <div class="field">
+                        <label class="label" for="tfs">TFS</label>
+                        <div class="control">
+                            <input class="input" type="number" step="0.01" id="tfs">
                         </div>
-            <div class="form-group">
-                <label for="top_k">Top K:</label>
-                <input type="number" id="top_k">
                     </div>
-            <div class="form-group">
-                <label for="min_p">Min P:</label>
-                <input type="number" step="0.01" id="min_p">
-            </div>
-            <div class="form-group">
-                <label for="presence_penalty">Presence penalty:</label>
-                <input type="number" step="0.01" id="presence_penalty">
-            </div>
-            <div class="form-group">
-                <label for="frequency_penalty">Frequency penalty:</label>
-                <input type="number" step="0.01" id="frequency_penalty">
-            </div>
-            <div class="form-group">
-                <label for="typical_p">Typical P:</label>
-                <input type="number" step="0.01" id="typical_p">
-            </div>
-            <div class="form-group">
-                <label for="tfs">TFS:</label>
-                <input type="number" step="0.01" id="tfs">
                 </div>
             </div>
-    </main>
 
+            <div id="rules-tab" class="tab-content is-hidden">
+                <h2 class="title is-4">
+                    <span class="icon is-small"><img data-icon="clipboarddata" data-size="16" src="../resources/img/clipboarddata-light-16.png" alt=""></span>
+                    <span>Classification Rules</span>
+                </h2>
+                <div id="rules-container"></div>
+                <button class="button is-link" id="add-rule" type="button">Add Rule</button>
+            </div>
+
+            <div id="maintenance-tab" class="tab-content is-hidden">
+                <h2 class="title is-4">
+                    <span class="icon is-small"><img data-icon="gear" data-size="16" src="../resources/img/gear-light-16.png" alt=""></span>
+                    <span>Maintenance</span>
+                </h2>
+                <table class="table is-fullwidth">
+                    <tbody>
+                        <tr><th>Rule count</th><td id="rule-count"></td></tr>
+                        <tr><th>Cache entries</th><td id="cache-count"></td></tr>
+                        <tr><th>Queue items</th><td id="queue-count"></td></tr>
+                        <tr><th>Current run time</th><td id="current-time">--:--:--</td></tr>
+                        <tr><th>Last run time</th><td id="last-time">--:--:--</td></tr>
+                        <tr><th>Average run time</th><td id="average-time">--:--:--</td></tr>
+                        <tr><th>Total run time</th><td id="total-time">--:--:--</td></tr>
+                        <tr><th>Messages per hour</th><td id="per-hour">0</td></tr>
+                        <tr><th>Messages per day</th><td id="per-day">0</td></tr>
+                    </tbody>
+                </table>
+                <button class="button is-danger" id="clear-cache" type="button">
+                    <span class="icon is-small"><img data-icon="trash" data-size="16" src="../resources/img/trash-light-16.png" alt=""></span>
+                    <span>Clear Cache</span>
+                </button>
+                <div class="field mt-4">
+                    <label class="label">Data categories</label>
+                    <div class="control">
+                        <label class="checkbox mr-3"><input class="transfer-category" type="checkbox" value="settings" checked> Settings</label>
+                        <label class="checkbox mr-3"><input class="transfer-category" type="checkbox" value="rules" checked> Rules</label>
+                        <label class="checkbox"><input class="transfer-category" type="checkbox" value="cache" checked> Cache</label>
+                    </div>
+                </div>
+                <div class="field is-grouped mt-4">
+                    <p class="control">
+                        <button class="button" id="export-data" type="button">
+                            <span class="icon is-small"><img data-icon="download" data-size="16" src="../resources/img/download-light-16.png" alt=""></span>
+                            <span>Export Data</span>
+                        </button>
+                    </p>
+                    <p class="control">
+                        <button class="button" id="import-data" type="button">
+                            <span class="icon is-small"><img data-icon="upload" data-size="16" src="../resources/img/upload-light-16.png" alt=""></span>
+                            <span>Import Data</span>
+                        </button>
+                        <input class="is-hidden" type="file" id="import-file" accept="application/json">
+                    </p>
+                </div>
+            </div>
+
+            <div id="debug-tab" class="tab-content is-hidden">
+                <h2 class="title is-4">
+                    <span class="icon is-small"><img data-icon="average" data-size="16" src="../resources/img/average-light-16.png" alt=""></span>
+                    <span>Debug</span>
+                </h2>
+                <pre id="payload-display"></pre>
+                <div id="diff-container" class="mt-4 is-hidden">
+                    <label class="label">Prompt diff</label>
+                    <div id="diff-display" class="box content is-family-monospace"></div>
+                </div>
+            </div>
+        </div>
+    </section>
+    <script src="../resources/js/diff_match_patch_uncompressed.js"></script>
     <script src="options.js"></script>
 </body>
 </html>

options/options.js

@@ -1,28 +1,105 @@
 document.addEventListener('DOMContentLoaded', async () => {
+    const storage = (globalThis.messenger ?? browser).storage;
     const logger = await import(browser.runtime.getURL('logger.js'));
-    const defaults = await browser.storage.local.get([
+    const AiClassifier = await import(browser.runtime.getURL('modules/AiClassifier.js'));
+    const dataTransfer = await import(browser.runtime.getURL('options/dataTransfer.js'));
+    const { detectSystemTheme } = await import(browser.runtime.getURL('modules/themeUtils.js'));
+    const { DEFAULT_AI_PARAMS } = await import(browser.runtime.getURL('modules/defaultParams.js'));
+    const defaults = await storage.local.get([
         'endpoint',
         'templateName',
         'customTemplate',
         'customSystemPrompt',
         'aiParams',
-        'debugLogging'
+        'debugLogging',
+        'htmlToMarkdown',
+        'stripUrlParams',
+        'altTextImages',
+        'collapseWhitespace',
+        'tokenReduction',
+        'aiRules',
+        'aiCache',
+        'theme',
+        'showDebugTab',
+        'lastPayload',
+        'lastFullText',
+        'lastPromptText'
     ]);
+    const tabButtons = document.querySelectorAll('#main-tabs li');
+    const tabs = document.querySelectorAll('.tab-content');
+    tabButtons.forEach(btn => btn.addEventListener('click', () => {
+        tabButtons.forEach(b => b.classList.remove('is-active'));
+        btn.classList.add('is-active');
+        tabs.forEach(tab => {
+            tab.classList.toggle('is-hidden', tab.id !== `${btn.dataset.tab}-tab`);
+        });
+    }));
+    tabButtons[0]?.click();
+
+    const saveBtn = document.getElementById('save');
+    let initialized = false;
+    let dragRule = null;
+    function markDirty() {
+        if (initialized) saveBtn.disabled = false;
+    }
+    document.addEventListener('input', markDirty, true);
+    document.addEventListener('change', markDirty, true);
     logger.setDebug(defaults.debugLogging === true);
-    const DEFAULT_AI_PARAMS = {
-        max_tokens: 4096,
-        temperature: 0.6,
-        top_p: 0.95,
-        seed: -1,
-        repetition_penalty: 1.0,
-        top_k: 20,
-        min_p: 0,
-        presence_penalty: 0,
-        frequency_penalty: 0,
-        typical_p: 1,
-        tfs: 1
-    };
-    document.getElementById('endpoint').value = defaults.endpoint || 'http://127.0.0.1:5000/v1/classify';
+
+    const themeSelect = document.getElementById('theme-select');
+    themeSelect.value = defaults.theme || 'auto';
+
+    function updateIcons(theme) {
+        document.querySelectorAll('img[data-icon]').forEach(img => {
+            const name = img.dataset.icon;
+            const size = img.dataset.size || 16;
+            if (name === 'full-logo') {
+                img.src = `../resources/img/full-logo${theme === 'dark' ? '-white' : ''}.png`;
+            } else {
+                img.src = `../resources/img/${name}-${theme}-${size}.png`;
+            }
+        });
+    }
+
+    async function applyTheme(setting) {
+        const mode = setting === 'auto' ? await detectSystemTheme() : setting;
+        document.documentElement.dataset.theme = mode;
+        updateIcons(mode);
+    }
+
+    await applyTheme(themeSelect.value);
+    const payloadDisplay = document.getElementById('payload-display');
+    const diffDisplay = document.getElementById('diff-display');
+    const diffContainer = document.getElementById('diff-container');
+
+    let lastFullText = defaults.lastFullText || '';
+    let lastPromptText = defaults.lastPromptText || '';
+    let lastPayload = defaults.lastPayload ? JSON.stringify(defaults.lastPayload, null, 2) : '';
+
+    if (lastPayload) {
+        payloadDisplay.textContent = lastPayload;
+    }
+    if (lastFullText && lastPromptText && diff_match_patch) {
+        const dmp = new diff_match_patch();
+        dmp.Diff_EditCost = 4;
+        const diffs = dmp.diff_main(lastFullText, lastPromptText);
+        dmp.diff_cleanupEfficiency(diffs);
+        const hasDiff = diffs.some(d => d[0] !== 0);
+        if (hasDiff) {
+            diffDisplay.innerHTML = dmp.diff_prettyHtml(diffs);
+            diffContainer.classList.remove('is-hidden');
+        } else {
+            diffDisplay.innerHTML = '';
+            diffContainer.classList.add('is-hidden');
+        }
+    } else {
+        diffContainer.classList.add('is-hidden');
+    }
+    themeSelect.addEventListener('change', async () => {
+        markDirty();
+        await applyTheme(themeSelect.value);
+    });
+    document.getElementById('endpoint').value = defaults.endpoint || 'http://127.0.0.1:5000/v1/completions';
 
     const templates = {
         openai: browser.i18n.getMessage('template.openai'),
@@ -44,7 +121,7 @@ document.addEventListener('DOMContentLoaded', async () => {
     customTemplate.value = defaults.customTemplate || '';
 
     function updateVisibility() {
-        customBox.style.display = templateSelect.value === 'custom' ? 'block' : 'none';
+        customBox.classList.toggle('is-hidden', templateSelect.value !== 'custom');
     }
     templateSelect.addEventListener('change', updateVisibility);
     updateVisibility();
@@ -52,18 +129,66 @@ document.addEventListener('DOMContentLoaded', async () => {
     const advancedBox = document.getElementById('advanced-options');
     const advancedBtn = document.getElementById('toggle-advanced');
     advancedBtn.addEventListener('click', () => {
-        advancedBox.style.display = advancedBox.style.display === 'none' ? 'block' : 'none';
+        advancedBox.classList.toggle('is-hidden');
     });
 
     const debugToggle = document.getElementById('debug-logging');
     debugToggle.checked = defaults.debugLogging === true;
 
+    const htmlToggle = document.getElementById('html-to-markdown');
+    htmlToggle.checked = defaults.htmlToMarkdown === true;
+
+    const stripUrlToggle = document.getElementById('strip-url-params');
+    stripUrlToggle.checked = defaults.stripUrlParams === true;
+
+    const altTextToggle = document.getElementById('alt-text-images');
+    altTextToggle.checked = defaults.altTextImages === true;
+
+    const collapseWhitespaceToggle = document.getElementById('collapse-whitespace');
+    collapseWhitespaceToggle.checked = defaults.collapseWhitespace === true;
+
+    const tokenReductionToggle = document.getElementById('token-reduction');
+    tokenReductionToggle.checked = defaults.tokenReduction === true;
+
+    const debugTabToggle = document.getElementById('show-debug-tab');
+    const debugTabBtn = document.getElementById('debug-tab-button');
+    function updateDebugTab() {
+        const visible = debugTabToggle.checked;
+        debugTabBtn.classList.toggle('is-hidden', !visible);
+    }
+    debugTabToggle.checked = defaults.showDebugTab === true;
+    debugTabToggle.addEventListener('change', () => { updateDebugTab(); markDirty(); });
+    updateDebugTab();
+
+
     const aiParams = Object.assign({}, DEFAULT_AI_PARAMS, defaults.aiParams || {});
     for (const [key, val] of Object.entries(aiParams)) {
         const el = document.getElementById(key);
         if (el) el.value = val;
     }
 
+    let tagList = [];
+    let folderList = [];
+    let accountList = [];
+    try {
+        tagList = await messenger.messages.tags.list();
+    } catch (e) {
+        logger.aiLog('failed to list tags', {level:'error'}, e);
+    }
+    try {
+        const accounts = await messenger.accounts.list(true);
+        accountList = accounts.map(a => ({ id: a.id, name: a.name }));
+        const collect = (f, prefix='') => {
+            folderList.push({ id: f.id ?? f.path, name: prefix + f.name });
+            (f.subFolders || []).forEach(sf => collect(sf, prefix + f.name + '/'));
+        };
+        for (const acct of accounts) {
+            (acct.folders || []).forEach(f => collect(f, `${acct.name}/`));
+        }
+    } catch (e) {
+        logger.aiLog('failed to list folders', {level:'error'}, e);
+    }
+
     const DEFAULT_SYSTEM = 'Determine whether the email satisfies the user\'s criterion.';
     const systemBox = document.getElementById('system-instructions');
     systemBox.value = defaults.customSystemPrompt || DEFAULT_SYSTEM;
@@ -71,6 +196,615 @@ document.addEventListener('DOMContentLoaded', async () => {
         systemBox.value = DEFAULT_SYSTEM;
     });
 
+    const rulesContainer = document.getElementById('rules-container');
+    const addRuleBtn = document.getElementById('add-rule');
+
+    const ruleCountEl = document.getElementById('rule-count');
+    const cacheCountEl = document.getElementById('cache-count');
+    const queueCountEl = document.getElementById('queue-count');
+    const currentTimeEl = document.getElementById('current-time');
+    const lastTimeEl = document.getElementById('last-time');
+    const averageTimeEl = document.getElementById('average-time');
+    const totalTimeEl = document.getElementById('total-time');
+    const perHourEl = document.getElementById('per-hour');
+    const perDayEl = document.getElementById('per-day');
+    let timingLogged = false;
+    ruleCountEl.textContent = (defaults.aiRules || []).length;
+    cacheCountEl.textContent = defaults.aiCache ? Object.keys(defaults.aiCache).length : 0;
+
+    function createActionRow(action = {type: 'tag'}) {
+        const row = document.createElement('div');
+        row.className = 'action-row field is-grouped mb-2';
+
+        const typeWrapper = document.createElement('div');
+        typeWrapper.className = 'select is-small mr-2';
+        const typeSelect = document.createElement('select');
+        ['tag','move','copy','junk','read','flag','delete','archive','forward','reply'].forEach(t => {
+            const opt = document.createElement('option');
+            opt.value = t;
+            opt.textContent = t;
+            typeSelect.appendChild(opt);
+        });
+        typeSelect.value = action.type;
+        typeWrapper.appendChild(typeSelect);
+
+        const paramSpan = document.createElement('span');
+
+        function updateParams() {
+            paramSpan.innerHTML = '';
+            if (typeSelect.value === 'tag') {
+                const wrap = document.createElement('div');
+                wrap.className = 'select is-small';
+                const sel = document.createElement('select');
+                sel.className = 'tag-select';
+                for (const t of tagList) {
+                    const opt = document.createElement('option');
+                    opt.value = t.key;
+                    opt.textContent = t.tag;
+                    sel.appendChild(opt);
+                }
+                sel.value = action.tagKey || '';
+                wrap.appendChild(sel);
+                paramSpan.appendChild(wrap);
+            } else if (typeSelect.value === 'move' || typeSelect.value === 'copy') {
+                const wrap = document.createElement('div');
+                wrap.className = 'select is-small';
+                const sel = document.createElement('select');
+                sel.className = 'folder-select';
+                for (const f of folderList) {
+                    const opt = document.createElement('option');
+                    opt.value = f.id;
+                    opt.textContent = f.name;
+                    sel.appendChild(opt);
+                }
+                sel.value = action.folder || action.copyTarget || '';
+                wrap.appendChild(sel);
+                paramSpan.appendChild(wrap);
+            } else if (typeSelect.value === 'junk') {
+                const wrap = document.createElement('div');
+                wrap.className = 'select is-small';
+                const sel = document.createElement('select');
+                sel.className = 'junk-select';
+                sel.appendChild(new Option('mark junk','true'));
+                sel.appendChild(new Option('mark not junk','false'));
+                sel.value = String(action.junk ?? true);
+                wrap.appendChild(sel);
+                paramSpan.appendChild(wrap);
+            } else if (typeSelect.value === 'read') {
+                const wrap = document.createElement('div');
+                wrap.className = 'select is-small';
+                const sel = document.createElement('select');
+                sel.className = 'read-select';
+                sel.appendChild(new Option('mark read','true'));
+                sel.appendChild(new Option('mark unread','false'));
+                sel.value = String(action.read ?? true);
+                wrap.appendChild(sel);
+                paramSpan.appendChild(wrap);
+            } else if (typeSelect.value === 'flag') {
+                const wrap = document.createElement('div');
+                wrap.className = 'select is-small';
+                const sel = document.createElement('select');
+                sel.className = 'flag-select';
+                sel.appendChild(new Option('flag','true'));
+                sel.appendChild(new Option('unflag','false'));
+                sel.value = String(action.flagged ?? true);
+                wrap.appendChild(sel);
+                paramSpan.appendChild(wrap);
+            } else if (typeSelect.value === 'forward') {
+                const input = document.createElement('input');
+                input.type = 'text';
+                input.className = 'input is-small forward-input';
+                input.placeholder = 'address@example.com';
+                input.value = action.address || '';
+                paramSpan.appendChild(input);
+            } else if (typeSelect.value === 'reply') {
+                const wrap = document.createElement('div');
+                wrap.className = 'select is-small';
+                const sel = document.createElement('select');
+                sel.className = 'reply-select';
+                sel.appendChild(new Option('all','all'));
+                sel.appendChild(new Option('sender','sender'));
+                sel.value = action.replyType || 'all';
+                wrap.appendChild(sel);
+                paramSpan.appendChild(wrap);
+            } else if (typeSelect.value === 'delete' || typeSelect.value === 'archive') {
+                paramSpan.appendChild(document.createElement('span'));
+            }
+        }
+
+        typeSelect.addEventListener('change', updateParams);
+        updateParams();
+
+        const removeBtn = document.createElement('button');
+        removeBtn.textContent = 'Remove';
+        removeBtn.type = 'button';
+        removeBtn.className = 'button is-small is-danger is-light';
+        removeBtn.addEventListener('click', () => row.remove());
+
+        row.appendChild(typeWrapper);
+        row.appendChild(paramSpan);
+        row.appendChild(removeBtn);
+
+        return row;
+    }
+
+    function createConditionButton(label, sectionEl, checkbox, clearFn) {
+        const btn = document.createElement('button');
+        btn.type = 'button';
+        btn.className = 'button is-small is-light';
+        const icon = document.createElement('img');
+        icon.width = 16;
+        icon.height = 16;
+        icon.className = 'mr-1';
+        btn.appendChild(icon);
+        btn.append(label);
+
+        let active = checkbox ? checkbox.checked : sectionEl && !sectionEl.classList.contains('is-hidden');
+
+        function update() {
+            btn.classList.toggle('is-active', active);
+            icon.src = browser.runtime.getURL(`resources/svg/${active ? 'circledot' : 'circle'}.svg`);
+            if (sectionEl) sectionEl.classList.toggle('is-hidden', !active);
+            if (checkbox) checkbox.checked = active;
+            if (!active && typeof clearFn === 'function') {
+                clearFn();
+            }
+        }
+
+        btn.addEventListener('click', () => {
+            active = !active;
+            markDirty();
+            update();
+        });
+
+        update();
+        return btn;
+    }
+
+    function renderRules(rules = []) {
+        ruleCountEl.textContent = rules.length;
+        rulesContainer.innerHTML = '';
+        for (const rule of rules) {
+            const article = document.createElement('article');
+            article.className = 'rule message mb-4';
+            article.draggable = true;
+            article.addEventListener('dragstart', ev => { dragRule = article; ev.dataTransfer.setData('text/plain', ''); });
+            article.addEventListener('dragover', ev => ev.preventDefault());
+            article.addEventListener('drop', ev => {
+                ev.preventDefault();
+                if (dragRule && dragRule !== article) {
+                    const children = Array.from(rulesContainer.children);
+                    const dragIndex = children.indexOf(dragRule);
+                    const dropIndex = children.indexOf(article);
+                    if (dragIndex < dropIndex) {
+                        rulesContainer.insertBefore(dragRule, article.nextSibling);
+                    } else {
+                        rulesContainer.insertBefore(dragRule, article);
+                    }
+                    markDirty();
+                }
+            });
+
+            const critInput = document.createElement('input');
+            critInput.type = 'text';
+            critInput.placeholder = 'Criterion';
+            critInput.value = rule.criterion || '';
+            critInput.className = 'input criterion mr-2';
+            critInput.style.flexGrow = '1';
+
+            const header = document.createElement('div');
+            header.className = 'message-header';
+
+            const leftWrap = document.createElement('div');
+            leftWrap.style.display = 'flex';
+            leftWrap.style.alignItems = 'center';
+            leftWrap.style.flexGrow = '1';
+
+            const statusSpan = document.createElement('span');
+            statusSpan.className = 'rule-status has-text-weight-semibold mr-2';
+
+            leftWrap.appendChild(statusSpan);
+            leftWrap.appendChild(critInput);
+            header.appendChild(leftWrap);
+
+            const btnWrap = document.createElement('div');
+            btnWrap.style.display = 'flex';
+            btnWrap.style.gap = '0.25em';
+
+            let enabled = rule.enabled !== false;
+
+            const toggleBtn = document.createElement('button');
+            toggleBtn.type = 'button';
+            toggleBtn.className = 'button is-small is-light rule-toggle';
+            const toggleIcon = document.createElement('img');
+            toggleIcon.width = 16;
+            toggleIcon.height = 16;
+            toggleBtn.appendChild(toggleIcon);
+
+            const delBtn = document.createElement('button');
+            delBtn.type = 'button';
+            delBtn.className = 'button is-small is-danger is-light rule-delete';
+            const delIcon = document.createElement('img');
+            delIcon.src = browser.runtime.getURL('resources/svg/trash.svg');
+            delIcon.width = 16;
+            delIcon.height = 16;
+            delBtn.appendChild(delIcon);
+
+            function updateToggle() {
+                toggleIcon.src = browser.runtime.getURL(
+                    `resources/svg/${enabled ? 'circleslash' : 'check'}.svg`
+                );
+                statusSpan.textContent = enabled ? '' : '(Disabled)';
+                article.dataset.enabled = String(enabled);
+            }
+
+            toggleBtn.addEventListener('click', () => {
+                enabled = !enabled;
+                markDirty();
+                updateToggle();
+            });
+
+            delBtn.addEventListener('click', () => {
+                article.remove();
+                ruleCountEl.textContent = rulesContainer.querySelectorAll('.rule').length;
+                markDirty();
+            });
+
+            btnWrap.appendChild(toggleBtn);
+            btnWrap.appendChild(delBtn);
+            header.appendChild(btnWrap);
+
+            updateToggle();
+
+            const actionsContainer = document.createElement('div');
+            actionsContainer.className = 'rule-actions mb-2';
+
+            for (const act of (rule.actions || [])) {
+                actionsContainer.appendChild(createActionRow(act));
+            }
+
+            const addAction = document.createElement('button');
+            addAction.textContent = 'Add Action';
+            addAction.type = 'button';
+            addAction.className = 'button is-small mb-2';
+            addAction.addEventListener('click', () => actionsContainer.appendChild(createActionRow()));
+
+           const stopLabel = document.createElement('label');
+           stopLabel.className = 'checkbox mt-2 is-hidden';
+           const stopCheck = document.createElement('input');
+           stopCheck.type = 'checkbox';
+           stopCheck.className = 'stop-processing';
+           stopCheck.checked = rule.stopProcessing === true;
+           stopLabel.appendChild(stopCheck);
+           stopLabel.append(' Stop after match');
+
+            const unreadLabel = document.createElement('label');
+            unreadLabel.className = 'checkbox mt-2 ml-4 is-hidden';
+            const unreadCheck = document.createElement('input');
+            unreadCheck.type = 'checkbox';
+            unreadCheck.className = 'unread-only';
+            unreadCheck.checked = rule.unreadOnly === true;
+            unreadLabel.appendChild(unreadCheck);
+            unreadLabel.append(' Only apply to unread messages');
+
+            const ageBox = document.createElement('div');
+            ageBox.className = 'field is-grouped mt-2 is-hidden';
+            const minInput = document.createElement('input');
+            minInput.type = 'number';
+            minInput.placeholder = 'Min days';
+            minInput.className = 'input is-small min-age mr-2';
+            minInput.style.width = '6em';
+            if (typeof rule.minAgeDays === 'number') minInput.value = rule.minAgeDays;
+            const maxInput = document.createElement('input');
+            maxInput.type = 'number';
+            maxInput.placeholder = 'Max days';
+            maxInput.className = 'input is-small max-age';
+            maxInput.style.width = '6em';
+            if (typeof rule.maxAgeDays === 'number') maxInput.value = rule.maxAgeDays;
+            ageBox.appendChild(minInput);
+            ageBox.appendChild(maxInput);
+
+            const acctBox = document.createElement('div');
+            acctBox.className = 'field mt-2 is-hidden';
+            const acctLabel = document.createElement('label');
+            acctLabel.className = 'label';
+            acctLabel.textContent = 'Accounts';
+            const acctControl = document.createElement('div');
+            const acctWrap = document.createElement('div');
+            acctWrap.className = 'select is-multiple is-small';
+            const acctSel = document.createElement('select');
+            acctSel.className = 'account-select';
+            acctSel.multiple = true;
+            acctSel.size = Math.min(accountList.length, 4) || 1;
+            for (const a of accountList) {
+                const opt = document.createElement('option');
+                opt.value = a.id;
+                opt.textContent = a.name;
+                if ((rule.accounts || []).includes(a.id)) opt.selected = true;
+                acctSel.appendChild(opt);
+            }
+            acctWrap.appendChild(acctSel);
+            acctControl.appendChild(acctWrap);
+            acctBox.appendChild(acctLabel);
+            acctBox.appendChild(acctControl);
+
+            const folderBox = document.createElement('div');
+            folderBox.className = 'field mt-2 is-hidden';
+            const folderLabel = document.createElement('label');
+            folderLabel.className = 'label';
+            folderLabel.textContent = 'Folders';
+            const folderControl = document.createElement('div');
+            const folderWrap = document.createElement('div');
+            folderWrap.className = 'select is-multiple is-small';
+            const folderSel = document.createElement('select');
+            folderSel.className = 'folder-filter-select';
+            folderSel.multiple = true;
+            folderSel.size = Math.min(folderList.length, 6) || 1;
+            for (const f of folderList) {
+                const opt = document.createElement('option');
+                opt.value = f.id;
+                opt.textContent = f.name;
+                if ((rule.folders || []).includes(f.id)) opt.selected = true;
+                folderSel.appendChild(opt);
+            }
+            folderWrap.appendChild(folderSel);
+            folderControl.appendChild(folderWrap);
+            folderBox.appendChild(folderLabel);
+            folderBox.appendChild(folderControl);
+
+            if (typeof rule.minAgeDays === 'number' || typeof rule.maxAgeDays === 'number') {
+                ageBox.classList.remove('is-hidden');
+            }
+            if ((rule.accounts || []).length) {
+                acctBox.classList.remove('is-hidden');
+            }
+            if ((rule.folders || []).length) {
+                folderBox.classList.remove('is-hidden');
+            }
+
+            const condButtons = document.createElement('div');
+            condButtons.className = 'field is-grouped is-grouped-multiline mb-2';
+
+            function addCond(btn) {
+                const p = document.createElement('p');
+                p.className = 'control';
+                p.appendChild(btn);
+                condButtons.appendChild(p);
+            }
+
+            addCond(createConditionButton('Stop', null, stopCheck, () => {
+                stopCheck.checked = false;
+            }));
+            addCond(createConditionButton('Unread', null, unreadCheck, () => {
+                unreadCheck.checked = false;
+            }));
+            addCond(createConditionButton('Age', ageBox, null, () => {
+                minInput.value = '';
+                maxInput.value = '';
+            }));
+            addCond(createConditionButton('Accounts', acctBox, null, () => {
+                for (const opt of acctSel.options) opt.selected = false;
+            }));
+            addCond(createConditionButton('Folders', folderBox, null, () => {
+                for (const opt of folderSel.options) opt.selected = false;
+            }));
+
+            const body = document.createElement('div');
+            body.className = 'message-body';
+            body.appendChild(actionsContainer);
+            body.appendChild(addAction);
+            const condDivider = document.createElement('hr');
+            condDivider.className = 'mt-3 mb-2';
+            body.appendChild(condDivider);
+            body.appendChild(condButtons);
+            body.appendChild(stopLabel);
+            body.appendChild(unreadLabel);
+            body.appendChild(ageBox);
+            body.appendChild(acctBox);
+            body.appendChild(folderBox);
+
+            article.appendChild(header);
+            article.appendChild(body);
+
+            rulesContainer.appendChild(article);
+        }
+    }
+
+    addRuleBtn.addEventListener('click', () => {
+        const data = [...rulesContainer.querySelectorAll('.rule')].map(ruleEl => {
+            const criterion = ruleEl.querySelector('.criterion').value;
+            const actions = [...ruleEl.querySelectorAll('.action-row')].map(row => {
+                const type = row.querySelector('select').value;
+                if (type === 'tag') {
+                    return { type, tagKey: row.querySelector('.tag-select').value };
+                }
+                if (type === 'move') {
+                    return { type, folder: row.querySelector('.folder-select').value };
+                }
+                if (type === 'copy') {
+                    return { type, copyTarget: row.querySelector('.folder-select').value };
+                }
+                if (type === 'junk') {
+                    return { type, junk: row.querySelector('.junk-select').value === 'true' };
+                }
+                if (type === 'read') {
+                    return { type, read: row.querySelector('.read-select').value === 'true' };
+                }
+                if (type === 'flag') {
+                    return { type, flagged: row.querySelector('.flag-select').value === 'true' };
+                }
+                if (type === 'delete' || type === 'archive') {
+                    return { type };
+                }
+                return { type };
+            });
+            const stopProcessing = ruleEl.querySelector('.stop-processing')?.checked;
+            const unreadOnly = ruleEl.querySelector('.unread-only')?.checked;
+            const enabled = ruleEl.dataset.enabled !== 'false';
+            const minAgeDays = parseFloat(ruleEl.querySelector('.min-age')?.value);
+            const maxAgeDays = parseFloat(ruleEl.querySelector('.max-age')?.value);
+            const accounts = [...(ruleEl.querySelector('.account-select')?.selectedOptions || [])].map(o => o.value);
+            const folders = [...(ruleEl.querySelector('.folder-filter-select')?.selectedOptions || [])].map(o => o.value);
+            const rule = { criterion, actions, enabled };
+            if (unreadOnly) rule.unreadOnly = true;
+            if (stopProcessing) rule.stopProcessing = true;
+            if (!isNaN(minAgeDays)) rule.minAgeDays = minAgeDays;
+            if (!isNaN(maxAgeDays)) rule.maxAgeDays = maxAgeDays;
+            if (accounts.length) rule.accounts = accounts;
+            if (folders.length) rule.folders = folders;
+            return rule;
+        });
+        data.push({ criterion: '', actions: [], unreadOnly: false, stopProcessing: false, enabled: true, accounts: [], folders: [] });
+        renderRules(data);
+    });
+
+    renderRules((defaults.aiRules || []).map(r => {
+        if (r.actions) {
+            if (!Array.isArray(r.accounts)) r.accounts = [];
+            if (!Array.isArray(r.folders)) r.folders = [];
+            if (r.enabled !== false) r.enabled = true; else r.enabled = false;
+            return r;
+        }
+        const actions = [];
+        if (r.tag) actions.push({ type: 'tag', tagKey: r.tag });
+        if (r.moveTo) actions.push({ type: 'move', folder: r.moveTo });
+        if (r.copyTarget || r.copyTo) actions.push({ type: 'copy', copyTarget: r.copyTarget || r.copyTo });
+        const rule = { criterion: r.criterion, actions };
+        if (r.stopProcessing) rule.stopProcessing = true;
+        if (r.unreadOnly) rule.unreadOnly = true;
+        if (typeof r.minAgeDays === 'number') rule.minAgeDays = r.minAgeDays;
+        if (typeof r.maxAgeDays === 'number') rule.maxAgeDays = r.maxAgeDays;
+        if (Array.isArray(r.accounts)) rule.accounts = r.accounts;
+        if (Array.isArray(r.folders)) rule.folders = r.folders;
+        rule.enabled = r.enabled !== false;
+        return rule;
+    }));
+
+
+    function format(ms) {
+        if (ms < 0) return '--:--:--';
+        let totalSec = Math.floor(ms / 1000);
+        const sec = totalSec % 60;
+        totalSec = (totalSec - sec) / 60;
+        const min = totalSec % 60;
+        const hr = (totalSec - min) / 60;
+        return `${String(hr).padStart(2, '0')}:${String(min).padStart(2, '0')}:${String(sec).padStart(2, '0')}`;
+    }
+
+    async function refreshMaintenance() {
+        try {
+            const stats = await browser.runtime.sendMessage({ type: 'sortana:getTiming' });
+            queueCountEl.textContent = stats.count;
+            currentTimeEl.classList.remove('has-text-danger');
+            lastTimeEl.classList.remove('has-text-success','has-text-danger');
+            let arrow = '';
+            if (stats.last >= 0) {
+                if (stats.stddev > 0 && stats.last - stats.average > stats.stddev) {
+                    lastTimeEl.classList.add('has-text-danger');
+                    arrow = ' ▲';
+                } else if (stats.stddev > 0 && stats.average - stats.last > stats.stddev) {
+                    lastTimeEl.classList.add('has-text-success');
+                    arrow = ' ▼';
+                }
+                lastTimeEl.textContent = format(stats.last) + arrow;
+            } else {
+                lastTimeEl.textContent = '--:--:--';
+            }
+            if (stats.current >= 0) {
+                if (stats.stddev > 0 && stats.current - stats.average > stats.stddev) {
+                    currentTimeEl.classList.add('has-text-danger');
+                }
+                currentTimeEl.textContent = format(stats.current);
+            } else {
+                currentTimeEl.textContent = '--:--:--';
+            }
+            averageTimeEl.textContent = stats.runs > 0 ? format(stats.average) : '--:--:--';
+            totalTimeEl.textContent = format(stats.total);
+            const perHour = stats.average > 0 ? Math.round(3600000 / stats.average) : 0;
+            const perDay = stats.average > 0 ? Math.round(86400000 / stats.average) : 0;
+            perHourEl.textContent = perHour;
+            perDayEl.textContent = perDay;
+            if (!timingLogged) {
+                logger.aiLog('retrieved timing stats', {debug: true});
+                timingLogged = true;
+            }
+        } catch (e) {
+            queueCountEl.textContent = '?';
+            currentTimeEl.textContent = '--:--:--';
+            lastTimeEl.textContent = '--:--:--';
+            averageTimeEl.textContent = '--:--:--';
+            totalTimeEl.textContent = '--:--:--';
+            perHourEl.textContent = '0';
+            perDayEl.textContent = '0';
+        }
+
+        try {
+            const { aiCache } = await storage.local.get('aiCache');
+            cacheCountEl.textContent = aiCache ? Object.keys(aiCache).length : 0;
+        } catch {
+            cacheCountEl.textContent = '?';
+        }
+
+        try {
+            if (debugTabToggle.checked) {
+                const latest = await storage.local.get(['lastPayload', 'lastFullText', 'lastPromptText']);
+                const payloadStr = latest.lastPayload ? JSON.stringify(latest.lastPayload, null, 2) : '';
+                if (payloadStr !== lastPayload) {
+                    lastPayload = payloadStr;
+                    payloadDisplay.textContent = payloadStr;
+                }
+                if (latest.lastFullText !== lastFullText || latest.lastPromptText !== lastPromptText) {
+                    lastFullText = latest.lastFullText || '';
+                    lastPromptText = latest.lastPromptText || '';
+                    if (lastFullText && lastPromptText && diff_match_patch) {
+                        const dmp = new diff_match_patch();
+                        dmp.Diff_EditCost = 4;
+                        const diffs = dmp.diff_main(lastFullText, lastPromptText);
+                        dmp.diff_cleanupEfficiency(diffs);
+                        const hasDiff = diffs.some(d => d[0] !== 0);
+                        if (hasDiff) {
+                            diffDisplay.innerHTML = dmp.diff_prettyHtml(diffs);
+                            diffContainer.classList.remove('is-hidden');
+                        } else {
+                            diffDisplay.innerHTML = '';
+                            diffContainer.classList.add('is-hidden');
+                        }
+                    } else {
+                        diffDisplay.innerHTML = '';
+                        diffContainer.classList.add('is-hidden');
+                    }
+                }
+            }
+        } catch {}
+    }
+
+    refreshMaintenance();
+    setInterval(refreshMaintenance, 1000);
+
+    document.getElementById('clear-cache').addEventListener('click', async () => {
+        await AiClassifier.clearCache();
+        cacheCountEl.textContent = '0';
+    });
+
+    function selectedCategories() {
+        return [...document.querySelectorAll('.transfer-category:checked')].map(el => el.value);
+    }
+
+    document.getElementById('export-data').addEventListener('click', () => {
+        dataTransfer.exportData(selectedCategories());
+    });
+
+    const importInput = document.getElementById('import-file');
+    document.getElementById('import-data').addEventListener('click', () => importInput.click());
+    importInput.addEventListener('change', async () => {
+        if (importInput.files.length) {
+            await dataTransfer.importData(importInput.files[0], selectedCategories());
+            location.reload();
+        }
+    });
+
+    initialized = true;
+
     document.getElementById('save').addEventListener('click', async () => {
         const endpoint = document.getElementById('endpoint').value;
         const templateName = templateSelect.value;
@@ -85,12 +819,67 @@ document.addEventListener('DOMContentLoaded', async () => {
             }
         }
         const debugLogging = debugToggle.checked;
-        await browser.storage.local.set({ endpoint, templateName, customTemplate: customTemplateText, customSystemPrompt, aiParams: aiParamsSave, debugLogging });
+        const htmlToMarkdown = htmlToggle.checked;
+        const rules = [...rulesContainer.querySelectorAll('.rule')].map(ruleEl => {
+            const criterion = ruleEl.querySelector('.criterion').value;
+            const actions = [...ruleEl.querySelectorAll('.action-row')].map(row => {
+                const type = row.querySelector('select').value;
+                if (type === 'tag') {
+                    return { type, tagKey: row.querySelector('.tag-select').value };
+                }
+                if (type === 'move') {
+                    return { type, folder: row.querySelector('.folder-select').value };
+                }
+                if (type === 'copy') {
+                    return { type, copyTarget: row.querySelector('.folder-select').value };
+                }
+                if (type === 'junk') {
+                    return { type, junk: row.querySelector('.junk-select').value === 'true' };
+                }
+                if (type === 'read') {
+                    return { type, read: row.querySelector('.read-select').value === 'true' };
+                }
+                if (type === 'flag') {
+                    return { type, flagged: row.querySelector('.flag-select').value === 'true' };
+                }
+                if (type === 'forward') {
+                    return { type, address: row.querySelector('.forward-input').value.trim() };
+                }
+                if (type === 'reply') {
+                    return { type, replyType: row.querySelector('.reply-select').value };
+                }
+                return { type };
+            });
+            const stopProcessing = ruleEl.querySelector('.stop-processing')?.checked;
+            const unreadOnly = ruleEl.querySelector('.unread-only')?.checked;
+            const enabled = ruleEl.dataset.enabled !== 'false';
+            const minAgeDays = parseFloat(ruleEl.querySelector('.min-age')?.value);
+            const maxAgeDays = parseFloat(ruleEl.querySelector('.max-age')?.value);
+            const accounts = [...(ruleEl.querySelector('.account-select')?.selectedOptions || [])].map(o => o.value);
+            const folders = [...(ruleEl.querySelector('.folder-filter-select')?.selectedOptions || [])].map(o => o.value);
+            const rule = { criterion, actions, enabled };
+            if (unreadOnly) rule.unreadOnly = true;
+            if (stopProcessing) rule.stopProcessing = true;
+            if (!isNaN(minAgeDays)) rule.minAgeDays = minAgeDays;
+            if (!isNaN(maxAgeDays)) rule.maxAgeDays = maxAgeDays;
+            if (accounts.length) rule.accounts = accounts;
+            if (folders.length) rule.folders = folders;
+            return rule;
+        }).filter(r => r.criterion);
+        const stripUrlParams = stripUrlToggle.checked;
+        const altTextImages = altTextToggle.checked;
+        const collapseWhitespace = collapseWhitespaceToggle.checked;
+        const tokenReduction = tokenReductionToggle.checked;
+        const showDebugTab = debugTabToggle.checked;
+        const theme = themeSelect.value;
+        await storage.local.set({ endpoint, templateName, customTemplate: customTemplateText, customSystemPrompt, aiParams: aiParamsSave, debugLogging, htmlToMarkdown, stripUrlParams, altTextImages, collapseWhitespace, tokenReduction, aiRules: rules, theme, showDebugTab });
+        await applyTheme(theme);
         try {
-            await browser.aiFilter.initConfig({ endpoint, templateName, customTemplate: customTemplateText, customSystemPrompt, aiParams: aiParamsSave, debugLogging });
+            await AiClassifier.setConfig({ endpoint, templateName, customTemplate: customTemplateText, customSystemPrompt, aiParams: aiParamsSave, debugLogging });
             logger.setDebug(debugLogging);
         } catch (e) {
             logger.aiLog('[options] failed to apply config', {level: 'error'}, e);
         }
+        saveBtn.disabled = true;
     });
 });