diff --git a/AGENTS.md b/AGENTS.md index ab79c66..401f962 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -5,8 +5,9 @@ This file provides guidelines for codex agents contributing to the Sortana proje ## Repository Overview - `background.js`: Handles startup tasks and coordinates message passing within the extension. -- `modules/`: Contains reusable JavaScript modules such as `AiClassifier.js`. -- `options/`: The options page HTML, JavaScript and Bulma CSS. +- `modules/`: Contains reusable JavaScript modules such as `AiClassifier.js`, + `defaultParams.js` and `themeUtils.js`. +- `options/`: The options page HTML, JavaScript and bundled Bulma CSS (v1.0.3). - `details.html` and `details.js`: View AI reasoning and clear cache for a message. - `resources/`: Images and other static files. - `prompt_templates/`: Prompt template files for the AI service. @@ -27,7 +28,7 @@ This file provides guidelines for codex agents contributing to the Sortana proje ## Testing -There are currently no automated tests for this project. If you add tests in the future, specify the commands to run them here. For now, verification must happen manually in Thunderbird. +There are currently no automated tests for this project. If you add tests in the future, specify the commands to run them here. For now, verification must happen manually in Thunderbird. Do **not** run the `ps1` build script or the SVG processing script. ## Documentation @@ -41,7 +42,7 @@ Additional documentation exists outside this repository. - Thunderbird Add-on Store Policies - [Third Party Library Usage](https://extensionworkshop.com/documentation/publish/third-party-library-usage/) - Third Party Libraries - - [Bulma.css](https://github.com/jgthms/bulma) + - [Bulma.css v1.0.3](https://github.com/jgthms/bulma/blob/1.0.3/css/bulma.css) - Issue tracker: [Thunderbird tracker on Bugzilla](https://bugzilla.mozilla.org/describecomponents.cgi?product=Thunderbird) @@ -66,3 +67,10 @@ text extracted from all text parts. properties. Any legacy `aiReasonCache` data is merged into `aiCache` the first time the add-on loads after an update. +### Icon Set Usage + +Toolbar and menu icons reside under `resources/img` and are provided in 16, 32 +and 64 pixel variants. When changing these icons, pass a dictionary mapping the +sizes to the paths in `browserAction.setIcon` or `messageDisplayAction.setIcon`. +Use `resources/svg2img.ps1` to regenerate PNGs from the SVG sources. + diff --git a/README.md b/README.md index 2baa5e8..57a0285 100644 --- a/README.md +++ b/README.md @@ -14,11 +14,17 @@ message meets a specified criterion. - **Custom system prompts** – tailor the instructions sent to the model for more precise results. - **Persistent result caching** – classification results and reasoning are saved to disk so messages aren't re-evaluated across restarts. - **Advanced parameters** – tune generation settings like temperature, top‑p and more from the options page. +- **Markdown conversion** – optionally convert HTML bodies to Markdown before sending them to the AI service. - **Debug logging** – optional colorized logs help troubleshoot interactions with the AI service. -- **Automatic rules** – create rules that tag or move new messages based on AI classification. +- **Debug tab** – view the last request payload and a diff between the unaltered message text and the final prompt. +- **Light/Dark themes** – automatically match Thunderbird's appearance with optional manual override. +- **Automatic rules** – create rules that tag, move, copy, forward, reply, delete, archive, mark read/unread or flag/unflag messages based on AI classification. Rules can optionally apply only to unread messages and can ignore messages outside a chosen age range. - **Rule ordering** – drag rules to prioritize them and optionally stop processing after a match. +- **Rule enable/disable** – temporarily turn a rule off without removing it. +- **Account & folder filters** – limit rules to specific accounts or folders. - **Context menu** – apply AI rules from the message list or the message display action button. -- **Status icons** – toolbar icons show when classification is in progress and briefly display success or error states. +- **Status icons** – toolbar icons show when classification is in progress and briefly display success states. If a failure occurs the icon turns red until you dismiss the notification. +- **Error notification** – failed classification displays a notification with a button to clear the error and reset the icon. - **View reasoning** – inspect why rules matched via the Details popup. - **Cache management** – clear cached results from the context menu or options page. - **Queue & timing stats** – monitor processing time on the Maintenance tab. @@ -57,21 +63,29 @@ Sortana is implemented entirely with standard WebExtension scripts—no custom e 1. Ensure PowerShell is available (for Windows) or adapt the script for other environments. -2. Ensure the Bulma stylesheet (v1.0.4) is saved as `options/bulma.css`. You can - download it from . +2. The Bulma stylesheet (v1.0.3) is already included as `options/bulma.css`. 3. Run `powershell ./build-xpi.ps1` from the repository root. The script reads the version from `manifest.json` and creates an XPI in the `release` folder. 4. Install the generated XPI in Thunderbird via the Add-ons Manager. During development you can also load the directory as a temporary add-on. +5. To regenerate PNG icons from the SVG sources, run `resources/svg2img.ps1`. ## Usage 1. Open the add-on's options and set the URL of your classification service. -2. Use the **Classification Rules** section to add a criterion and optional - actions such as tagging or moving a message when it matches. Drag rules to - reorder them and check *Stop after match* to halt further processing. + 2. Use the **Classification Rules** section to add a criterion and optional + actions such as tagging, moving, copying, forwarding, replying, + deleting or archiving a message when it matches. Drag rules to + reorder them, check *Only apply to unread messages* to skip read mail, + set optional minimum or maximum message age limits, select the accounts or + folders a rule should apply to. Use the + slashed-circle/check button to disable or re-enable a rule. The small + circle buttons for optional conditions show a filled dot when active, and + check *Stop after match* to halt further processing. Forward and reply actions + open a compose window using the account that received the message. 3. Save your settings. New mail will be evaluated automatically using the configured rules. +4. If the toolbar icon shows a red X, click the notification's **Dismiss** button to clear the error. ### Example Filters @@ -99,7 +113,7 @@ Here are some useful and fun example criteria you can use in your filters. Filte For when you're ready to filter based on vibes. You can define as many filters as you'd like, each using a different prompt and -triggering tags, moves, or actions based on the model's classification. +triggering tags, moves, copies, deletes, archives, read/unread changes or flag updates based on the model's classification. ## Required Permissions @@ -107,11 +121,14 @@ Sortana requests the following Thunderbird permissions: - `storage` – store configuration and cached classification results. - `messagesRead` – read message contents for classification. -- `messagesMove` – move messages when a rule specifies a target folder. -- `messagesUpdate` – change message properties such as tags and junk status. +- `messagesMove` – move or copy messages when a rule specifies a target folder. + - `messagesUpdate` – change message properties such as tags, junk status, read/unread state and flags. - `messagesTagsList` – retrieve existing message tags for rule actions. -- `accountsRead` – list accounts and folders for move actions. +- `accountsRead` – list accounts and folders for move or copy actions. - `menus` – add context menu commands. +- `tabs` – open new tabs and query the active tab. +- `notifications` – display error notifications. +- `compose` – create reply and forward compose windows for matching rules. ## Thunderbird Add-on Store Disclosures @@ -120,12 +137,17 @@ requires disclosure of third party libraries that are included in the add-on. Ev the disclosure is only required for add-on review, they'll be listed here as well. Sortana uses the following third party libraries: -- [Bulma.css v1.0.4](https://github.com/jgthms/bulma/blob/1.0.4/css/bulma.css) +- [Bulma.css v1.0.3](https://github.com/jgthms/bulma/blob/1.0.3/css/bulma.css) + - MIT License +- [turndown v7.2.0](https://github.com/mixmark-io/turndown/tree/v7.2.0) + - MIT License +- [diff](https://github.com/google/diff-match-patch/blob/62f2e689f498f9c92dbc588c58750addec9b1654/javascript/diff_match_patch_uncompressed.js) + - Apache-2.0 license ## License This project is licensed under the terms of the GNU General Public License -version 3. See `LICENSE` for the full text. +version 3. See `LICENSE` for the full text. Third party libraries are licensed seperately. ## Acknowledgments @@ -135,3 +157,5 @@ Sortana builds upon knowledge gained from open-source projects. In particular, how Thunderbird's WebExtension and experiment APIs can be extended. Their code provided invaluable guidance during development. +- Icons from [cc0-icons.jonh.eu](https://cc0-icons.jonh.eu/) are used under the CC0 license. + diff --git a/_locales/en-US/messages.json b/_locales/en-US/messages.json index 591373d..65181c8 100644 --- a/_locales/en-US/messages.json +++ b/_locales/en-US/messages.json @@ -14,5 +14,24 @@ "template.mistral": { "message": "Mistral" }, "template.custom": { "message": "Custom" }, "options.save": { "message": "Save" }, - "options.debugLogging": { "message": "Enable debug logging" } + "options.debugLogging": { "message": "Enable debug logging" }, + "options.htmlToMarkdown": { "message": "Convert HTML body to Markdown" }, + "options.stripUrlParams": { "message": "Remove URL tracking parameters" }, + "options.altTextImages": { "message": "Replace images with alt text" }, + "options.collapseWhitespace": { "message": "Collapse long whitespace" }, + "options.tokenReduction": { "message": "Aggressive token reduction" } + ,"action.read": { "message": "read" } + ,"action.flag": { "message": "flag" } + ,"action.copy": { "message": "copy" } + ,"action.delete": { "message": "delete" } + ,"action.archive": { "message": "archive" } + ,"action.forward": { "message": "forward" } + ,"action.reply": { "message": "reply" } + ,"param.markRead": { "message": "mark read" } + ,"param.markUnread": { "message": "mark unread" } + ,"param.flag": { "message": "flag" } + ,"param.unflag": { "message": "unflag" } + ,"param.address": { "message": "address" } + ,"param.replyAll": { "message": "reply all" } + ,"param.replySender": { "message": "reply sender" } } diff --git a/ai-filter.sln b/ai-filter.sln index 86ceaed..ea71f1b 100644 --- a/ai-filter.sln +++ b/ai-filter.sln @@ -47,14 +47,41 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "prompt_templates", "prompt_ EndProjectSection EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "resources", "resources", "{68A87938-5C2B-49F5-8AAA-8A34FBBFD854}" + ProjectSection(SolutionItems) = preProject + resources\svg2img.ps1 = resources\svg2img.ps1 + EndProjectSection EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "img", "img", "{F266602F-1755-4A95-A11B-6C90C701C5BF}" ProjectSection(SolutionItems) = preProject - resources\img\brain.png = resources\img\brain.png - resources\img\busy.png = resources\img\busy.png - resources\img\done.png = resources\img\done.png - resources\img\error.png = resources\img\error.png + resources\img\average-16.png = resources\img\average-16.png + resources\img\average-32.png = resources\img\average-32.png + resources\img\average-64.png = resources\img\average-64.png + resources\img\check-16.png = resources\img\check-16.png + resources\img\check-32.png = resources\img\check-32.png + resources\img\check-64.png = resources\img\check-64.png + resources\img\circle-16.png = resources\img\circle-16.png + resources\img\circle-32.png = resources\img\circle-32.png + resources\img\circle-64.png = resources\img\circle-64.png + resources\img\circledots-16.png = resources\img\circledots-16.png + resources\img\circledots-32.png = resources\img\circledots-32.png + resources\img\circledots-64.png = resources\img\circledots-64.png + resources\img\clipboarddata-16.png = resources\img\clipboarddata-16.png + resources\img\clipboarddata-32.png = resources\img\clipboarddata-32.png + resources\img\clipboarddata-64.png = resources\img\clipboarddata-64.png + resources\img\download-16.png = resources\img\download-16.png + resources\img\download-32.png = resources\img\download-32.png + resources\img\download-64.png = resources\img\download-64.png + resources\img\eye-16.png = resources\img\eye-16.png + resources\img\eye-32.png = resources\img\eye-32.png + resources\img\eye-64.png = resources\img\eye-64.png + resources\img\flag-16.png = resources\img\flag-16.png + resources\img\flag-32.png = resources\img\flag-32.png + resources\img\flag-64.png = resources\img\flag-64.png + resources\img\full-logo-white.png = resources\img\full-logo-white.png resources\img\full-logo.png = resources\img\full-logo.png + resources\img\gear-16.png = resources\img\gear-16.png + resources\img\gear-32.png = resources\img\gear-32.png + resources\img\gear-64.png = resources\img\gear-64.png resources\img\logo.png = resources\img\logo.png resources\img\logo128.png = resources\img\logo128.png resources\img\logo16.png = resources\img\logo16.png @@ -62,6 +89,45 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "img", "img", "{F266602F-175 resources\img\logo48.png = resources\img\logo48.png resources\img\logo64.png = resources\img\logo64.png resources\img\logo96.png = resources\img\logo96.png + resources\img\reply-16.png = resources\img\reply-16.png + resources\img\reply-32.png = resources\img\reply-32.png + resources\img\reply-64.png = resources\img\reply-64.png + resources\img\settings-16.png = resources\img\settings-16.png + resources\img\settings-32.png = resources\img\settings-32.png + resources\img\settings-64.png = resources\img\settings-64.png + resources\img\trash-16.png = resources\img\trash-16.png + resources\img\trash-32.png = resources\img\trash-32.png + resources\img\trash-64.png = resources\img\trash-64.png + resources\img\upload-16.png = resources\img\upload-16.png + resources\img\upload-32.png = resources\img\upload-32.png + resources\img\upload-64.png = resources\img\upload-64.png + resources\img\x-16.png = resources\img\x-16.png + resources\img\x-32.png = resources\img\x-32.png + resources\img\x-64.png = resources\img\x-64.png + EndProjectSection +EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "js", "js", "{21D2A42C-3F85-465C-9141-C106AFD92B68}" + ProjectSection(SolutionItems) = preProject + resources\js\diff_match_patch_uncompressed.js = resources\js\diff_match_patch_uncompressed.js + resources\js\turndown.js = resources\js\turndown.js + EndProjectSection +EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "svg", "svg", "{D4E9C905-4884-488E-B763-5BD39049C1B1}" + ProjectSection(SolutionItems) = preProject + resources\svg\average.svg = resources\svg\average.svg + resources\svg\check.svg = resources\svg\check.svg + resources\svg\circle.svg = resources\svg\circle.svg + resources\svg\circledots.svg = resources\svg\circledots.svg + resources\svg\clipboarddata.svg = resources\svg\clipboarddata.svg + resources\svg\download.svg = resources\svg\download.svg + resources\svg\eye.svg = resources\svg\eye.svg + resources\svg\flag.svg = resources\svg\flag.svg + resources\svg\gear.svg = resources\svg\gear.svg + resources\svg\reply.svg = resources\svg\reply.svg + resources\svg\settings.svg = resources\svg\settings.svg + resources\svg\trash.svg = resources\svg\trash.svg + resources\svg\upload.svg = resources\svg\upload.svg + resources\svg\x.svg = resources\svg\x.svg EndProjectSection EndProject Global @@ -76,5 +142,7 @@ Global {86516D53-50D4-4FE2-9D8A-977A8F5EBDBD} = {BCC6E6D2-343B-4C48-854D-5FE3BBC3CB70} {68A87938-5C2B-49F5-8AAA-8A34FBBFD854} = {BCC6E6D2-343B-4C48-854D-5FE3BBC3CB70} {F266602F-1755-4A95-A11B-6C90C701C5BF} = {68A87938-5C2B-49F5-8AAA-8A34FBBFD854} + {21D2A42C-3F85-465C-9141-C106AFD92B68} = {68A87938-5C2B-49F5-8AAA-8A34FBBFD854} + {D4E9C905-4884-488E-B763-5BD39049C1B1} = {68A87938-5C2B-49F5-8AAA-8A34FBBFD854} EndGlobalSection EndGlobal diff --git a/background.js b/background.js index d0425c2..fc585ff 100644 --- a/background.js +++ b/background.js @@ -22,6 +22,60 @@ let iconTimer = null; let timingStats = { count: 0, mean: 0, m2: 0, total: 0, last: -1 }; let currentStart = 0; let logGetTiming = true; +let htmlToMarkdown = false; +let stripUrlParams = false; +let altTextImages = false; +let collapseWhitespace = false; +let tokenReduction = false; +let maxTokens = 4096; +let TurndownService = null; +let userTheme = 'auto'; +let currentTheme = 'light'; +let detectSystemTheme; +let errorPending = false; +let showDebugTab = false; +const ERROR_NOTIFICATION_ID = 'sortana-error'; + +function normalizeRules(rules) { + return Array.isArray(rules) ? rules.map(r => { + if (r.actions) { + if (!Array.isArray(r.accounts)) r.accounts = []; + if (!Array.isArray(r.folders)) r.folders = []; + r.enabled = r.enabled !== false; + return r; + } + const actions = []; + if (r.tag) actions.push({ type: 'tag', tagKey: r.tag }); + if (r.moveTo) actions.push({ type: 'move', folder: r.moveTo }); + if (r.copyTarget || r.copyTo) actions.push({ type: 'copy', copyTarget: r.copyTarget || r.copyTo }); + const rule = { criterion: r.criterion, actions }; + if (r.stopProcessing) rule.stopProcessing = true; + if (r.unreadOnly) rule.unreadOnly = true; + if (typeof r.minAgeDays === 'number') rule.minAgeDays = r.minAgeDays; + if (typeof r.maxAgeDays === 'number') rule.maxAgeDays = r.maxAgeDays; + if (Array.isArray(r.accounts)) rule.accounts = r.accounts; + if (Array.isArray(r.folders)) rule.folders = r.folders; + rule.enabled = r.enabled !== false; + return rule; + }) : []; +} + +function iconPaths(name) { + return { + 16: `resources/img/${name}-${currentTheme}-16.png`, + 32: `resources/img/${name}-${currentTheme}-32.png`, + 64: `resources/img/${name}-${currentTheme}-64.png` + }; +} + + +const ICONS = { + logo: () => 'resources/img/logo.png', + circledots: () => iconPaths('circledots'), + circle: () => iconPaths('circle'), + average: () => iconPaths('average'), + error: () => iconPaths('x') +}; function setIcon(path) { if (browser.browserAction) { @@ -33,27 +87,67 @@ function setIcon(path) { } function updateActionIcon() { - let path = "resources/img/brain.png"; - if (processing || queuedCount > 0) { - path = "resources/img/busy.png"; + let path = ICONS.logo(); + if (errorPending) { + path = ICONS.error(); + } else if (processing || queuedCount > 0) { + path = ICONS.circledots(); } setIcon(path); } -function showTransientIcon(path, delay = 1500) { +function showTransientIcon(factory, delay = 1500) { + if (errorPending) { + return; + } clearTimeout(iconTimer); + const path = typeof factory === 'function' ? factory() : factory; setIcon(path); iconTimer = setTimeout(updateActionIcon, delay); } +async function clearError() { + errorPending = false; + await storage.local.set({ errorPending: false }); + await browser.notifications.clear(ERROR_NOTIFICATION_ID); + updateActionIcon(); +} + +function refreshMenuIcons() { + browser.menus.update('apply-ai-rules-list', { icons: iconPaths('eye') }); + browser.menus.update('apply-ai-rules-display', { icons: iconPaths('eye') }); + browser.menus.update('clear-ai-cache-list', { icons: iconPaths('trash') }); + browser.menus.update('clear-ai-cache-display', { icons: iconPaths('trash') }); + browser.menus.update('view-ai-reason-list', { icons: iconPaths('clipboarddata') }); + browser.menus.update('view-ai-reason-display', { icons: iconPaths('clipboarddata') }); +} + function byteSize(str) { return new TextEncoder().encode(str || "").length; } function replaceInlineBase64(text) { - return text.replace(/[A-Za-z0-9+/]{100,}={0,2}/g, - m => `[base64: ${byteSize(m)} bytes]`); + return text.replace(/(?:data:[^;]+;base64,)?[A-Za-z0-9+/=\r\n]{100,}/g, + m => tokenReduction ? '__BASE64__' : `[base64: ${byteSize(m)} bytes]`); +} + +function sanitizeString(text) { + let t = String(text); + if (tokenReduction) { + t = t.replace(//gs, '') + .replace(/url\([^\)]*\)/gi, 'url(__IMG__)'); + } + if (stripUrlParams) { + t = t.replace(/https?:\/\/[^\s)]+/g, m => { + const idx = m.indexOf('?'); + return idx >= 0 ? m.slice(0, idx) : m; + }); + } + if (collapseWhitespace) { + t = t.replace(/[\u200B-\u200D\u2060\s]{2,}/g, ' ').replace(/\n{3,}/g, '\n\n'); + } + return t; } function collectText(part, bodyParts, attachments) { @@ -70,21 +164,236 @@ function collectText(part, bodyParts, attachments) { attachments.push(`${name} (${ct}, ${part.size || byteSize(body)} bytes)`); } else if (ct.startsWith("text/html")) { const doc = new DOMParser().parseFromString(body, 'text/html'); - bodyParts.push(replaceInlineBase64(doc.body.textContent || "")); + if (tokenReduction) { + doc.querySelectorAll('script,style').forEach(el => el.remove()); + const walker = doc.createTreeWalker(doc, NodeFilter.SHOW_COMMENT); + let node; + while ((node = walker.nextNode())) { + node.parentNode.removeChild(node); + } + doc.querySelectorAll('*').forEach(el => { + for (const attr of Array.from(el.attributes)) { + if (!['href','src','alt'].includes(attr.name)) { + el.removeAttribute(attr.name); + } + } + }); + } + doc.querySelectorAll('img').forEach(img => { + const alt = img.getAttribute('alt') || ''; + const text = altTextImages ? alt : '__IMG__'; + img.replaceWith(doc.createTextNode(text)); + }); + if (stripUrlParams) { + doc.querySelectorAll('[href]').forEach(a => { + const href = a.getAttribute('href'); + if (href) a.setAttribute('href', href.split('?')[0]); + }); + doc.querySelectorAll('[src]').forEach(e => { + const src = e.getAttribute('src'); + if (src) e.setAttribute('src', src.split('?')[0]); + }); + } + if (htmlToMarkdown && TurndownService) { + try { + const td = new TurndownService(); + const md = sanitizeString(td.turndown(doc.body.innerHTML || body)); + bodyParts.push(replaceInlineBase64(`[HTML Body converted to Markdown]\n${md}`)); + } catch (e) { + bodyParts.push(replaceInlineBase64(sanitizeString(doc.body.textContent || ""))); + } + } else { + bodyParts.push(replaceInlineBase64(sanitizeString(doc.body.textContent || ""))); + } } else { - bodyParts.push(replaceInlineBase64(body)); + bodyParts.push(replaceInlineBase64(sanitizeString(body))); } } -function buildEmailText(full) { +function collectRawText(part, bodyParts, attachments) { + if (part.parts && part.parts.length) { + for (const p of part.parts) collectRawText(p, bodyParts, attachments); + return; + } + const ct = (part.contentType || "text/plain").toLowerCase(); + const cd = (part.headers?.["content-disposition"]?.[0] || "").toLowerCase(); + const body = String(part.body || ""); + if (cd.includes("attachment") || !ct.startsWith("text/")) { + const nameMatch = /filename\s*=\s*"?([^";]+)/i.exec(cd) || /name\s*=\s*"?([^";]+)/i.exec(part.headers?.["content-type"]?.[0] || ""); + const name = nameMatch ? nameMatch[1] : ""; + attachments.push(`${name} (${ct}, ${part.size || byteSize(body)} bytes)`); + } else if (ct.startsWith("text/html")) { + const doc = new DOMParser().parseFromString(body, 'text/html'); + bodyParts.push(doc.body.textContent || ""); + } else { + bodyParts.push(body); + } +} + +function buildEmailText(full, applyTransforms = true) { const bodyParts = []; const attachments = []; - collectText(full, bodyParts, attachments); + const collect = applyTransforms ? collectText : collectRawText; + collect(full, bodyParts, attachments); const headers = Object.entries(full.headers || {}) - .map(([k,v]) => `${k}: ${v.join(' ')}`) + .map(([k, v]) => `${k}: ${v.join(' ')}`) .join('\n'); - const attachInfo = `Attachments: ${attachments.length}` + (attachments.length ? "\n" + attachments.map(a => ` - ${a}`).join('\n') : ""); - return `${headers}\n${attachInfo}\n\n${bodyParts.join('\n')}`.trim(); + const attachInfo = `Attachments: ${attachments.length}` + + (attachments.length ? "\n" + attachments.map(a => ` - ${a}`).join('\n') : ""); + let combined = `${headers}\n${attachInfo}\n\n${bodyParts.join('\n')}`.trim(); + if (applyTransforms && tokenReduction) { + const seen = new Set(); + combined = combined.split('\n').filter(l => { + if (seen.has(l)) return false; + seen.add(l); + return true; + }).join('\n'); + } + return applyTransforms ? sanitizeString(combined) : combined; +} + +function updateTimingStats(elapsed) { + const t = timingStats; + t.count += 1; + t.total += elapsed; + t.last = elapsed; + const delta = elapsed - t.mean; + t.mean += delta / t.count; + t.m2 += delta * (elapsed - t.mean); +} + +async function getAllMessageIds(list) { + const ids = []; + if (!list) { + return ids; + } + let page = list; + ids.push(...(page.messages || []).map(m => m.id)); + while (page.id) { + page = await messenger.messages.continueList(page.id); + ids.push(...(page.messages || []).map(m => m.id)); + } + return ids; +} + +async function processMessage(id) { + processing = true; + currentStart = Date.now(); + queuedCount--; + updateActionIcon(); + try { + const full = await messenger.messages.getFull(id); + const originalText = buildEmailText(full, false); + let text = buildEmailText(full); + if (tokenReduction && maxTokens > 0) { + const limit = Math.floor(maxTokens * 0.9); + if (text.length > limit) { + text = text.slice(0, limit); + } + } + if (showDebugTab) { + await storage.local.set({ lastFullText: originalText, lastPromptText: text }); + } + let hdr; + let currentTags = []; + let alreadyRead = false; + let identityId = null; + try { + hdr = await messenger.messages.get(id); + currentTags = Array.isArray(hdr.tags) ? [...hdr.tags] : []; + alreadyRead = hdr.read === true; + const ids = await messenger.identities.list(hdr.folder.accountId); + identityId = ids[0]?.id || null; + } catch (e) { + currentTags = []; + alreadyRead = false; + identityId = null; + } + + for (const rule of aiRules) { + if (rule.enabled === false) { + continue; + } + if (hdr && Array.isArray(rule.accounts) && rule.accounts.length && + !rule.accounts.includes(hdr.folder.accountId)) { + continue; + } + if (hdr && Array.isArray(rule.folders) && rule.folders.length && + !rule.folders.includes(hdr.folder.path)) { + continue; + } + if (rule.unreadOnly && alreadyRead) { + continue; + } + if (hdr && (typeof rule.minAgeDays === 'number' || typeof rule.maxAgeDays === 'number')) { + const msgTime = new Date(hdr.date).getTime(); + if (!isNaN(msgTime)) { + const ageDays = (Date.now() - msgTime) / 86400000; + if (typeof rule.minAgeDays === 'number' && ageDays < rule.minAgeDays) { + continue; + } + if (typeof rule.maxAgeDays === 'number' && ageDays > rule.maxAgeDays) { + continue; + } + } + } + const cacheKey = await AiClassifier.buildCacheKey(id, rule.criterion); + const matched = await AiClassifier.classifyText(text, rule.criterion, cacheKey); + if (matched) { + for (const act of (rule.actions || [])) { + if (act.type === 'tag' && act.tagKey) { + if (!currentTags.includes(act.tagKey)) { + currentTags.push(act.tagKey); + await messenger.messages.update(id, { tags: currentTags }); + } + } else if (act.type === 'move' && act.folder) { + await messenger.messages.move([id], act.folder); + } else if (act.type === 'copy' && act.copyTarget) { + await messenger.messages.copy([id], act.copyTarget); + } else if (act.type === 'junk') { + await messenger.messages.update(id, { junk: !!act.junk }); + } else if (act.type === 'read') { + await messenger.messages.update(id, { read: !!act.read }); + } else if (act.type === 'flag') { + await messenger.messages.update(id, { flagged: !!act.flagged }); + } else if (act.type === 'delete') { + await messenger.messages.delete([id]); + } else if (act.type === 'archive') { + await messenger.messages.archive([id]); + } else if (act.type === 'forward' && act.address && identityId) { + await browser.compose.beginForward(id, { to: [act.address], identityId }); + } else if (act.type === 'reply' && act.replyType && identityId) { + await browser.compose.beginReply(id, { replyType: act.replyType, identityId }); + } + } + if (rule.stopProcessing) { + break; + } + } + } + processing = false; + const elapsed = Date.now() - currentStart; + currentStart = 0; + updateTimingStats(elapsed); + await storage.local.set({ classifyStats: timingStats }); + showTransientIcon(ICONS.circle); + } catch (e) { + processing = false; + const elapsed = Date.now() - currentStart; + currentStart = 0; + updateTimingStats(elapsed); + await storage.local.set({ classifyStats: timingStats, errorPending: true }); + errorPending = true; + logger.aiLog("failed to apply AI rules", { level: 'error' }, e); + setIcon(ICONS.error()); + browser.notifications.create(ERROR_NOTIFICATION_ID, { + type: 'basic', + iconUrl: browser.runtime.getURL('resources/img/logo.png'), + title: 'Sortana Error', + message: 'Failed to apply AI rules', + buttons: [{ title: 'Dismiss' }] + }); + } } async function applyAiRules(idsInput) { const ids = Array.isArray(idsInput) ? idsInput : [idsInput]; @@ -92,86 +401,14 @@ async function applyAiRules(idsInput) { if (!aiRules.length) { const { aiRules: stored } = await storage.local.get("aiRules"); - aiRules = Array.isArray(stored) ? stored.map(r => { - if (r.actions) return r; - const actions = []; - if (r.tag) actions.push({ type: 'tag', tagKey: r.tag }); - if (r.moveTo) actions.push({ type: 'move', folder: r.moveTo }); - const rule = { criterion: r.criterion, actions }; - if (r.stopProcessing) rule.stopProcessing = true; - return rule; - }) : []; + aiRules = normalizeRules(stored); } for (const msg of ids) { const id = msg?.id ?? msg; queuedCount++; updateActionIcon(); - queue = queue.then(async () => { - processing = true; - currentStart = Date.now(); - queuedCount--; - updateActionIcon(); - try { - const full = await messenger.messages.getFull(id); - const text = buildEmailText(full); - let currentTags = []; - try { - const hdr = await messenger.messages.get(id); - currentTags = Array.isArray(hdr.tags) ? [...hdr.tags] : []; - } catch (e) { - currentTags = []; - } - - for (const rule of aiRules) { - const cacheKey = await AiClassifier.buildCacheKey(id, rule.criterion); - const matched = await AiClassifier.classifyText(text, rule.criterion, cacheKey); - if (matched) { - for (const act of (rule.actions || [])) { - if (act.type === 'tag' && act.tagKey) { - if (!currentTags.includes(act.tagKey)) { - currentTags.push(act.tagKey); - await messenger.messages.update(id, { tags: currentTags }); - } - } else if (act.type === 'move' && act.folder) { - await messenger.messages.move([id], act.folder); - } else if (act.type === 'junk') { - await messenger.messages.update(id, { junk: !!act.junk }); - } - } - if (rule.stopProcessing) { - break; - } - } - } - processing = false; - const elapsed = Date.now() - currentStart; - currentStart = 0; - const t = timingStats; - t.count += 1; - t.total += elapsed; - t.last = elapsed; - const delta = elapsed - t.mean; - t.mean += delta / t.count; - t.m2 += delta * (elapsed - t.mean); - await storage.local.set({ classifyStats: t }); - showTransientIcon("resources/img/done.png"); - } catch (e) { - processing = false; - const elapsed = Date.now() - currentStart; - currentStart = 0; - const t = timingStats; - t.count += 1; - t.total += elapsed; - t.last = elapsed; - const delta = elapsed - t.mean; - t.mean += delta / t.count; - t.m2 += delta * (elapsed - t.mean); - await storage.local.set({ classifyStats: t }); - logger.aiLog("failed to apply AI rules", { level: 'error' }, e); - showTransientIcon("resources/img/error.png"); - } - }); + queue = queue.then(() => processMessage(id)); } return queue; @@ -183,15 +420,7 @@ async function clearCacheForMessages(idsInput) { if (!aiRules.length) { const { aiRules: stored } = await storage.local.get("aiRules"); - aiRules = Array.isArray(stored) ? stored.map(r => { - if (r.actions) return r; - const actions = []; - if (r.tag) actions.push({ type: 'tag', tagKey: r.tag }); - if (r.moveTo) actions.push({ type: 'move', folder: r.moveTo }); - const rule = { criterion: r.criterion, actions }; - if (r.stopProcessing) rule.stopProcessing = true; - return rule; - }) : []; + aiRules = normalizeRules(stored); } const keys = []; @@ -204,25 +433,40 @@ async function clearCacheForMessages(idsInput) { } if (keys.length) { await AiClassifier.removeCacheEntries(keys); - showTransientIcon("resources/img/done.png"); + showTransientIcon(ICONS.circle); } } (async () => { logger = await import(browser.runtime.getURL("logger.js")); + ({ detectSystemTheme } = await import(browser.runtime.getURL('modules/themeUtils.js'))); try { AiClassifier = await import(browser.runtime.getURL("modules/AiClassifier.js")); - logger.aiLog("AiClassifier imported", {debug: true}); + logger.aiLog("AiClassifier imported", { debug: true }); + const td = await import(browser.runtime.getURL("resources/js/turndown.js")); + TurndownService = td.default || td.TurndownService; } catch (e) { console.error("failed to import AiClassifier", e); return; } try { - const store = await storage.local.get(["endpoint", "templateName", "customTemplate", "customSystemPrompt", "aiParams", "debugLogging", "aiRules"]); + const store = await storage.local.get(["endpoint", "templateName", "customTemplate", "customSystemPrompt", "aiParams", "debugLogging", "htmlToMarkdown", "stripUrlParams", "altTextImages", "collapseWhitespace", "tokenReduction", "aiRules", "theme", "errorPending", "showDebugTab"]); logger.setDebug(store.debugLogging); await AiClassifier.setConfig(store); + userTheme = store.theme || 'auto'; + currentTheme = userTheme === 'auto' ? await detectSystemTheme() : userTheme; await AiClassifier.init(); + htmlToMarkdown = store.htmlToMarkdown === true; + stripUrlParams = store.stripUrlParams === true; + altTextImages = store.altTextImages === true; + collapseWhitespace = store.collapseWhitespace === true; + tokenReduction = store.tokenReduction === true; + if (store.aiParams && typeof store.aiParams.max_tokens !== 'undefined') { + maxTokens = parseInt(store.aiParams.max_tokens) || maxTokens; + } + errorPending = store.errorPending === true; + showDebugTab = store.showDebugTab === true; const savedStats = await storage.local.get('classifyStats'); if (savedStats.classifyStats && typeof savedStats.classifyStats === 'object') { Object.assign(timingStats, savedStats.classifyStats); @@ -230,93 +474,152 @@ async function clearCacheForMessages(idsInput) { if (typeof timingStats.last !== 'number') { timingStats.last = -1; } - aiRules = Array.isArray(store.aiRules) ? store.aiRules.map(r => { - if (r.actions) return r; - const actions = []; - if (r.tag) actions.push({ type: 'tag', tagKey: r.tag }); - if (r.moveTo) actions.push({ type: 'move', folder: r.moveTo }); - const rule = { criterion: r.criterion, actions }; - if (r.stopProcessing) rule.stopProcessing = true; - return rule; - }) : []; - logger.aiLog("configuration loaded", {debug: true}, store); + aiRules = normalizeRules(store.aiRules); + logger.aiLog("configuration loaded", { debug: true }, store); storage.onChanged.addListener(async changes => { if (changes.aiRules) { const newRules = changes.aiRules.newValue || []; - aiRules = newRules.map(r => { - if (r.actions) return r; - const actions = []; - if (r.tag) actions.push({ type: 'tag', tagKey: r.tag }); - if (r.moveTo) actions.push({ type: 'move', folder: r.moveTo }); - const rule = { criterion: r.criterion, actions }; - if (r.stopProcessing) rule.stopProcessing = true; - return rule; - }); - logger.aiLog("aiRules updated from storage change", {debug: true}, aiRules); + aiRules = normalizeRules(newRules); + logger.aiLog("aiRules updated from storage change", { debug: true }, aiRules); + } + if (changes.endpoint || changes.templateName || changes.customTemplate || changes.customSystemPrompt || changes.aiParams || changes.debugLogging) { + const config = {}; + if (changes.endpoint) config.endpoint = changes.endpoint.newValue; + if (changes.templateName) config.templateName = changes.templateName.newValue; + if (changes.customTemplate) config.customTemplate = changes.customTemplate.newValue; + if (changes.customSystemPrompt) config.customSystemPrompt = changes.customSystemPrompt.newValue; + if (changes.aiParams) { + config.aiParams = changes.aiParams.newValue; + if (changes.aiParams.newValue && typeof changes.aiParams.newValue.max_tokens !== 'undefined') { + maxTokens = parseInt(changes.aiParams.newValue.max_tokens) || maxTokens; + } + } + if (changes.debugLogging) { + config.debugLogging = changes.debugLogging.newValue === true; + logger.setDebug(config.debugLogging); + } + await AiClassifier.setConfig(config); + logger.aiLog("AiClassifier config updated from storage change", { debug: true }, config); + } + if (changes.htmlToMarkdown) { + htmlToMarkdown = changes.htmlToMarkdown.newValue === true; + logger.aiLog("htmlToMarkdown updated from storage change", { debug: true }, htmlToMarkdown); + } + if (changes.stripUrlParams) { + stripUrlParams = changes.stripUrlParams.newValue === true; + logger.aiLog("stripUrlParams updated from storage change", { debug: true }, stripUrlParams); + } + if (changes.altTextImages) { + altTextImages = changes.altTextImages.newValue === true; + logger.aiLog("altTextImages updated from storage change", { debug: true }, altTextImages); + } + if (changes.collapseWhitespace) { + collapseWhitespace = changes.collapseWhitespace.newValue === true; + logger.aiLog("collapseWhitespace updated from storage change", { debug: true }, collapseWhitespace); + } + if (changes.tokenReduction) { + tokenReduction = changes.tokenReduction.newValue === true; + logger.aiLog("tokenReduction updated from storage change", { debug: true }, tokenReduction); + } + if (changes.showDebugTab) { + showDebugTab = changes.showDebugTab.newValue === true; + } + if (changes.errorPending) { + errorPending = changes.errorPending.newValue === true; + updateActionIcon(); + } + if (changes.theme) { + userTheme = changes.theme.newValue || 'auto'; + currentTheme = userTheme === 'auto' ? await detectSystemTheme() : userTheme; + updateActionIcon(); + refreshMenuIcons(); } }); + + if (browser.theme?.onUpdated) { + browser.theme.onUpdated.addListener(async () => { + if (userTheme === 'auto') { + const theme = await detectSystemTheme(); + if (theme !== currentTheme) { + currentTheme = theme; + updateActionIcon(); + refreshMenuIcons(); + } + } + }); + } } catch (err) { - logger.aiLog("failed to load config", {level: 'error'}, err); + logger.aiLog("failed to load config", { level: 'error' }, err); } - logger.aiLog("background.js loaded – ready to classify", {debug: true}); + logger.aiLog("background.js loaded – ready to classify", { debug: true }); + updateActionIcon(); if (browser.messageDisplayAction) { browser.messageDisplayAction.setTitle({ title: "Details" }); if (browser.messageDisplayAction.setLabel) { browser.messageDisplayAction.setLabel({ label: "Details" }); } + } browser.menus.create({ id: "apply-ai-rules-list", title: "Apply AI Rules", contexts: ["message_list"], + icons: iconPaths('eye') }); browser.menus.create({ id: "apply-ai-rules-display", title: "Apply AI Rules", contexts: ["message_display_action"], + icons: iconPaths('eye') }); browser.menus.create({ id: "clear-ai-cache-list", title: "Clear AI Cache", contexts: ["message_list"], + icons: iconPaths('trash') }); browser.menus.create({ id: "clear-ai-cache-display", title: "Clear AI Cache", contexts: ["message_display_action"], + icons: iconPaths('trash') }); browser.menus.create({ id: "view-ai-reason-list", title: "View Reasoning", contexts: ["message_list"], - icons: { "16": "resources/img/brain.png" } + icons: iconPaths('clipboarddata') }); browser.menus.create({ id: "view-ai-reason-display", title: "View Reasoning", contexts: ["message_display_action"], - icons: { "16": "resources/img/brain.png" } + icons: iconPaths('clipboarddata') }); + refreshMenuIcons(); - - - browser.menus.onClicked.addListener(async info => { + browser.menus.onClicked.addListener(async (info, tab) => { if (info.menuItemId === "apply-ai-rules-list" || info.menuItemId === "apply-ai-rules-display") { - const ids = info.selectedMessages?.messages?.map(m => m.id) || - (info.messageId ? [info.messageId] : []); + let ids = info.messageId ? [info.messageId] : []; + if (info.selectedMessages) { + ids = await getAllMessageIds(info.selectedMessages); + } await applyAiRules(ids); } else if (info.menuItemId === "clear-ai-cache-list" || info.menuItemId === "clear-ai-cache-display") { - const ids = info.selectedMessages?.messages?.map(m => m.id) || - (info.messageId ? [info.messageId] : []); + let ids = info.messageId ? [info.messageId] : []; + if (info.selectedMessages) { + ids = await getAllMessageIds(info.selectedMessages); + } await clearCacheForMessages(ids); } else if (info.menuItemId === "view-ai-reason-list" || info.menuItemId === "view-ai-reason-display") { - const id = info.messageId || info.selectedMessages?.messages?.[0]?.id; - if (id) { - const url = browser.runtime.getURL(`details.html?mid=${id}`); - browser.tabs.create({ url }); - } + const [header] = await browser.messageDisplay.getDisplayedMessages(tab.id); + if (!header) { return; } + + const url = `${browser.runtime.getURL("details.html")}?mid=${header.id}`; + + await browser.tabs.create({ url }); } }); @@ -328,134 +631,139 @@ async function clearCacheForMessages(idsInput) { } if (msg?.type === "sortana:test") { - const { text = "", criterion = "" } = msg; - logger.aiLog("sortana:test – text", {debug: true}, text); - logger.aiLog("sortana:test – criterion", {debug: true}, criterion); + const { text = "", criterion = "" } = msg; + logger.aiLog("sortana:test – text", { debug: true }, text); + logger.aiLog("sortana:test – criterion", { debug: true }, criterion); - try { - logger.aiLog("Calling AiClassifier.classifyText()", {debug: true}); - const result = await AiClassifier.classifyText(text, criterion); - logger.aiLog("classify() returned", {debug: true}, result); - return { match: result }; - } - catch (err) { - logger.aiLog("Error in classify()", {level: 'error'}, err); - // rethrow so the caller sees the failure - throw err; - } - } else if (msg?.type === "sortana:clearCacheForDisplayed") { - try { - const tabs = await browser.tabs.query({ active: true, currentWindow: true }); - const tabId = tabs[0]?.id; - const msgs = tabId ? await browser.messageDisplay.getDisplayedMessages(tabId) : []; - const ids = msgs.map(m => m.id); - await clearCacheForMessages(ids); - } catch (e) { - logger.aiLog("failed to clear cache from message script", { level: 'error' }, e); - } - } else if (msg?.type === "sortana:getReasons") { - try { - const id = msg.id; - const hdr = await messenger.messages.get(id); - const subject = hdr?.subject || ""; - if (!aiRules.length) { - const { aiRules: stored } = await storage.local.get("aiRules"); - aiRules = Array.isArray(stored) ? stored.map(r => { - if (r.actions) return r; - const actions = []; - if (r.tag) actions.push({ type: 'tag', tagKey: r.tag }); - if (r.moveTo) actions.push({ type: 'move', folder: r.moveTo }); - const rule = { criterion: r.criterion, actions }; - if (r.stopProcessing) rule.stopProcessing = true; - return rule; - }) : []; + try { + logger.aiLog("Calling AiClassifier.classifyText()", { debug: true }); + const result = await AiClassifier.classifyText(text, criterion); + logger.aiLog("classify() returned", { debug: true }, result); + return { match: result }; } - const reasons = []; - for (const rule of aiRules) { - const key = await AiClassifier.buildCacheKey(id, rule.criterion); - const reason = AiClassifier.getReason(key); - if (reason) { - reasons.push({ criterion: rule.criterion, reason }); + catch (err) { + logger.aiLog("Error in classify()", { level: 'error' }, err); + // rethrow so the caller sees the failure + throw err; + } + } else if (msg?.type === "sortana:clearCacheForDisplayed") { + try { + const msgs = await browser.messageDisplay.getDisplayedMessages(); + const ids = msgs.map(m => m.id); + await clearCacheForMessages(ids); + } catch (e) { + logger.aiLog("failed to clear cache from message script", { level: 'error' }, e); + } + } else if (msg?.type === "sortana:getReasons") { + try { + const id = msg.id; + const hdr = await messenger.messages.get(id); + const subject = hdr?.subject || ""; + if (!aiRules.length) { + const { aiRules: stored } = await storage.local.get("aiRules"); + aiRules = normalizeRules(stored); } - } - return { subject, reasons }; - } catch (e) { - logger.aiLog("failed to collect reasons", { level: 'error' }, e); - return { subject: '', reasons: [] }; - } - } else if (msg?.type === "sortana:getDetails") { - try { - const id = msg.id; - const hdr = await messenger.messages.get(id); - const subject = hdr?.subject || ""; - if (!aiRules.length) { - const { aiRules: stored } = await storage.local.get("aiRules"); - aiRules = Array.isArray(stored) ? stored.map(r => { - if (r.actions) return r; - const actions = []; - if (r.tag) actions.push({ type: 'tag', tagKey: r.tag }); - if (r.moveTo) actions.push({ type: 'move', folder: r.moveTo }); - const rule = { criterion: r.criterion, actions }; - if (r.stopProcessing) rule.stopProcessing = true; - return rule; - }) : []; - } - const results = []; - for (const rule of aiRules) { - const key = await AiClassifier.buildCacheKey(id, rule.criterion); - const matched = AiClassifier.getCachedResult(key); - const reason = AiClassifier.getReason(key); - if (matched !== null || reason) { - results.push({ criterion: rule.criterion, matched, reason }); + const reasons = []; + for (const rule of aiRules) { + const key = await AiClassifier.buildCacheKey(id, rule.criterion); + const reason = AiClassifier.getReason(key); + if (reason) { + reasons.push({ criterion: rule.criterion, reason }); + } } + return { subject, reasons }; + } catch (e) { + logger.aiLog("failed to collect reasons", { level: 'error' }, e); + return { subject: '', reasons: [] }; } - return { subject, results }; - } catch (e) { - logger.aiLog("failed to collect details", { level: 'error' }, e); - return { subject: '', results: [] }; - } - } else if (msg?.type === "sortana:clearCacheForMessage") { - try { - await clearCacheForMessages([msg.id]); - return { ok: true }; - } catch (e) { - logger.aiLog("failed to clear cache for message", { level: 'error' }, e); - return { ok: false }; - } - } else if (msg?.type === "sortana:getQueueCount") { - return { count: queuedCount + (processing ? 1 : 0) }; - } else if (msg?.type === "sortana:getTiming") { - const t = timingStats; - const std = t.count > 1 ? Math.sqrt(t.m2 / (t.count - 1)) : 0; - return { - count: queuedCount + (processing ? 1 : 0), - current: currentStart ? Date.now() - currentStart : -1, - last: t.last, - runs: t.count, - average: t.mean, - total: t.total, - stddev: std - }; - } else { - logger.aiLog("Unknown message type, ignoring", {level: 'warn'}, msg?.type); - } -}); + } else if (msg?.type === "sortana:getDetails") { + try { + const id = msg.id; + const hdr = await messenger.messages.get(id); + const subject = hdr?.subject || ""; + if (!aiRules.length) { + const { aiRules: stored } = await storage.local.get("aiRules"); + aiRules = normalizeRules(stored); + } + const results = []; + for (const rule of aiRules) { + const key = await AiClassifier.buildCacheKey(id, rule.criterion); + const matched = AiClassifier.getCachedResult(key); + const reason = AiClassifier.getReason(key); + if (matched !== null || reason) { + results.push({ criterion: rule.criterion, matched, reason }); + } + } + return { subject, results }; + } catch (e) { + logger.aiLog("failed to collect details", { level: 'error' }, e); + return { subject: '', results: [] }; + } + } else if (msg?.type === "sortana:getDisplayedMessages") { + try { + const [tab] = await browser.tabs.query({ active: true, currentWindow: true }); + const messages = await browser.messageDisplay.getDisplayedMessages(tab?.id); + const ids = messages.map(hdr => hdr.id); -// Automatically classify new messages -if (typeof messenger !== "undefined" && messenger.messages?.onNewMailReceived) { - messenger.messages.onNewMailReceived.addListener(async (folder, messages) => { - logger.aiLog("onNewMailReceived", {debug: true}, messages); - const ids = (messages?.messages || messages || []).map(m => m.id ?? m); - await applyAiRules(ids); + return { ids }; + } catch (e) { + logger.aiLog("failed to get displayed messages", { level: 'error' }, e); + return { messages: [] }; + } + } else if (msg?.type === "sortana:clearCacheForMessage") { + try { + await clearCacheForMessages([msg.id]); + return { ok: true }; + } catch (e) { + logger.aiLog("failed to clear cache for message", { level: 'error' }, e); + return { ok: false }; + } + } else if (msg?.type === "sortana:getQueueCount") { + return { count: queuedCount + (processing ? 1 : 0) }; + } else if (msg?.type === "sortana:getTiming") { + const t = timingStats; + const std = t.count > 1 ? Math.sqrt(t.m2 / (t.count - 1)) : 0; + return { + count: queuedCount + (processing ? 1 : 0), + current: currentStart ? Date.now() - currentStart : -1, + last: t.last, + runs: t.count, + average: t.mean, + total: t.total, + stddev: std + }; + } else { + logger.aiLog("Unknown message type, ignoring", { level: 'warn' }, msg?.type); + } }); -} else { - logger.aiLog("messenger.messages API unavailable, skipping new mail listener", { level: 'warn' }); -} -// Catch any unhandled rejections -window.addEventListener("unhandledrejection", ev => { - logger.aiLog("Unhandled promise rejection", {level: 'error'}, ev.reason); -}); + // Automatically classify new messages + if (typeof messenger !== "undefined" && messenger.messages?.onNewMailReceived) { + messenger.messages.onNewMailReceived.addListener(async (folder, messages) => { + logger.aiLog("onNewMailReceived", { debug: true }, messages); + const ids = (messages?.messages || messages || []).map(m => m.id ?? m); + await applyAiRules(ids); + }); + } else { + logger.aiLog("messenger.messages API unavailable, skipping new mail listener", { level: 'warn' }); + } + + // Catch any unhandled rejections + window.addEventListener("unhandledrejection", ev => { + logger.aiLog("Unhandled promise rejection", { level: 'error' }, ev.reason); + }); + + browser.notifications.onClicked.addListener(id => { + if (id === ERROR_NOTIFICATION_ID) { + clearError(); + } + }); + + browser.notifications.onButtonClicked.addListener((id) => { + if (id === ERROR_NOTIFICATION_ID) { + clearError(); + } + }); browser.runtime.onInstalled.addListener(async ({ reason }) => { if (reason === "install") { diff --git a/details.html b/details.html index 1502471..d15a3c9 100644 --- a/details.html +++ b/details.html @@ -15,6 +15,6 @@ - + diff --git a/details.js b/details.js index 0d190d5..f306e01 100644 --- a/details.js +++ b/details.js @@ -1,23 +1,51 @@ -document.addEventListener('DOMContentLoaded', async () => { - const params = new URLSearchParams(location.search); - let id = parseInt(params.get('mid'), 10); +const aiLog = (await import(browser.runtime.getURL("logger.js"))).aiLog; +const storage = (globalThis.messenger ?? browser).storage; +const { detectSystemTheme } = await import(browser.runtime.getURL('modules/themeUtils.js')); +const { theme } = await storage.local.get('theme'); +const mode = (theme || 'auto') === 'auto' + ? await detectSystemTheme() + : theme; +document.documentElement.dataset.theme = mode; - if (!id) { - try { +const qMid = parseInt(new URLSearchParams(location.search).get("mid"), 10); +if (!isNaN(qMid)) { + loadMessage(qMid); +} else { + const { ids } = await browser.runtime.sendMessage({ + type: "sortana:getDisplayedMessages", + }); + if (ids && ids[0]) { + loadMessage(ids[0]); + } else { const tabs = await browser.tabs.query({ active: true, currentWindow: true }); const tabId = tabs[0]?.id; const msgs = tabId ? await browser.messageDisplay.getDisplayedMessages(tabId) : []; - id = msgs[0]?.id; - } catch (e) { - console.error('failed to determine message id', e); - } + let id = msgs[0]?.id; + if (id) { + loadMessage(id); + } + else { + aiLog("Details popup: no displayed message found"); + } } - if (!id) return; +} + +async function loadMessage(id) { + const storage = (globalThis.messenger ?? browser).storage; + const logMod = await import(browser.runtime.getURL('logger.js')); + const { debugLogging } = await storage.local.get('debugLogging'); + logMod.setDebug(debugLogging === true); + const log = logMod.aiLog; + + log('details page loaded', { debug: true }); try { + log('requesting message details', {}, id); const { subject, results } = await browser.runtime.sendMessage({ type: 'sortana:getDetails', id }); + log('received details', { debug: true }, { subject, results }); document.getElementById('subject').textContent = subject; const container = document.getElementById('rules'); for (const r of results) { + log('rendering rule result', { debug: true }, r); const article = document.createElement('article'); const color = r.matched === true ? 'is-success' : 'is-danger'; article.className = `message ${color} mb-4`; @@ -37,10 +65,11 @@ document.addEventListener('DOMContentLoaded', async () => { container.appendChild(article); } document.getElementById('clear').addEventListener('click', async () => { + log('clearing cache for message', {}, id); await browser.runtime.sendMessage({ type: 'sortana:clearCacheForMessage', id }); window.close(); }); } catch (e) { - console.error('failed to load details', e); + log('failed to load details', { level: 'error' }, e); } -}); +} diff --git a/manifest.json b/manifest.json index 3f17845..e7cb9d8 100644 --- a/manifest.json +++ b/manifest.json @@ -1,13 +1,13 @@ { "manifest_version": 2, "name": "Sortana", - "version": "2.0.0", + "version": "2.2.0", "default_locale": "en-US", "applications": { "gecko": { "id": "ai-filter@jordanwages", "strict_min_version": "128.0", - "strict_max_version": "*" + "strict_max_version": "140.*" } }, "icons": { @@ -22,7 +22,7 @@ "default_icon": "resources/img/logo32.png" }, "message_display_action": { - "default_icon": "resources/img/brain.png", + "default_icon": "resources/img/logo.png", "default_title": "Details", "default_label": "Details", "default_popup": "details.html" @@ -32,14 +32,18 @@ "page": "options/options.html", "open_in_tab": true }, - "permissions": [ - "storage", - "messagesRead", - "messagesMove", - "messagesUpdate", - "messagesTagsList", - "accountsRead", - "menus", - "scripting" - ] + "permissions": [ + "storage", + "messagesRead", + "messagesMove", + "messagesUpdate", + "messagesTagsList", + "accountsRead", + "menus", + "notifications", + "scripting", + "tabs", + "theme", + "compose" + ] } diff --git a/modules/AiClassifier.js b/modules/AiClassifier.js index 3c526f8..8313654 100644 --- a/modules/AiClassifier.js +++ b/modules/AiClassifier.js @@ -1,5 +1,6 @@ "use strict"; import { aiLog, setDebug } from "../logger.js"; +import { DEFAULT_AI_PARAMS } from "./defaultParams.js"; const storage = (globalThis.messenger ?? globalThis.browser).storage; @@ -33,19 +34,7 @@ let gCustomTemplate = ""; let gCustomSystemPrompt = DEFAULT_CUSTOM_SYSTEM_PROMPT; let gTemplateText = ""; -let gAiParams = { - max_tokens: 4096, - temperature: 0.6, - top_p: 0.95, - seed: -1, - repetition_penalty: 1.0, - top_k: 20, - min_p: 0, - presence_penalty: 0, - frequency_penalty: 0, - typical_p: 1, - tfs: 1, -}; +let gAiParams = Object.assign({}, DEFAULT_AI_PARAMS); let gCache = new Map(); let gCacheLoaded = false; @@ -72,10 +61,6 @@ async function sha256Hex(str) { return sha256HexSync(str); } -function buildCacheKeySync(id, criterion) { - return sha256HexSync(`${id}|${criterion}`); -} - async function resolveHeaderId(id) { if (typeof id === "number" && typeof messenger?.messages?.get === "function") { try { @@ -93,7 +78,7 @@ async function resolveHeaderId(id) { async function buildCacheKey(id, criterion) { const resolvedId = await resolveHeaderId(id); if (Services) { - return buildCacheKeySync(resolvedId, criterion); + return sha256HexSync(`${resolvedId}|${criterion}`); } return sha256Hex(`${resolvedId}|${criterion}`); } @@ -133,16 +118,6 @@ async function loadCache() { gCacheLoaded = true; } -function loadCacheSync() { - if (!gCacheLoaded) { - if (!Services?.tm?.spinEventLoopUntil) { - throw new Error("loadCacheSync requires Services"); - } - let done = false; - loadCache().finally(() => { done = true; }); - Services.tm.spinEventLoopUntil(() => done); - } -} async function saveCache(updatedKey, updatedValue) { if (typeof updatedKey !== "undefined") { @@ -233,11 +208,7 @@ function buildPrompt(body, criterion) { function getCachedResult(cacheKey) { if (!gCacheLoaded) { - if (Services?.tm?.spinEventLoopUntil) { - loadCacheSync(); - } else { - return null; - } + return null; } if (cacheKey && gCache.has(cacheKey)) { aiLog(`[AiClassifier] Cache hit for key: ${cacheKey}`, {debug: true}); @@ -249,11 +220,7 @@ function getCachedResult(cacheKey) { function getReason(cacheKey) { if (!gCacheLoaded) { - if (Services?.tm?.spinEventLoopUntil) { - loadCacheSync(); - } else { - return null; - } + return null; } const entry = gCache.get(cacheKey); return cacheKey && entry ? entry.reason || null : null; @@ -330,48 +297,6 @@ async function getCacheSize() { return gCache.size; } -function classifyTextSync(text, criterion, cacheKey = null) { - if (!Services?.tm?.spinEventLoopUntil) { - throw new Error("classifyTextSync requires Services"); - } - const cached = getCachedResult(cacheKey); - if (cached !== null) { - return cached; - } - - const payload = buildPayload(text, criterion); - - aiLog(`[AiClassifier] Sending classification request to ${gEndpoint}`, {debug: true}); - - let result; - let done = false; - (async () => { - try { - const response = await fetch(gEndpoint, { - method: "POST", - headers: { "Content-Type": "application/json" }, - body: payload, - }); - if (response.ok) { - const json = await response.json(); - aiLog(`[AiClassifier] Received response:`, {debug: true}, json); - result = parseMatch(json); - cacheEntry(cacheKey, result.matched, result.reason); - result = result.matched; - } else { - aiLog(`HTTP status ${response.status}`, {level: 'warn'}); - result = false; - } - } catch (e) { - aiLog(`HTTP request failed`, {level: 'error'}, e); - result = false; - } finally { - done = true; - } - })(); - Services.tm.spinEventLoopUntil(() => done); - return result; -} async function classifyText(text, criterion, cacheKey = null) { if (!gCacheLoaded) { @@ -383,6 +308,11 @@ async function classifyText(text, criterion, cacheKey = null) { } const payload = buildPayload(text, criterion); + try { + await storage.local.set({ lastPayload: JSON.parse(payload) }); + } catch (e) { + aiLog('failed to save last payload', { level: 'warn' }, e); + } aiLog(`[AiClassifier] Sending classification request to ${gEndpoint}`, {debug: true}); aiLog(`[AiClassifier] Classification request payload:`, { debug: true }, payload); @@ -414,4 +344,4 @@ async function init() { await loadCache(); } -export { classifyText, classifyTextSync, setConfig, removeCacheEntries, clearCache, getReason, getCachedResult, buildCacheKey, buildCacheKeySync, getCacheSize, init }; +export { classifyText, setConfig, removeCacheEntries, clearCache, getReason, getCachedResult, buildCacheKey, getCacheSize, init }; diff --git a/modules/defaultParams.js b/modules/defaultParams.js new file mode 100644 index 0000000..a8afe53 --- /dev/null +++ b/modules/defaultParams.js @@ -0,0 +1,16 @@ +"use strict"; + +export const DEFAULT_AI_PARAMS = { + max_tokens: 4096, + temperature: 0.6, + top_p: 0.95, + seed: -1, + repetition_penalty: 1.0, + top_k: 20, + min_p: 0, + presence_penalty: 0, + frequency_penalty: 0, + typical_p: 1, + tfs: 1, +}; + diff --git a/modules/themeUtils.js b/modules/themeUtils.js new file mode 100644 index 0000000..58728f1 --- /dev/null +++ b/modules/themeUtils.js @@ -0,0 +1,20 @@ +"use strict"; + +export async function detectSystemTheme() { + try { + const t = await browser.theme.getCurrent(); + const scheme = t?.properties?.color_scheme; + if (scheme === 'dark' || scheme === 'light') { + return scheme; + } + const color = t?.colors?.frame || t?.colors?.toolbar; + if (color && /^#/.test(color)) { + const r = parseInt(color.slice(1, 3), 16); + const g = parseInt(color.slice(3, 5), 16); + const b = parseInt(color.slice(5, 7), 16); + const lum = (0.299 * r + 0.587 * g + 0.114 * b) / 255; + return lum < 0.5 ? 'dark' : 'light'; + } + } catch {} + return 'light'; +} diff --git a/options/dataTransfer.js b/options/dataTransfer.js new file mode 100644 index 0000000..b289c02 --- /dev/null +++ b/options/dataTransfer.js @@ -0,0 +1,45 @@ +"use strict"; +const storage = (globalThis.messenger ?? browser).storage; +const KEY_GROUPS = { + settings: [ + 'endpoint', + 'templateName', + 'customTemplate', + 'customSystemPrompt', + 'aiParams', + 'debugLogging', + 'htmlToMarkdown', + 'stripUrlParams', + 'altTextImages', + 'collapseWhitespace' + ], + rules: ['aiRules'], + cache: ['aiCache'] +}; + +function collectKeys(categories = Object.keys(KEY_GROUPS)) { + return categories.flatMap(cat => KEY_GROUPS[cat] || []); +} + +export async function exportData(categories) { + const data = await storage.local.get(collectKeys(categories)); + const blob = new Blob([JSON.stringify(data, null, 2)], { type: 'application/json' }); + const url = URL.createObjectURL(blob); + const a = document.createElement('a'); + a.href = url; + a.download = 'sortana-export.json'; + document.body.appendChild(a); + a.click(); + a.remove(); + URL.revokeObjectURL(url); +} + +export async function importData(file, categories) { + const text = await file.text(); + const parsed = JSON.parse(text); + const data = {}; + for (const key of collectKeys(categories)) { + if (key in parsed) data[key] = parsed[key]; + } + await storage.local.set(data); +} diff --git a/options/options.html b/options/options.html index cb9668a..ddc5ee0 100644 --- a/options/options.html +++ b/options/options.html @@ -31,31 +31,43 @@ .tag { --bulma-tag-h: 318; } + #diff-display { + white-space: pre-wrap; + font-family: monospace; + }
- AI Filter Logo + AI Filter Logo
- +
+

+ + Settings +

@@ -87,9 +99,28 @@
+
+ +
+
+ +
+
+
+
- - + +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
@@ -168,13 +229,19 @@
+ +
+ diff --git a/options/options.js b/options/options.js index 58dcd4f..d881d6a 100644 --- a/options/options.js +++ b/options/options.js @@ -2,6 +2,9 @@ document.addEventListener('DOMContentLoaded', async () => { const storage = (globalThis.messenger ?? browser).storage; const logger = await import(browser.runtime.getURL('logger.js')); const AiClassifier = await import(browser.runtime.getURL('modules/AiClassifier.js')); + const dataTransfer = await import(browser.runtime.getURL('options/dataTransfer.js')); + const { detectSystemTheme } = await import(browser.runtime.getURL('modules/themeUtils.js')); + const { DEFAULT_AI_PARAMS } = await import(browser.runtime.getURL('modules/defaultParams.js')); const defaults = await storage.local.get([ 'endpoint', 'templateName', @@ -9,8 +12,18 @@ document.addEventListener('DOMContentLoaded', async () => { 'customSystemPrompt', 'aiParams', 'debugLogging', + 'htmlToMarkdown', + 'stripUrlParams', + 'altTextImages', + 'collapseWhitespace', + 'tokenReduction', 'aiRules', - 'aiCache' + 'aiCache', + 'theme', + 'showDebugTab', + 'lastPayload', + 'lastFullText', + 'lastPromptText' ]); const tabButtons = document.querySelectorAll('#main-tabs li'); const tabs = document.querySelectorAll('.tab-content'); @@ -32,20 +45,61 @@ document.addEventListener('DOMContentLoaded', async () => { document.addEventListener('input', markDirty, true); document.addEventListener('change', markDirty, true); logger.setDebug(defaults.debugLogging === true); - const DEFAULT_AI_PARAMS = { - max_tokens: 4096, - temperature: 0.6, - top_p: 0.95, - seed: -1, - repetition_penalty: 1.0, - top_k: 20, - min_p: 0, - presence_penalty: 0, - frequency_penalty: 0, - typical_p: 1, - tfs: 1 - }; - document.getElementById('endpoint').value = defaults.endpoint || 'http://127.0.0.1:5000/v1/classify'; + + const themeSelect = document.getElementById('theme-select'); + themeSelect.value = defaults.theme || 'auto'; + + function updateIcons(theme) { + document.querySelectorAll('img[data-icon]').forEach(img => { + const name = img.dataset.icon; + const size = img.dataset.size || 16; + if (name === 'full-logo') { + img.src = `../resources/img/full-logo${theme === 'dark' ? '-white' : ''}.png`; + } else { + img.src = `../resources/img/${name}-${theme}-${size}.png`; + } + }); + } + + async function applyTheme(setting) { + const mode = setting === 'auto' ? await detectSystemTheme() : setting; + document.documentElement.dataset.theme = mode; + updateIcons(mode); + } + + await applyTheme(themeSelect.value); + const payloadDisplay = document.getElementById('payload-display'); + const diffDisplay = document.getElementById('diff-display'); + const diffContainer = document.getElementById('diff-container'); + + let lastFullText = defaults.lastFullText || ''; + let lastPromptText = defaults.lastPromptText || ''; + let lastPayload = defaults.lastPayload ? JSON.stringify(defaults.lastPayload, null, 2) : ''; + + if (lastPayload) { + payloadDisplay.textContent = lastPayload; + } + if (lastFullText && lastPromptText && diff_match_patch) { + const dmp = new diff_match_patch(); + dmp.Diff_EditCost = 4; + const diffs = dmp.diff_main(lastFullText, lastPromptText); + dmp.diff_cleanupEfficiency(diffs); + const hasDiff = diffs.some(d => d[0] !== 0); + if (hasDiff) { + diffDisplay.innerHTML = dmp.diff_prettyHtml(diffs); + diffContainer.classList.remove('is-hidden'); + } else { + diffDisplay.innerHTML = ''; + diffContainer.classList.add('is-hidden'); + } + } else { + diffContainer.classList.add('is-hidden'); + } + themeSelect.addEventListener('change', async () => { + markDirty(); + await applyTheme(themeSelect.value); + }); + document.getElementById('endpoint').value = defaults.endpoint || 'http://127.0.0.1:5000/v1/completions'; const templates = { openai: browser.i18n.getMessage('template.openai'), @@ -81,6 +135,32 @@ document.addEventListener('DOMContentLoaded', async () => { const debugToggle = document.getElementById('debug-logging'); debugToggle.checked = defaults.debugLogging === true; + const htmlToggle = document.getElementById('html-to-markdown'); + htmlToggle.checked = defaults.htmlToMarkdown === true; + + const stripUrlToggle = document.getElementById('strip-url-params'); + stripUrlToggle.checked = defaults.stripUrlParams === true; + + const altTextToggle = document.getElementById('alt-text-images'); + altTextToggle.checked = defaults.altTextImages === true; + + const collapseWhitespaceToggle = document.getElementById('collapse-whitespace'); + collapseWhitespaceToggle.checked = defaults.collapseWhitespace === true; + + const tokenReductionToggle = document.getElementById('token-reduction'); + tokenReductionToggle.checked = defaults.tokenReduction === true; + + const debugTabToggle = document.getElementById('show-debug-tab'); + const debugTabBtn = document.getElementById('debug-tab-button'); + function updateDebugTab() { + const visible = debugTabToggle.checked; + debugTabBtn.classList.toggle('is-hidden', !visible); + } + debugTabToggle.checked = defaults.showDebugTab === true; + debugTabToggle.addEventListener('change', () => { updateDebugTab(); markDirty(); }); + updateDebugTab(); + + const aiParams = Object.assign({}, DEFAULT_AI_PARAMS, defaults.aiParams || {}); for (const [key, val] of Object.entries(aiParams)) { const el = document.getElementById(key); @@ -89,6 +169,7 @@ document.addEventListener('DOMContentLoaded', async () => { let tagList = []; let folderList = []; + let accountList = []; try { tagList = await messenger.messages.tags.list(); } catch (e) { @@ -96,6 +177,7 @@ document.addEventListener('DOMContentLoaded', async () => { } try { const accounts = await messenger.accounts.list(true); + accountList = accounts.map(a => ({ id: a.id, name: a.name })); const collect = (f, prefix='') => { folderList.push({ id: f.id ?? f.path, name: prefix + f.name }); (f.subFolders || []).forEach(sf => collect(sf, prefix + f.name + '/')); @@ -117,6 +199,19 @@ document.addEventListener('DOMContentLoaded', async () => { const rulesContainer = document.getElementById('rules-container'); const addRuleBtn = document.getElementById('add-rule'); + const ruleCountEl = document.getElementById('rule-count'); + const cacheCountEl = document.getElementById('cache-count'); + const queueCountEl = document.getElementById('queue-count'); + const currentTimeEl = document.getElementById('current-time'); + const lastTimeEl = document.getElementById('last-time'); + const averageTimeEl = document.getElementById('average-time'); + const totalTimeEl = document.getElementById('total-time'); + const perHourEl = document.getElementById('per-hour'); + const perDayEl = document.getElementById('per-day'); + let timingLogged = false; + ruleCountEl.textContent = (defaults.aiRules || []).length; + cacheCountEl.textContent = defaults.aiCache ? Object.keys(defaults.aiCache).length : 0; + function createActionRow(action = {type: 'tag'}) { const row = document.createElement('div'); row.className = 'action-row field is-grouped mb-2'; @@ -124,7 +219,7 @@ document.addEventListener('DOMContentLoaded', async () => { const typeWrapper = document.createElement('div'); typeWrapper.className = 'select is-small mr-2'; const typeSelect = document.createElement('select'); - ['tag','move','junk'].forEach(t => { + ['tag','move','copy','junk','read','flag','delete','archive','forward','reply'].forEach(t => { const opt = document.createElement('option'); opt.value = t; opt.textContent = t; @@ -151,7 +246,7 @@ document.addEventListener('DOMContentLoaded', async () => { sel.value = action.tagKey || ''; wrap.appendChild(sel); paramSpan.appendChild(wrap); - } else if (typeSelect.value === 'move') { + } else if (typeSelect.value === 'move' || typeSelect.value === 'copy') { const wrap = document.createElement('div'); wrap.className = 'select is-small'; const sel = document.createElement('select'); @@ -162,7 +257,7 @@ document.addEventListener('DOMContentLoaded', async () => { opt.textContent = f.name; sel.appendChild(opt); } - sel.value = action.folder || ''; + sel.value = action.folder || action.copyTarget || ''; wrap.appendChild(sel); paramSpan.appendChild(wrap); } else if (typeSelect.value === 'junk') { @@ -175,6 +270,45 @@ document.addEventListener('DOMContentLoaded', async () => { sel.value = String(action.junk ?? true); wrap.appendChild(sel); paramSpan.appendChild(wrap); + } else if (typeSelect.value === 'read') { + const wrap = document.createElement('div'); + wrap.className = 'select is-small'; + const sel = document.createElement('select'); + sel.className = 'read-select'; + sel.appendChild(new Option('mark read','true')); + sel.appendChild(new Option('mark unread','false')); + sel.value = String(action.read ?? true); + wrap.appendChild(sel); + paramSpan.appendChild(wrap); + } else if (typeSelect.value === 'flag') { + const wrap = document.createElement('div'); + wrap.className = 'select is-small'; + const sel = document.createElement('select'); + sel.className = 'flag-select'; + sel.appendChild(new Option('flag','true')); + sel.appendChild(new Option('unflag','false')); + sel.value = String(action.flagged ?? true); + wrap.appendChild(sel); + paramSpan.appendChild(wrap); + } else if (typeSelect.value === 'forward') { + const input = document.createElement('input'); + input.type = 'text'; + input.className = 'input is-small forward-input'; + input.placeholder = 'address@example.com'; + input.value = action.address || ''; + paramSpan.appendChild(input); + } else if (typeSelect.value === 'reply') { + const wrap = document.createElement('div'); + wrap.className = 'select is-small'; + const sel = document.createElement('select'); + sel.className = 'reply-select'; + sel.appendChild(new Option('all','all')); + sel.appendChild(new Option('sender','sender')); + sel.value = action.replyType || 'all'; + wrap.appendChild(sel); + paramSpan.appendChild(wrap); + } else if (typeSelect.value === 'delete' || typeSelect.value === 'archive') { + paramSpan.appendChild(document.createElement('span')); } } @@ -194,7 +328,41 @@ document.addEventListener('DOMContentLoaded', async () => { return row; } + function createConditionButton(label, sectionEl, checkbox, clearFn) { + const btn = document.createElement('button'); + btn.type = 'button'; + btn.className = 'button is-small is-light'; + const icon = document.createElement('img'); + icon.width = 16; + icon.height = 16; + icon.className = 'mr-1'; + btn.appendChild(icon); + btn.append(label); + + let active = checkbox ? checkbox.checked : sectionEl && !sectionEl.classList.contains('is-hidden'); + + function update() { + btn.classList.toggle('is-active', active); + icon.src = browser.runtime.getURL(`resources/svg/${active ? 'circledot' : 'circle'}.svg`); + if (sectionEl) sectionEl.classList.toggle('is-hidden', !active); + if (checkbox) checkbox.checked = active; + if (!active && typeof clearFn === 'function') { + clearFn(); + } + } + + btn.addEventListener('click', () => { + active = !active; + markDirty(); + update(); + }); + + update(); + return btn; + } + function renderRules(rules = []) { + ruleCountEl.textContent = rules.length; rulesContainer.innerHTML = ''; for (const rule of rules) { const article = document.createElement('article'); @@ -226,13 +394,67 @@ document.addEventListener('DOMContentLoaded', async () => { const header = document.createElement('div'); header.className = 'message-header'; - header.appendChild(critInput); + + const leftWrap = document.createElement('div'); + leftWrap.style.display = 'flex'; + leftWrap.style.alignItems = 'center'; + leftWrap.style.flexGrow = '1'; + + const statusSpan = document.createElement('span'); + statusSpan.className = 'rule-status has-text-weight-semibold mr-2'; + + leftWrap.appendChild(statusSpan); + leftWrap.appendChild(critInput); + header.appendChild(leftWrap); + + const btnWrap = document.createElement('div'); + btnWrap.style.display = 'flex'; + btnWrap.style.gap = '0.25em'; + + let enabled = rule.enabled !== false; + + const toggleBtn = document.createElement('button'); + toggleBtn.type = 'button'; + toggleBtn.className = 'button is-small is-light rule-toggle'; + const toggleIcon = document.createElement('img'); + toggleIcon.width = 16; + toggleIcon.height = 16; + toggleBtn.appendChild(toggleIcon); const delBtn = document.createElement('button'); - delBtn.className = 'delete'; - delBtn.setAttribute('aria-label', 'delete'); - delBtn.addEventListener('click', () => article.remove()); - header.appendChild(delBtn); + delBtn.type = 'button'; + delBtn.className = 'button is-small is-danger is-light rule-delete'; + const delIcon = document.createElement('img'); + delIcon.src = browser.runtime.getURL('resources/svg/trash.svg'); + delIcon.width = 16; + delIcon.height = 16; + delBtn.appendChild(delIcon); + + function updateToggle() { + toggleIcon.src = browser.runtime.getURL( + `resources/svg/${enabled ? 'circleslash' : 'check'}.svg` + ); + statusSpan.textContent = enabled ? '' : '(Disabled)'; + article.dataset.enabled = String(enabled); + } + + toggleBtn.addEventListener('click', () => { + enabled = !enabled; + markDirty(); + updateToggle(); + }); + + delBtn.addEventListener('click', () => { + article.remove(); + ruleCountEl.textContent = rulesContainer.querySelectorAll('.rule').length; + markDirty(); + }); + + btnWrap.appendChild(toggleBtn); + btnWrap.appendChild(delBtn); + header.appendChild(btnWrap); + + updateToggle(); const actionsContainer = document.createElement('div'); actionsContainer.className = 'rule-actions mb-2'; @@ -247,20 +469,139 @@ document.addEventListener('DOMContentLoaded', async () => { addAction.className = 'button is-small mb-2'; addAction.addEventListener('click', () => actionsContainer.appendChild(createActionRow())); - const stopLabel = document.createElement('label'); - stopLabel.className = 'checkbox mt-2'; - const stopCheck = document.createElement('input'); - stopCheck.type = 'checkbox'; - stopCheck.className = 'stop-processing'; - stopCheck.checked = rule.stopProcessing === true; - stopLabel.appendChild(stopCheck); - stopLabel.append(' Stop after match'); + const stopLabel = document.createElement('label'); + stopLabel.className = 'checkbox mt-2 is-hidden'; + const stopCheck = document.createElement('input'); + stopCheck.type = 'checkbox'; + stopCheck.className = 'stop-processing'; + stopCheck.checked = rule.stopProcessing === true; + stopLabel.appendChild(stopCheck); + stopLabel.append(' Stop after match'); + + const unreadLabel = document.createElement('label'); + unreadLabel.className = 'checkbox mt-2 ml-4 is-hidden'; + const unreadCheck = document.createElement('input'); + unreadCheck.type = 'checkbox'; + unreadCheck.className = 'unread-only'; + unreadCheck.checked = rule.unreadOnly === true; + unreadLabel.appendChild(unreadCheck); + unreadLabel.append(' Only apply to unread messages'); + + const ageBox = document.createElement('div'); + ageBox.className = 'field is-grouped mt-2 is-hidden'; + const minInput = document.createElement('input'); + minInput.type = 'number'; + minInput.placeholder = 'Min days'; + minInput.className = 'input is-small min-age mr-2'; + minInput.style.width = '6em'; + if (typeof rule.minAgeDays === 'number') minInput.value = rule.minAgeDays; + const maxInput = document.createElement('input'); + maxInput.type = 'number'; + maxInput.placeholder = 'Max days'; + maxInput.className = 'input is-small max-age'; + maxInput.style.width = '6em'; + if (typeof rule.maxAgeDays === 'number') maxInput.value = rule.maxAgeDays; + ageBox.appendChild(minInput); + ageBox.appendChild(maxInput); + + const acctBox = document.createElement('div'); + acctBox.className = 'field mt-2 is-hidden'; + const acctLabel = document.createElement('label'); + acctLabel.className = 'label'; + acctLabel.textContent = 'Accounts'; + const acctControl = document.createElement('div'); + const acctWrap = document.createElement('div'); + acctWrap.className = 'select is-multiple is-small'; + const acctSel = document.createElement('select'); + acctSel.className = 'account-select'; + acctSel.multiple = true; + acctSel.size = Math.min(accountList.length, 4) || 1; + for (const a of accountList) { + const opt = document.createElement('option'); + opt.value = a.id; + opt.textContent = a.name; + if ((rule.accounts || []).includes(a.id)) opt.selected = true; + acctSel.appendChild(opt); + } + acctWrap.appendChild(acctSel); + acctControl.appendChild(acctWrap); + acctBox.appendChild(acctLabel); + acctBox.appendChild(acctControl); + + const folderBox = document.createElement('div'); + folderBox.className = 'field mt-2 is-hidden'; + const folderLabel = document.createElement('label'); + folderLabel.className = 'label'; + folderLabel.textContent = 'Folders'; + const folderControl = document.createElement('div'); + const folderWrap = document.createElement('div'); + folderWrap.className = 'select is-multiple is-small'; + const folderSel = document.createElement('select'); + folderSel.className = 'folder-filter-select'; + folderSel.multiple = true; + folderSel.size = Math.min(folderList.length, 6) || 1; + for (const f of folderList) { + const opt = document.createElement('option'); + opt.value = f.id; + opt.textContent = f.name; + if ((rule.folders || []).includes(f.id)) opt.selected = true; + folderSel.appendChild(opt); + } + folderWrap.appendChild(folderSel); + folderControl.appendChild(folderWrap); + folderBox.appendChild(folderLabel); + folderBox.appendChild(folderControl); + + if (typeof rule.minAgeDays === 'number' || typeof rule.maxAgeDays === 'number') { + ageBox.classList.remove('is-hidden'); + } + if ((rule.accounts || []).length) { + acctBox.classList.remove('is-hidden'); + } + if ((rule.folders || []).length) { + folderBox.classList.remove('is-hidden'); + } + + const condButtons = document.createElement('div'); + condButtons.className = 'field is-grouped is-grouped-multiline mb-2'; + + function addCond(btn) { + const p = document.createElement('p'); + p.className = 'control'; + p.appendChild(btn); + condButtons.appendChild(p); + } + + addCond(createConditionButton('Stop', null, stopCheck, () => { + stopCheck.checked = false; + })); + addCond(createConditionButton('Unread', null, unreadCheck, () => { + unreadCheck.checked = false; + })); + addCond(createConditionButton('Age', ageBox, null, () => { + minInput.value = ''; + maxInput.value = ''; + })); + addCond(createConditionButton('Accounts', acctBox, null, () => { + for (const opt of acctSel.options) opt.selected = false; + })); + addCond(createConditionButton('Folders', folderBox, null, () => { + for (const opt of folderSel.options) opt.selected = false; + })); const body = document.createElement('div'); body.className = 'message-body'; body.appendChild(actionsContainer); body.appendChild(addAction); + const condDivider = document.createElement('hr'); + condDivider.className = 'mt-3 mb-2'; + body.appendChild(condDivider); + body.appendChild(condButtons); body.appendChild(stopLabel); + body.appendChild(unreadLabel); + body.appendChild(ageBox); + body.appendChild(acctBox); + body.appendChild(folderBox); article.appendChild(header); article.appendChild(body); @@ -280,38 +621,65 @@ document.addEventListener('DOMContentLoaded', async () => { if (type === 'move') { return { type, folder: row.querySelector('.folder-select').value }; } + if (type === 'copy') { + return { type, copyTarget: row.querySelector('.folder-select').value }; + } if (type === 'junk') { return { type, junk: row.querySelector('.junk-select').value === 'true' }; } + if (type === 'read') { + return { type, read: row.querySelector('.read-select').value === 'true' }; + } + if (type === 'flag') { + return { type, flagged: row.querySelector('.flag-select').value === 'true' }; + } + if (type === 'delete' || type === 'archive') { + return { type }; + } return { type }; }); const stopProcessing = ruleEl.querySelector('.stop-processing')?.checked; - return { criterion, actions, stopProcessing }; + const unreadOnly = ruleEl.querySelector('.unread-only')?.checked; + const enabled = ruleEl.dataset.enabled !== 'false'; + const minAgeDays = parseFloat(ruleEl.querySelector('.min-age')?.value); + const maxAgeDays = parseFloat(ruleEl.querySelector('.max-age')?.value); + const accounts = [...(ruleEl.querySelector('.account-select')?.selectedOptions || [])].map(o => o.value); + const folders = [...(ruleEl.querySelector('.folder-filter-select')?.selectedOptions || [])].map(o => o.value); + const rule = { criterion, actions, enabled }; + if (unreadOnly) rule.unreadOnly = true; + if (stopProcessing) rule.stopProcessing = true; + if (!isNaN(minAgeDays)) rule.minAgeDays = minAgeDays; + if (!isNaN(maxAgeDays)) rule.maxAgeDays = maxAgeDays; + if (accounts.length) rule.accounts = accounts; + if (folders.length) rule.folders = folders; + return rule; }); - data.push({ criterion: '', actions: [], stopProcessing: false }); + data.push({ criterion: '', actions: [], unreadOnly: false, stopProcessing: false, enabled: true, accounts: [], folders: [] }); renderRules(data); }); renderRules((defaults.aiRules || []).map(r => { - if (r.actions) return r; + if (r.actions) { + if (!Array.isArray(r.accounts)) r.accounts = []; + if (!Array.isArray(r.folders)) r.folders = []; + if (r.enabled !== false) r.enabled = true; else r.enabled = false; + return r; + } const actions = []; if (r.tag) actions.push({ type: 'tag', tagKey: r.tag }); if (r.moveTo) actions.push({ type: 'move', folder: r.moveTo }); + if (r.copyTarget || r.copyTo) actions.push({ type: 'copy', copyTarget: r.copyTarget || r.copyTo }); const rule = { criterion: r.criterion, actions }; if (r.stopProcessing) rule.stopProcessing = true; + if (r.unreadOnly) rule.unreadOnly = true; + if (typeof r.minAgeDays === 'number') rule.minAgeDays = r.minAgeDays; + if (typeof r.maxAgeDays === 'number') rule.maxAgeDays = r.maxAgeDays; + if (Array.isArray(r.accounts)) rule.accounts = r.accounts; + if (Array.isArray(r.folders)) rule.folders = r.folders; + rule.enabled = r.enabled !== false; return rule; })); - const ruleCountEl = document.getElementById('rule-count'); - const cacheCountEl = document.getElementById('cache-count'); - const queueCountEl = document.getElementById('queue-count'); - const currentTimeEl = document.getElementById('current-time'); - const lastTimeEl = document.getElementById('last-time'); - const averageTimeEl = document.getElementById('average-time'); - const totalTimeEl = document.getElementById('total-time'); - let timingLogged = false; - ruleCountEl.textContent = (defaults.aiRules || []).length; - cacheCountEl.textContent = defaults.aiCache ? Object.keys(defaults.aiCache).length : 0; function format(ms) { if (ms < 0) return '--:--:--'; @@ -352,6 +720,10 @@ document.addEventListener('DOMContentLoaded', async () => { } averageTimeEl.textContent = stats.runs > 0 ? format(stats.average) : '--:--:--'; totalTimeEl.textContent = format(stats.total); + const perHour = stats.average > 0 ? Math.round(3600000 / stats.average) : 0; + const perDay = stats.average > 0 ? Math.round(86400000 / stats.average) : 0; + perHourEl.textContent = perHour; + perDayEl.textContent = perDay; if (!timingLogged) { logger.aiLog('retrieved timing stats', {debug: true}); timingLogged = true; @@ -362,14 +734,48 @@ document.addEventListener('DOMContentLoaded', async () => { lastTimeEl.textContent = '--:--:--'; averageTimeEl.textContent = '--:--:--'; totalTimeEl.textContent = '--:--:--'; + perHourEl.textContent = '0'; + perDayEl.textContent = '0'; } - ruleCountEl.textContent = document.querySelectorAll('#rules-container .rule').length; try { - cacheCountEl.textContent = await AiClassifier.getCacheSize(); + const { aiCache } = await storage.local.get('aiCache'); + cacheCountEl.textContent = aiCache ? Object.keys(aiCache).length : 0; } catch { cacheCountEl.textContent = '?'; } + + try { + if (debugTabToggle.checked) { + const latest = await storage.local.get(['lastPayload', 'lastFullText', 'lastPromptText']); + const payloadStr = latest.lastPayload ? JSON.stringify(latest.lastPayload, null, 2) : ''; + if (payloadStr !== lastPayload) { + lastPayload = payloadStr; + payloadDisplay.textContent = payloadStr; + } + if (latest.lastFullText !== lastFullText || latest.lastPromptText !== lastPromptText) { + lastFullText = latest.lastFullText || ''; + lastPromptText = latest.lastPromptText || ''; + if (lastFullText && lastPromptText && diff_match_patch) { + const dmp = new diff_match_patch(); + dmp.Diff_EditCost = 4; + const diffs = dmp.diff_main(lastFullText, lastPromptText); + dmp.diff_cleanupEfficiency(diffs); + const hasDiff = diffs.some(d => d[0] !== 0); + if (hasDiff) { + diffDisplay.innerHTML = dmp.diff_prettyHtml(diffs); + diffContainer.classList.remove('is-hidden'); + } else { + diffDisplay.innerHTML = ''; + diffContainer.classList.add('is-hidden'); + } + } else { + diffDisplay.innerHTML = ''; + diffContainer.classList.add('is-hidden'); + } + } + } + } catch {} } refreshMaintenance(); @@ -379,6 +785,24 @@ document.addEventListener('DOMContentLoaded', async () => { await AiClassifier.clearCache(); cacheCountEl.textContent = '0'; }); + + function selectedCategories() { + return [...document.querySelectorAll('.transfer-category:checked')].map(el => el.value); + } + + document.getElementById('export-data').addEventListener('click', () => { + dataTransfer.exportData(selectedCategories()); + }); + + const importInput = document.getElementById('import-file'); + document.getElementById('import-data').addEventListener('click', () => importInput.click()); + importInput.addEventListener('change', async () => { + if (importInput.files.length) { + await dataTransfer.importData(importInput.files[0], selectedCategories()); + location.reload(); + } + }); + initialized = true; document.getElementById('save').addEventListener('click', async () => { @@ -395,6 +819,7 @@ document.addEventListener('DOMContentLoaded', async () => { } } const debugLogging = debugToggle.checked; + const htmlToMarkdown = htmlToggle.checked; const rules = [...rulesContainer.querySelectorAll('.rule')].map(ruleEl => { const criterion = ruleEl.querySelector('.criterion').value; const actions = [...ruleEl.querySelectorAll('.action-row')].map(row => { @@ -405,15 +830,50 @@ document.addEventListener('DOMContentLoaded', async () => { if (type === 'move') { return { type, folder: row.querySelector('.folder-select').value }; } + if (type === 'copy') { + return { type, copyTarget: row.querySelector('.folder-select').value }; + } if (type === 'junk') { return { type, junk: row.querySelector('.junk-select').value === 'true' }; } + if (type === 'read') { + return { type, read: row.querySelector('.read-select').value === 'true' }; + } + if (type === 'flag') { + return { type, flagged: row.querySelector('.flag-select').value === 'true' }; + } + if (type === 'forward') { + return { type, address: row.querySelector('.forward-input').value.trim() }; + } + if (type === 'reply') { + return { type, replyType: row.querySelector('.reply-select').value }; + } return { type }; }); const stopProcessing = ruleEl.querySelector('.stop-processing')?.checked; - return { criterion, actions, stopProcessing }; + const unreadOnly = ruleEl.querySelector('.unread-only')?.checked; + const enabled = ruleEl.dataset.enabled !== 'false'; + const minAgeDays = parseFloat(ruleEl.querySelector('.min-age')?.value); + const maxAgeDays = parseFloat(ruleEl.querySelector('.max-age')?.value); + const accounts = [...(ruleEl.querySelector('.account-select')?.selectedOptions || [])].map(o => o.value); + const folders = [...(ruleEl.querySelector('.folder-filter-select')?.selectedOptions || [])].map(o => o.value); + const rule = { criterion, actions, enabled }; + if (unreadOnly) rule.unreadOnly = true; + if (stopProcessing) rule.stopProcessing = true; + if (!isNaN(minAgeDays)) rule.minAgeDays = minAgeDays; + if (!isNaN(maxAgeDays)) rule.maxAgeDays = maxAgeDays; + if (accounts.length) rule.accounts = accounts; + if (folders.length) rule.folders = folders; + return rule; }).filter(r => r.criterion); - await storage.local.set({ endpoint, templateName, customTemplate: customTemplateText, customSystemPrompt, aiParams: aiParamsSave, debugLogging, aiRules: rules }); + const stripUrlParams = stripUrlToggle.checked; + const altTextImages = altTextToggle.checked; + const collapseWhitespace = collapseWhitespaceToggle.checked; + const tokenReduction = tokenReductionToggle.checked; + const showDebugTab = debugTabToggle.checked; + const theme = themeSelect.value; + await storage.local.set({ endpoint, templateName, customTemplate: customTemplateText, customSystemPrompt, aiParams: aiParamsSave, debugLogging, htmlToMarkdown, stripUrlParams, altTextImages, collapseWhitespace, tokenReduction, aiRules: rules, theme, showDebugTab }); + await applyTheme(theme); try { await AiClassifier.setConfig({ endpoint, templateName, customTemplate: customTemplateText, customSystemPrompt, aiParams: aiParamsSave, debugLogging }); logger.setDebug(debugLogging); diff --git a/resources/img/average-dark-16.png b/resources/img/average-dark-16.png new file mode 100644 index 0000000..42ccab5 Binary files /dev/null and b/resources/img/average-dark-16.png differ diff --git a/resources/img/average-dark-32.png b/resources/img/average-dark-32.png new file mode 100644 index 0000000..3eb264e Binary files /dev/null and b/resources/img/average-dark-32.png differ diff --git a/resources/img/average-dark-64.png b/resources/img/average-dark-64.png new file mode 100644 index 0000000..1174698 Binary files /dev/null and b/resources/img/average-dark-64.png differ diff --git a/resources/img/average-light-16.png b/resources/img/average-light-16.png new file mode 100644 index 0000000..ff202ff Binary files /dev/null and b/resources/img/average-light-16.png differ diff --git a/resources/img/average-light-32.png b/resources/img/average-light-32.png new file mode 100644 index 0000000..e0fe4b4 Binary files /dev/null and b/resources/img/average-light-32.png differ diff --git a/resources/img/average-light-64.png b/resources/img/average-light-64.png new file mode 100644 index 0000000..bec89df Binary files /dev/null and b/resources/img/average-light-64.png differ diff --git a/resources/img/brain.png b/resources/img/brain.png deleted file mode 100644 index 296bb64..0000000 Binary files a/resources/img/brain.png and /dev/null differ diff --git a/resources/img/busy.png b/resources/img/busy.png deleted file mode 100644 index ee7d5d6..0000000 Binary files a/resources/img/busy.png and /dev/null differ diff --git a/resources/img/check-dark-16.png b/resources/img/check-dark-16.png new file mode 100644 index 0000000..37d8ccc Binary files /dev/null and b/resources/img/check-dark-16.png differ diff --git a/resources/img/check-dark-32.png b/resources/img/check-dark-32.png new file mode 100644 index 0000000..7c2e09c Binary files /dev/null and b/resources/img/check-dark-32.png differ diff --git a/resources/img/check-dark-64.png b/resources/img/check-dark-64.png new file mode 100644 index 0000000..4c9fb58 Binary files /dev/null and b/resources/img/check-dark-64.png differ diff --git a/resources/img/check-light-16.png b/resources/img/check-light-16.png new file mode 100644 index 0000000..2a82980 Binary files /dev/null and b/resources/img/check-light-16.png differ diff --git a/resources/img/check-light-32.png b/resources/img/check-light-32.png new file mode 100644 index 0000000..92e14fd Binary files /dev/null and b/resources/img/check-light-32.png differ diff --git a/resources/img/check-light-64.png b/resources/img/check-light-64.png new file mode 100644 index 0000000..175c4a2 Binary files /dev/null and b/resources/img/check-light-64.png differ diff --git a/resources/img/circle-dark-16.png b/resources/img/circle-dark-16.png new file mode 100644 index 0000000..87a8e74 Binary files /dev/null and b/resources/img/circle-dark-16.png differ diff --git a/resources/img/circle-dark-32.png b/resources/img/circle-dark-32.png new file mode 100644 index 0000000..bc13139 Binary files /dev/null and b/resources/img/circle-dark-32.png differ diff --git a/resources/img/circle-dark-64.png b/resources/img/circle-dark-64.png new file mode 100644 index 0000000..8de9fee Binary files /dev/null and b/resources/img/circle-dark-64.png differ diff --git a/resources/img/circle-light-16.png b/resources/img/circle-light-16.png new file mode 100644 index 0000000..9984541 Binary files /dev/null and b/resources/img/circle-light-16.png differ diff --git a/resources/img/circle-light-32.png b/resources/img/circle-light-32.png new file mode 100644 index 0000000..6023c0e Binary files /dev/null and b/resources/img/circle-light-32.png differ diff --git a/resources/img/circle-light-64.png b/resources/img/circle-light-64.png new file mode 100644 index 0000000..5381c54 Binary files /dev/null and b/resources/img/circle-light-64.png differ diff --git a/resources/img/circledot-dark-16.png b/resources/img/circledot-dark-16.png new file mode 100644 index 0000000..db602ab Binary files /dev/null and b/resources/img/circledot-dark-16.png differ diff --git a/resources/img/circledot-dark-32.png b/resources/img/circledot-dark-32.png new file mode 100644 index 0000000..d29ac16 Binary files /dev/null and b/resources/img/circledot-dark-32.png differ diff --git a/resources/img/circledot-dark-64.png b/resources/img/circledot-dark-64.png new file mode 100644 index 0000000..f29bb77 Binary files /dev/null and b/resources/img/circledot-dark-64.png differ diff --git a/resources/img/circledot-light-16.png b/resources/img/circledot-light-16.png new file mode 100644 index 0000000..4809c5a Binary files /dev/null and b/resources/img/circledot-light-16.png differ diff --git a/resources/img/circledot-light-32.png b/resources/img/circledot-light-32.png new file mode 100644 index 0000000..06c342f Binary files /dev/null and b/resources/img/circledot-light-32.png differ diff --git a/resources/img/circledot-light-64.png b/resources/img/circledot-light-64.png new file mode 100644 index 0000000..cb8634a Binary files /dev/null and b/resources/img/circledot-light-64.png differ diff --git a/resources/img/circledots-dark-16.png b/resources/img/circledots-dark-16.png new file mode 100644 index 0000000..90f8db9 Binary files /dev/null and b/resources/img/circledots-dark-16.png differ diff --git a/resources/img/circledots-dark-32.png b/resources/img/circledots-dark-32.png new file mode 100644 index 0000000..69c6218 Binary files /dev/null and b/resources/img/circledots-dark-32.png differ diff --git a/resources/img/circledots-dark-64.png b/resources/img/circledots-dark-64.png new file mode 100644 index 0000000..bddee31 Binary files /dev/null and b/resources/img/circledots-dark-64.png differ diff --git a/resources/img/circledots-light-16.png b/resources/img/circledots-light-16.png new file mode 100644 index 0000000..1443d22 Binary files /dev/null and b/resources/img/circledots-light-16.png differ diff --git a/resources/img/circledots-light-32.png b/resources/img/circledots-light-32.png new file mode 100644 index 0000000..e8dfa44 Binary files /dev/null and b/resources/img/circledots-light-32.png differ diff --git a/resources/img/circledots-light-64.png b/resources/img/circledots-light-64.png new file mode 100644 index 0000000..a78e768 Binary files /dev/null and b/resources/img/circledots-light-64.png differ diff --git a/resources/img/circleslash-dark-16.png b/resources/img/circleslash-dark-16.png new file mode 100644 index 0000000..a7057de Binary files /dev/null and b/resources/img/circleslash-dark-16.png differ diff --git a/resources/img/circleslash-dark-32.png b/resources/img/circleslash-dark-32.png new file mode 100644 index 0000000..1d34f01 Binary files /dev/null and b/resources/img/circleslash-dark-32.png differ diff --git a/resources/img/circleslash-dark-64.png b/resources/img/circleslash-dark-64.png new file mode 100644 index 0000000..cc27212 Binary files /dev/null and b/resources/img/circleslash-dark-64.png differ diff --git a/resources/img/circleslash-light-16.png b/resources/img/circleslash-light-16.png new file mode 100644 index 0000000..51c8c3f Binary files /dev/null and b/resources/img/circleslash-light-16.png differ diff --git a/resources/img/circleslash-light-32.png b/resources/img/circleslash-light-32.png new file mode 100644 index 0000000..a5cddaf Binary files /dev/null and b/resources/img/circleslash-light-32.png differ diff --git a/resources/img/circleslash-light-64.png b/resources/img/circleslash-light-64.png new file mode 100644 index 0000000..b348295 Binary files /dev/null and b/resources/img/circleslash-light-64.png differ diff --git a/resources/img/clipboarddata-dark-16.png b/resources/img/clipboarddata-dark-16.png new file mode 100644 index 0000000..f3fecfc Binary files /dev/null and b/resources/img/clipboarddata-dark-16.png differ diff --git a/resources/img/clipboarddata-dark-32.png b/resources/img/clipboarddata-dark-32.png new file mode 100644 index 0000000..ce74129 Binary files /dev/null and b/resources/img/clipboarddata-dark-32.png differ diff --git a/resources/img/clipboarddata-dark-64.png b/resources/img/clipboarddata-dark-64.png new file mode 100644 index 0000000..37e7245 Binary files /dev/null and b/resources/img/clipboarddata-dark-64.png differ diff --git a/resources/img/clipboarddata-light-16.png b/resources/img/clipboarddata-light-16.png new file mode 100644 index 0000000..6d615ef Binary files /dev/null and b/resources/img/clipboarddata-light-16.png differ diff --git a/resources/img/clipboarddata-light-32.png b/resources/img/clipboarddata-light-32.png new file mode 100644 index 0000000..a426878 Binary files /dev/null and b/resources/img/clipboarddata-light-32.png differ diff --git a/resources/img/clipboarddata-light-64.png b/resources/img/clipboarddata-light-64.png new file mode 100644 index 0000000..9d945d0 Binary files /dev/null and b/resources/img/clipboarddata-light-64.png differ diff --git a/resources/img/done.png b/resources/img/done.png deleted file mode 100644 index b70d728..0000000 Binary files a/resources/img/done.png and /dev/null differ diff --git a/resources/img/download-dark-16.png b/resources/img/download-dark-16.png new file mode 100644 index 0000000..7e926bb Binary files /dev/null and b/resources/img/download-dark-16.png differ diff --git a/resources/img/download-dark-32.png b/resources/img/download-dark-32.png new file mode 100644 index 0000000..1b92afc Binary files /dev/null and b/resources/img/download-dark-32.png differ diff --git a/resources/img/download-dark-64.png b/resources/img/download-dark-64.png new file mode 100644 index 0000000..62ac850 Binary files /dev/null and b/resources/img/download-dark-64.png differ diff --git a/resources/img/download-light-16.png b/resources/img/download-light-16.png new file mode 100644 index 0000000..620f868 Binary files /dev/null and b/resources/img/download-light-16.png differ diff --git a/resources/img/download-light-32.png b/resources/img/download-light-32.png new file mode 100644 index 0000000..840c82b Binary files /dev/null and b/resources/img/download-light-32.png differ diff --git a/resources/img/download-light-64.png b/resources/img/download-light-64.png new file mode 100644 index 0000000..67aa68d Binary files /dev/null and b/resources/img/download-light-64.png differ diff --git a/resources/img/error.png b/resources/img/error.png deleted file mode 100644 index 07705d9..0000000 Binary files a/resources/img/error.png and /dev/null differ diff --git a/resources/img/eye-dark-16.png b/resources/img/eye-dark-16.png new file mode 100644 index 0000000..a592ac5 Binary files /dev/null and b/resources/img/eye-dark-16.png differ diff --git a/resources/img/eye-dark-32.png b/resources/img/eye-dark-32.png new file mode 100644 index 0000000..6aceb84 Binary files /dev/null and b/resources/img/eye-dark-32.png differ diff --git a/resources/img/eye-dark-64.png b/resources/img/eye-dark-64.png new file mode 100644 index 0000000..539fe6f Binary files /dev/null and b/resources/img/eye-dark-64.png differ diff --git a/resources/img/eye-light-16.png b/resources/img/eye-light-16.png new file mode 100644 index 0000000..392340e Binary files /dev/null and b/resources/img/eye-light-16.png differ diff --git a/resources/img/eye-light-32.png b/resources/img/eye-light-32.png new file mode 100644 index 0000000..5531c1a Binary files /dev/null and b/resources/img/eye-light-32.png differ diff --git a/resources/img/eye-light-64.png b/resources/img/eye-light-64.png new file mode 100644 index 0000000..7d7fbd2 Binary files /dev/null and b/resources/img/eye-light-64.png differ diff --git a/resources/img/flag-dark-16.png b/resources/img/flag-dark-16.png new file mode 100644 index 0000000..4477f82 Binary files /dev/null and b/resources/img/flag-dark-16.png differ diff --git a/resources/img/flag-dark-32.png b/resources/img/flag-dark-32.png new file mode 100644 index 0000000..ecba78a Binary files /dev/null and b/resources/img/flag-dark-32.png differ diff --git a/resources/img/flag-dark-64.png b/resources/img/flag-dark-64.png new file mode 100644 index 0000000..25f44d4 Binary files /dev/null and b/resources/img/flag-dark-64.png differ diff --git a/resources/img/flag-light-16.png b/resources/img/flag-light-16.png new file mode 100644 index 0000000..3c2b58c Binary files /dev/null and b/resources/img/flag-light-16.png differ diff --git a/resources/img/flag-light-32.png b/resources/img/flag-light-32.png new file mode 100644 index 0000000..531dee2 Binary files /dev/null and b/resources/img/flag-light-32.png differ diff --git a/resources/img/flag-light-64.png b/resources/img/flag-light-64.png new file mode 100644 index 0000000..6446415 Binary files /dev/null and b/resources/img/flag-light-64.png differ diff --git a/resources/img/gear-dark-16.png b/resources/img/gear-dark-16.png new file mode 100644 index 0000000..b39b5ac Binary files /dev/null and b/resources/img/gear-dark-16.png differ diff --git a/resources/img/gear-dark-32.png b/resources/img/gear-dark-32.png new file mode 100644 index 0000000..cdfb442 Binary files /dev/null and b/resources/img/gear-dark-32.png differ diff --git a/resources/img/gear-dark-64.png b/resources/img/gear-dark-64.png new file mode 100644 index 0000000..743040e Binary files /dev/null and b/resources/img/gear-dark-64.png differ diff --git a/resources/img/gear-light-16.png b/resources/img/gear-light-16.png new file mode 100644 index 0000000..3e1839f Binary files /dev/null and b/resources/img/gear-light-16.png differ diff --git a/resources/img/gear-light-32.png b/resources/img/gear-light-32.png new file mode 100644 index 0000000..67300de Binary files /dev/null and b/resources/img/gear-light-32.png differ diff --git a/resources/img/gear-light-64.png b/resources/img/gear-light-64.png new file mode 100644 index 0000000..99c8a68 Binary files /dev/null and b/resources/img/gear-light-64.png differ diff --git a/resources/img/plus-dark-16.png b/resources/img/plus-dark-16.png new file mode 100644 index 0000000..89cfee9 Binary files /dev/null and b/resources/img/plus-dark-16.png differ diff --git a/resources/img/plus-dark-32.png b/resources/img/plus-dark-32.png new file mode 100644 index 0000000..fdb1a30 Binary files /dev/null and b/resources/img/plus-dark-32.png differ diff --git a/resources/img/plus-dark-64.png b/resources/img/plus-dark-64.png new file mode 100644 index 0000000..957956e Binary files /dev/null and b/resources/img/plus-dark-64.png differ diff --git a/resources/img/plus-light-16.png b/resources/img/plus-light-16.png new file mode 100644 index 0000000..dba6721 Binary files /dev/null and b/resources/img/plus-light-16.png differ diff --git a/resources/img/plus-light-32.png b/resources/img/plus-light-32.png new file mode 100644 index 0000000..337833b Binary files /dev/null and b/resources/img/plus-light-32.png differ diff --git a/resources/img/plus-light-64.png b/resources/img/plus-light-64.png new file mode 100644 index 0000000..4c0d1a8 Binary files /dev/null and b/resources/img/plus-light-64.png differ diff --git a/resources/img/reply-dark-16.png b/resources/img/reply-dark-16.png new file mode 100644 index 0000000..d500c54 Binary files /dev/null and b/resources/img/reply-dark-16.png differ diff --git a/resources/img/reply-dark-32.png b/resources/img/reply-dark-32.png new file mode 100644 index 0000000..ef976ca Binary files /dev/null and b/resources/img/reply-dark-32.png differ diff --git a/resources/img/reply-dark-64.png b/resources/img/reply-dark-64.png new file mode 100644 index 0000000..3cdb535 Binary files /dev/null and b/resources/img/reply-dark-64.png differ diff --git a/resources/img/reply-light-16.png b/resources/img/reply-light-16.png new file mode 100644 index 0000000..abda5b3 Binary files /dev/null and b/resources/img/reply-light-16.png differ diff --git a/resources/img/reply-light-32.png b/resources/img/reply-light-32.png new file mode 100644 index 0000000..193deac Binary files /dev/null and b/resources/img/reply-light-32.png differ diff --git a/resources/img/reply-light-64.png b/resources/img/reply-light-64.png new file mode 100644 index 0000000..d241386 Binary files /dev/null and b/resources/img/reply-light-64.png differ diff --git a/resources/img/settings-dark-16.png b/resources/img/settings-dark-16.png new file mode 100644 index 0000000..26be451 Binary files /dev/null and b/resources/img/settings-dark-16.png differ diff --git a/resources/img/settings-dark-32.png b/resources/img/settings-dark-32.png new file mode 100644 index 0000000..c9a7650 Binary files /dev/null and b/resources/img/settings-dark-32.png differ diff --git a/resources/img/settings-dark-64.png b/resources/img/settings-dark-64.png new file mode 100644 index 0000000..90e8bad Binary files /dev/null and b/resources/img/settings-dark-64.png differ diff --git a/resources/img/settings-light-16.png b/resources/img/settings-light-16.png new file mode 100644 index 0000000..bb7581d Binary files /dev/null and b/resources/img/settings-light-16.png differ diff --git a/resources/img/settings-light-32.png b/resources/img/settings-light-32.png new file mode 100644 index 0000000..cc059a4 Binary files /dev/null and b/resources/img/settings-light-32.png differ diff --git a/resources/img/settings-light-64.png b/resources/img/settings-light-64.png new file mode 100644 index 0000000..347d454 Binary files /dev/null and b/resources/img/settings-light-64.png differ diff --git a/resources/img/trash-dark-16.png b/resources/img/trash-dark-16.png new file mode 100644 index 0000000..58c4931 Binary files /dev/null and b/resources/img/trash-dark-16.png differ diff --git a/resources/img/trash-dark-32.png b/resources/img/trash-dark-32.png new file mode 100644 index 0000000..b76e1e2 Binary files /dev/null and b/resources/img/trash-dark-32.png differ diff --git a/resources/img/trash-dark-64.png b/resources/img/trash-dark-64.png new file mode 100644 index 0000000..c8fe013 Binary files /dev/null and b/resources/img/trash-dark-64.png differ diff --git a/resources/img/trash-light-16.png b/resources/img/trash-light-16.png new file mode 100644 index 0000000..43e1f7f Binary files /dev/null and b/resources/img/trash-light-16.png differ diff --git a/resources/img/trash-light-32.png b/resources/img/trash-light-32.png new file mode 100644 index 0000000..1968ed5 Binary files /dev/null and b/resources/img/trash-light-32.png differ diff --git a/resources/img/trash-light-64.png b/resources/img/trash-light-64.png new file mode 100644 index 0000000..bbbdd2d Binary files /dev/null and b/resources/img/trash-light-64.png differ diff --git a/resources/img/upload-dark-16.png b/resources/img/upload-dark-16.png new file mode 100644 index 0000000..6aae639 Binary files /dev/null and b/resources/img/upload-dark-16.png differ diff --git a/resources/img/upload-dark-32.png b/resources/img/upload-dark-32.png new file mode 100644 index 0000000..06556e1 Binary files /dev/null and b/resources/img/upload-dark-32.png differ diff --git a/resources/img/upload-dark-64.png b/resources/img/upload-dark-64.png new file mode 100644 index 0000000..3314ff1 Binary files /dev/null and b/resources/img/upload-dark-64.png differ diff --git a/resources/img/upload-light-16.png b/resources/img/upload-light-16.png new file mode 100644 index 0000000..408364c Binary files /dev/null and b/resources/img/upload-light-16.png differ diff --git a/resources/img/upload-light-32.png b/resources/img/upload-light-32.png new file mode 100644 index 0000000..e6efe06 Binary files /dev/null and b/resources/img/upload-light-32.png differ diff --git a/resources/img/upload-light-64.png b/resources/img/upload-light-64.png new file mode 100644 index 0000000..65629df Binary files /dev/null and b/resources/img/upload-light-64.png differ diff --git a/resources/img/x-dark-16.png b/resources/img/x-dark-16.png new file mode 100644 index 0000000..bdb9bbd Binary files /dev/null and b/resources/img/x-dark-16.png differ diff --git a/resources/img/x-dark-32.png b/resources/img/x-dark-32.png new file mode 100644 index 0000000..8f5c10e Binary files /dev/null and b/resources/img/x-dark-32.png differ diff --git a/resources/img/x-dark-64.png b/resources/img/x-dark-64.png new file mode 100644 index 0000000..00e8c35 Binary files /dev/null and b/resources/img/x-dark-64.png differ diff --git a/resources/img/x-light-16.png b/resources/img/x-light-16.png new file mode 100644 index 0000000..7ce54a5 Binary files /dev/null and b/resources/img/x-light-16.png differ diff --git a/resources/img/x-light-32.png b/resources/img/x-light-32.png new file mode 100644 index 0000000..785d7be Binary files /dev/null and b/resources/img/x-light-32.png differ diff --git a/resources/img/x-light-64.png b/resources/img/x-light-64.png new file mode 100644 index 0000000..37dce1a Binary files /dev/null and b/resources/img/x-light-64.png differ diff --git a/resources/js/diff_match_patch_uncompressed.js b/resources/js/diff_match_patch_uncompressed.js new file mode 100644 index 0000000..88a702c --- /dev/null +++ b/resources/js/diff_match_patch_uncompressed.js @@ -0,0 +1,2236 @@ +/** + * Diff Match and Patch + * Copyright 2018 The diff-match-patch Authors. + * https://github.com/google/diff-match-patch + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * @fileoverview Computes the difference between two texts to create a patch. + * Applies the patch onto another text, allowing for errors. + * @author fraser@google.com (Neil Fraser) + */ + +/** + * Class containing the diff, match and patch methods. + * @constructor + */ +var diff_match_patch = function() { + + // Defaults. + // Redefine these in your program to override the defaults. + + // Number of seconds to map a diff before giving up (0 for infinity). + this.Diff_Timeout = 1.0; + // Cost of an empty edit operation in terms of edit characters. + this.Diff_EditCost = 4; + // At what point is no match declared (0.0 = perfection, 1.0 = very loose). + this.Match_Threshold = 0.5; + // How far to search for a match (0 = exact location, 1000+ = broad match). + // A match this many characters away from the expected location will add + // 1.0 to the score (0.0 is a perfect match). + this.Match_Distance = 1000; + // When deleting a large block of text (over ~64 characters), how close do + // the contents have to be to match the expected contents. (0.0 = perfection, + // 1.0 = very loose). Note that Match_Threshold controls how closely the + // end points of a delete need to match. + this.Patch_DeleteThreshold = 0.5; + // Chunk size for context length. + this.Patch_Margin = 4; + + // The number of bits in an int. + this.Match_MaxBits = 32; +}; + + +// DIFF FUNCTIONS + + +/** + * The data structure representing a diff is an array of tuples: + * [[DIFF_DELETE, 'Hello'], [DIFF_INSERT, 'Goodbye'], [DIFF_EQUAL, ' world.']] + * which means: delete 'Hello', add 'Goodbye' and keep ' world.' + */ +var DIFF_DELETE = -1; +var DIFF_INSERT = 1; +var DIFF_EQUAL = 0; + +/** + * Class representing one diff tuple. + * Attempts to look like a two-element array (which is what this used to be). + * @param {number} op Operation, one of: DIFF_DELETE, DIFF_INSERT, DIFF_EQUAL. + * @param {string} text Text to be deleted, inserted, or retained. + * @constructor + */ +diff_match_patch.Diff = function(op, text) { + this[0] = op; + this[1] = text; +}; + +diff_match_patch.Diff.prototype.length = 2; + +/** + * Emulate the output of a two-element array. + * @return {string} Diff operation as a string. + */ +diff_match_patch.Diff.prototype.toString = function() { + return this[0] + ',' + this[1]; +}; + + +/** + * Find the differences between two texts. Simplifies the problem by stripping + * any common prefix or suffix off the texts before diffing. + * @param {string} text1 Old string to be diffed. + * @param {string} text2 New string to be diffed. + * @param {boolean=} opt_checklines Optional speedup flag. If present and false, + * then don't run a line-level diff first to identify the changed areas. + * Defaults to true, which does a faster, slightly less optimal diff. + * @param {number=} opt_deadline Optional time when the diff should be complete + * by. Used internally for recursive calls. Users should set DiffTimeout + * instead. + * @return {!Array.} Array of diff tuples. + */ +diff_match_patch.prototype.diff_main = function(text1, text2, opt_checklines, + opt_deadline) { + // Set a deadline by which time the diff must be complete. + if (typeof opt_deadline == 'undefined') { + if (this.Diff_Timeout <= 0) { + opt_deadline = Number.MAX_VALUE; + } else { + opt_deadline = (new Date).getTime() + this.Diff_Timeout * 1000; + } + } + var deadline = opt_deadline; + + // Check for null inputs. + if (text1 == null || text2 == null) { + throw new Error('Null input. (diff_main)'); + } + + // Check for equality (speedup). + if (text1 == text2) { + if (text1) { + return [new diff_match_patch.Diff(DIFF_EQUAL, text1)]; + } + return []; + } + + if (typeof opt_checklines == 'undefined') { + opt_checklines = true; + } + var checklines = opt_checklines; + + // Trim off common prefix (speedup). + var commonlength = this.diff_commonPrefix(text1, text2); + var commonprefix = text1.substring(0, commonlength); + text1 = text1.substring(commonlength); + text2 = text2.substring(commonlength); + + // Trim off common suffix (speedup). + commonlength = this.diff_commonSuffix(text1, text2); + var commonsuffix = text1.substring(text1.length - commonlength); + text1 = text1.substring(0, text1.length - commonlength); + text2 = text2.substring(0, text2.length - commonlength); + + // Compute the diff on the middle block. + var diffs = this.diff_compute_(text1, text2, checklines, deadline); + + // Restore the prefix and suffix. + if (commonprefix) { + diffs.unshift(new diff_match_patch.Diff(DIFF_EQUAL, commonprefix)); + } + if (commonsuffix) { + diffs.push(new diff_match_patch.Diff(DIFF_EQUAL, commonsuffix)); + } + this.diff_cleanupMerge(diffs); + return diffs; +}; + + +/** + * Find the differences between two texts. Assumes that the texts do not + * have any common prefix or suffix. + * @param {string} text1 Old string to be diffed. + * @param {string} text2 New string to be diffed. + * @param {boolean} checklines Speedup flag. If false, then don't run a + * line-level diff first to identify the changed areas. + * If true, then run a faster, slightly less optimal diff. + * @param {number} deadline Time when the diff should be complete by. + * @return {!Array.} Array of diff tuples. + * @private + */ +diff_match_patch.prototype.diff_compute_ = function(text1, text2, checklines, + deadline) { + var diffs; + + if (!text1) { + // Just add some text (speedup). + return [new diff_match_patch.Diff(DIFF_INSERT, text2)]; + } + + if (!text2) { + // Just delete some text (speedup). + return [new diff_match_patch.Diff(DIFF_DELETE, text1)]; + } + + var longtext = text1.length > text2.length ? text1 : text2; + var shorttext = text1.length > text2.length ? text2 : text1; + var i = longtext.indexOf(shorttext); + if (i != -1) { + // Shorter text is inside the longer text (speedup). + diffs = [new diff_match_patch.Diff(DIFF_INSERT, longtext.substring(0, i)), + new diff_match_patch.Diff(DIFF_EQUAL, shorttext), + new diff_match_patch.Diff(DIFF_INSERT, + longtext.substring(i + shorttext.length))]; + // Swap insertions for deletions if diff is reversed. + if (text1.length > text2.length) { + diffs[0][0] = diffs[2][0] = DIFF_DELETE; + } + return diffs; + } + + if (shorttext.length == 1) { + // Single character string. + // After the previous speedup, the character can't be an equality. + return [new diff_match_patch.Diff(DIFF_DELETE, text1), + new diff_match_patch.Diff(DIFF_INSERT, text2)]; + } + + // Check to see if the problem can be split in two. + var hm = this.diff_halfMatch_(text1, text2); + if (hm) { + // A half-match was found, sort out the return data. + var text1_a = hm[0]; + var text1_b = hm[1]; + var text2_a = hm[2]; + var text2_b = hm[3]; + var mid_common = hm[4]; + // Send both pairs off for separate processing. + var diffs_a = this.diff_main(text1_a, text2_a, checklines, deadline); + var diffs_b = this.diff_main(text1_b, text2_b, checklines, deadline); + // Merge the results. + return diffs_a.concat([new diff_match_patch.Diff(DIFF_EQUAL, mid_common)], + diffs_b); + } + + if (checklines && text1.length > 100 && text2.length > 100) { + return this.diff_lineMode_(text1, text2, deadline); + } + + return this.diff_bisect_(text1, text2, deadline); +}; + + +/** + * Do a quick line-level diff on both strings, then rediff the parts for + * greater accuracy. + * This speedup can produce non-minimal diffs. + * @param {string} text1 Old string to be diffed. + * @param {string} text2 New string to be diffed. + * @param {number} deadline Time when the diff should be complete by. + * @return {!Array.} Array of diff tuples. + * @private + */ +diff_match_patch.prototype.diff_lineMode_ = function(text1, text2, deadline) { + // Scan the text on a line-by-line basis first. + var a = this.diff_linesToChars_(text1, text2); + text1 = a.chars1; + text2 = a.chars2; + var linearray = a.lineArray; + + var diffs = this.diff_main(text1, text2, false, deadline); + + // Convert the diff back to original text. + this.diff_charsToLines_(diffs, linearray); + // Eliminate freak matches (e.g. blank lines) + this.diff_cleanupSemantic(diffs); + + // Rediff any replacement blocks, this time character-by-character. + // Add a dummy entry at the end. + diffs.push(new diff_match_patch.Diff(DIFF_EQUAL, '')); + var pointer = 0; + var count_delete = 0; + var count_insert = 0; + var text_delete = ''; + var text_insert = ''; + while (pointer < diffs.length) { + switch (diffs[pointer][0]) { + case DIFF_INSERT: + count_insert++; + text_insert += diffs[pointer][1]; + break; + case DIFF_DELETE: + count_delete++; + text_delete += diffs[pointer][1]; + break; + case DIFF_EQUAL: + // Upon reaching an equality, check for prior redundancies. + if (count_delete >= 1 && count_insert >= 1) { + // Delete the offending records and add the merged ones. + diffs.splice(pointer - count_delete - count_insert, + count_delete + count_insert); + pointer = pointer - count_delete - count_insert; + var subDiff = + this.diff_main(text_delete, text_insert, false, deadline); + for (var j = subDiff.length - 1; j >= 0; j--) { + diffs.splice(pointer, 0, subDiff[j]); + } + pointer = pointer + subDiff.length; + } + count_insert = 0; + count_delete = 0; + text_delete = ''; + text_insert = ''; + break; + } + pointer++; + } + diffs.pop(); // Remove the dummy entry at the end. + + return diffs; +}; + + +/** + * Find the 'middle snake' of a diff, split the problem in two + * and return the recursively constructed diff. + * See Myers 1986 paper: An O(ND) Difference Algorithm and Its Variations. + * @param {string} text1 Old string to be diffed. + * @param {string} text2 New string to be diffed. + * @param {number} deadline Time at which to bail if not yet complete. + * @return {!Array.} Array of diff tuples. + * @private + */ +diff_match_patch.prototype.diff_bisect_ = function(text1, text2, deadline) { + // Cache the text lengths to prevent multiple calls. + var text1_length = text1.length; + var text2_length = text2.length; + var max_d = Math.ceil((text1_length + text2_length) / 2); + var v_offset = max_d; + var v_length = 2 * max_d; + var v1 = new Array(v_length); + var v2 = new Array(v_length); + // Setting all elements to -1 is faster in Chrome & Firefox than mixing + // integers and undefined. + for (var x = 0; x < v_length; x++) { + v1[x] = -1; + v2[x] = -1; + } + v1[v_offset + 1] = 0; + v2[v_offset + 1] = 0; + var delta = text1_length - text2_length; + // If the total number of characters is odd, then the front path will collide + // with the reverse path. + var front = (delta % 2 != 0); + // Offsets for start and end of k loop. + // Prevents mapping of space beyond the grid. + var k1start = 0; + var k1end = 0; + var k2start = 0; + var k2end = 0; + for (var d = 0; d < max_d; d++) { + // Bail out if deadline is reached. + if ((new Date()).getTime() > deadline) { + break; + } + + // Walk the front path one step. + for (var k1 = -d + k1start; k1 <= d - k1end; k1 += 2) { + var k1_offset = v_offset + k1; + var x1; + if (k1 == -d || (k1 != d && v1[k1_offset - 1] < v1[k1_offset + 1])) { + x1 = v1[k1_offset + 1]; + } else { + x1 = v1[k1_offset - 1] + 1; + } + var y1 = x1 - k1; + while (x1 < text1_length && y1 < text2_length && + text1.charAt(x1) == text2.charAt(y1)) { + x1++; + y1++; + } + v1[k1_offset] = x1; + if (x1 > text1_length) { + // Ran off the right of the graph. + k1end += 2; + } else if (y1 > text2_length) { + // Ran off the bottom of the graph. + k1start += 2; + } else if (front) { + var k2_offset = v_offset + delta - k1; + if (k2_offset >= 0 && k2_offset < v_length && v2[k2_offset] != -1) { + // Mirror x2 onto top-left coordinate system. + var x2 = text1_length - v2[k2_offset]; + if (x1 >= x2) { + // Overlap detected. + return this.diff_bisectSplit_(text1, text2, x1, y1, deadline); + } + } + } + } + + // Walk the reverse path one step. + for (var k2 = -d + k2start; k2 <= d - k2end; k2 += 2) { + var k2_offset = v_offset + k2; + var x2; + if (k2 == -d || (k2 != d && v2[k2_offset - 1] < v2[k2_offset + 1])) { + x2 = v2[k2_offset + 1]; + } else { + x2 = v2[k2_offset - 1] + 1; + } + var y2 = x2 - k2; + while (x2 < text1_length && y2 < text2_length && + text1.charAt(text1_length - x2 - 1) == + text2.charAt(text2_length - y2 - 1)) { + x2++; + y2++; + } + v2[k2_offset] = x2; + if (x2 > text1_length) { + // Ran off the left of the graph. + k2end += 2; + } else if (y2 > text2_length) { + // Ran off the top of the graph. + k2start += 2; + } else if (!front) { + var k1_offset = v_offset + delta - k2; + if (k1_offset >= 0 && k1_offset < v_length && v1[k1_offset] != -1) { + var x1 = v1[k1_offset]; + var y1 = v_offset + x1 - k1_offset; + // Mirror x2 onto top-left coordinate system. + x2 = text1_length - x2; + if (x1 >= x2) { + // Overlap detected. + return this.diff_bisectSplit_(text1, text2, x1, y1, deadline); + } + } + } + } + } + // Diff took too long and hit the deadline or + // number of diffs equals number of characters, no commonality at all. + return [new diff_match_patch.Diff(DIFF_DELETE, text1), + new diff_match_patch.Diff(DIFF_INSERT, text2)]; +}; + + +/** + * Given the location of the 'middle snake', split the diff in two parts + * and recurse. + * @param {string} text1 Old string to be diffed. + * @param {string} text2 New string to be diffed. + * @param {number} x Index of split point in text1. + * @param {number} y Index of split point in text2. + * @param {number} deadline Time at which to bail if not yet complete. + * @return {!Array.} Array of diff tuples. + * @private + */ +diff_match_patch.prototype.diff_bisectSplit_ = function(text1, text2, x, y, + deadline) { + var text1a = text1.substring(0, x); + var text2a = text2.substring(0, y); + var text1b = text1.substring(x); + var text2b = text2.substring(y); + + // Compute both diffs serially. + var diffs = this.diff_main(text1a, text2a, false, deadline); + var diffsb = this.diff_main(text1b, text2b, false, deadline); + + return diffs.concat(diffsb); +}; + + +/** + * Split two texts into an array of strings. Reduce the texts to a string of + * hashes where each Unicode character represents one line. + * @param {string} text1 First string. + * @param {string} text2 Second string. + * @return {{chars1: string, chars2: string, lineArray: !Array.}} + * An object containing the encoded text1, the encoded text2 and + * the array of unique strings. + * The zeroth element of the array of unique strings is intentionally blank. + * @private + */ +diff_match_patch.prototype.diff_linesToChars_ = function(text1, text2) { + var lineArray = []; // e.g. lineArray[4] == 'Hello\n' + var lineHash = {}; // e.g. lineHash['Hello\n'] == 4 + + // '\x00' is a valid character, but various debuggers don't like it. + // So we'll insert a junk entry to avoid generating a null character. + lineArray[0] = ''; + + /** + * Split a text into an array of strings. Reduce the texts to a string of + * hashes where each Unicode character represents one line. + * Modifies linearray and linehash through being a closure. + * @param {string} text String to encode. + * @return {string} Encoded string. + * @private + */ + function diff_linesToCharsMunge_(text) { + var chars = ''; + // Walk the text, pulling out a substring for each line. + // text.split('\n') would would temporarily double our memory footprint. + // Modifying text would create many large strings to garbage collect. + var lineStart = 0; + var lineEnd = -1; + // Keeping our own length variable is faster than looking it up. + var lineArrayLength = lineArray.length; + while (lineEnd < text.length - 1) { + lineEnd = text.indexOf('\n', lineStart); + if (lineEnd == -1) { + lineEnd = text.length - 1; + } + var line = text.substring(lineStart, lineEnd + 1); + + if (lineHash.hasOwnProperty ? lineHash.hasOwnProperty(line) : + (lineHash[line] !== undefined)) { + chars += String.fromCharCode(lineHash[line]); + } else { + if (lineArrayLength == maxLines) { + // Bail out at 65535 because + // String.fromCharCode(65536) == String.fromCharCode(0) + line = text.substring(lineStart); + lineEnd = text.length; + } + chars += String.fromCharCode(lineArrayLength); + lineHash[line] = lineArrayLength; + lineArray[lineArrayLength++] = line; + } + lineStart = lineEnd + 1; + } + return chars; + } + // Allocate 2/3rds of the space for text1, the rest for text2. + var maxLines = 40000; + var chars1 = diff_linesToCharsMunge_(text1); + maxLines = 65535; + var chars2 = diff_linesToCharsMunge_(text2); + return {chars1: chars1, chars2: chars2, lineArray: lineArray}; +}; + + +/** + * Rehydrate the text in a diff from a string of line hashes to real lines of + * text. + * @param {!Array.} diffs Array of diff tuples. + * @param {!Array.} lineArray Array of unique strings. + * @private + */ +diff_match_patch.prototype.diff_charsToLines_ = function(diffs, lineArray) { + for (var i = 0; i < diffs.length; i++) { + var chars = diffs[i][1]; + var text = []; + for (var j = 0; j < chars.length; j++) { + text[j] = lineArray[chars.charCodeAt(j)]; + } + diffs[i][1] = text.join(''); + } +}; + + +/** + * Determine the common prefix of two strings. + * @param {string} text1 First string. + * @param {string} text2 Second string. + * @return {number} The number of characters common to the start of each + * string. + */ +diff_match_patch.prototype.diff_commonPrefix = function(text1, text2) { + // Quick check for common null cases. + if (!text1 || !text2 || text1.charAt(0) != text2.charAt(0)) { + return 0; + } + // Binary search. + // Performance analysis: https://neil.fraser.name/news/2007/10/09/ + var pointermin = 0; + var pointermax = Math.min(text1.length, text2.length); + var pointermid = pointermax; + var pointerstart = 0; + while (pointermin < pointermid) { + if (text1.substring(pointerstart, pointermid) == + text2.substring(pointerstart, pointermid)) { + pointermin = pointermid; + pointerstart = pointermin; + } else { + pointermax = pointermid; + } + pointermid = Math.floor((pointermax - pointermin) / 2 + pointermin); + } + return pointermid; +}; + + +/** + * Determine the common suffix of two strings. + * @param {string} text1 First string. + * @param {string} text2 Second string. + * @return {number} The number of characters common to the end of each string. + */ +diff_match_patch.prototype.diff_commonSuffix = function(text1, text2) { + // Quick check for common null cases. + if (!text1 || !text2 || + text1.charAt(text1.length - 1) != text2.charAt(text2.length - 1)) { + return 0; + } + // Binary search. + // Performance analysis: https://neil.fraser.name/news/2007/10/09/ + var pointermin = 0; + var pointermax = Math.min(text1.length, text2.length); + var pointermid = pointermax; + var pointerend = 0; + while (pointermin < pointermid) { + if (text1.substring(text1.length - pointermid, text1.length - pointerend) == + text2.substring(text2.length - pointermid, text2.length - pointerend)) { + pointermin = pointermid; + pointerend = pointermin; + } else { + pointermax = pointermid; + } + pointermid = Math.floor((pointermax - pointermin) / 2 + pointermin); + } + return pointermid; +}; + + +/** + * Determine if the suffix of one string is the prefix of another. + * @param {string} text1 First string. + * @param {string} text2 Second string. + * @return {number} The number of characters common to the end of the first + * string and the start of the second string. + * @private + */ +diff_match_patch.prototype.diff_commonOverlap_ = function(text1, text2) { + // Cache the text lengths to prevent multiple calls. + var text1_length = text1.length; + var text2_length = text2.length; + // Eliminate the null case. + if (text1_length == 0 || text2_length == 0) { + return 0; + } + // Truncate the longer string. + if (text1_length > text2_length) { + text1 = text1.substring(text1_length - text2_length); + } else if (text1_length < text2_length) { + text2 = text2.substring(0, text1_length); + } + var text_length = Math.min(text1_length, text2_length); + // Quick check for the worst case. + if (text1 == text2) { + return text_length; + } + + // Start by looking for a single character match + // and increase length until no match is found. + // Performance analysis: https://neil.fraser.name/news/2010/11/04/ + var best = 0; + var length = 1; + while (true) { + var pattern = text1.substring(text_length - length); + var found = text2.indexOf(pattern); + if (found == -1) { + return best; + } + length += found; + if (found == 0 || text1.substring(text_length - length) == + text2.substring(0, length)) { + best = length; + length++; + } + } +}; + + +/** + * Do the two texts share a substring which is at least half the length of the + * longer text? + * This speedup can produce non-minimal diffs. + * @param {string} text1 First string. + * @param {string} text2 Second string. + * @return {Array.} Five element Array, containing the prefix of + * text1, the suffix of text1, the prefix of text2, the suffix of + * text2 and the common middle. Or null if there was no match. + * @private + */ +diff_match_patch.prototype.diff_halfMatch_ = function(text1, text2) { + if (this.Diff_Timeout <= 0) { + // Don't risk returning a non-optimal diff if we have unlimited time. + return null; + } + var longtext = text1.length > text2.length ? text1 : text2; + var shorttext = text1.length > text2.length ? text2 : text1; + if (longtext.length < 4 || shorttext.length * 2 < longtext.length) { + return null; // Pointless. + } + var dmp = this; // 'this' becomes 'window' in a closure. + + /** + * Does a substring of shorttext exist within longtext such that the substring + * is at least half the length of longtext? + * Closure, but does not reference any external variables. + * @param {string} longtext Longer string. + * @param {string} shorttext Shorter string. + * @param {number} i Start index of quarter length substring within longtext. + * @return {Array.} Five element Array, containing the prefix of + * longtext, the suffix of longtext, the prefix of shorttext, the suffix + * of shorttext and the common middle. Or null if there was no match. + * @private + */ + function diff_halfMatchI_(longtext, shorttext, i) { + // Start with a 1/4 length substring at position i as a seed. + var seed = longtext.substring(i, i + Math.floor(longtext.length / 4)); + var j = -1; + var best_common = ''; + var best_longtext_a, best_longtext_b, best_shorttext_a, best_shorttext_b; + while ((j = shorttext.indexOf(seed, j + 1)) != -1) { + var prefixLength = dmp.diff_commonPrefix(longtext.substring(i), + shorttext.substring(j)); + var suffixLength = dmp.diff_commonSuffix(longtext.substring(0, i), + shorttext.substring(0, j)); + if (best_common.length < suffixLength + prefixLength) { + best_common = shorttext.substring(j - suffixLength, j) + + shorttext.substring(j, j + prefixLength); + best_longtext_a = longtext.substring(0, i - suffixLength); + best_longtext_b = longtext.substring(i + prefixLength); + best_shorttext_a = shorttext.substring(0, j - suffixLength); + best_shorttext_b = shorttext.substring(j + prefixLength); + } + } + if (best_common.length * 2 >= longtext.length) { + return [best_longtext_a, best_longtext_b, + best_shorttext_a, best_shorttext_b, best_common]; + } else { + return null; + } + } + + // First check if the second quarter is the seed for a half-match. + var hm1 = diff_halfMatchI_(longtext, shorttext, + Math.ceil(longtext.length / 4)); + // Check again based on the third quarter. + var hm2 = diff_halfMatchI_(longtext, shorttext, + Math.ceil(longtext.length / 2)); + var hm; + if (!hm1 && !hm2) { + return null; + } else if (!hm2) { + hm = hm1; + } else if (!hm1) { + hm = hm2; + } else { + // Both matched. Select the longest. + hm = hm1[4].length > hm2[4].length ? hm1 : hm2; + } + + // A half-match was found, sort out the return data. + var text1_a, text1_b, text2_a, text2_b; + if (text1.length > text2.length) { + text1_a = hm[0]; + text1_b = hm[1]; + text2_a = hm[2]; + text2_b = hm[3]; + } else { + text2_a = hm[0]; + text2_b = hm[1]; + text1_a = hm[2]; + text1_b = hm[3]; + } + var mid_common = hm[4]; + return [text1_a, text1_b, text2_a, text2_b, mid_common]; +}; + + +/** + * Reduce the number of edits by eliminating semantically trivial equalities. + * @param {!Array.} diffs Array of diff tuples. + */ +diff_match_patch.prototype.diff_cleanupSemantic = function(diffs) { + var changes = false; + var equalities = []; // Stack of indices where equalities are found. + var equalitiesLength = 0; // Keeping our own length var is faster in JS. + /** @type {?string} */ + var lastEquality = null; + // Always equal to diffs[equalities[equalitiesLength - 1]][1] + var pointer = 0; // Index of current position. + // Number of characters that changed prior to the equality. + var length_insertions1 = 0; + var length_deletions1 = 0; + // Number of characters that changed after the equality. + var length_insertions2 = 0; + var length_deletions2 = 0; + while (pointer < diffs.length) { + if (diffs[pointer][0] == DIFF_EQUAL) { // Equality found. + equalities[equalitiesLength++] = pointer; + length_insertions1 = length_insertions2; + length_deletions1 = length_deletions2; + length_insertions2 = 0; + length_deletions2 = 0; + lastEquality = diffs[pointer][1]; + } else { // An insertion or deletion. + if (diffs[pointer][0] == DIFF_INSERT) { + length_insertions2 += diffs[pointer][1].length; + } else { + length_deletions2 += diffs[pointer][1].length; + } + // Eliminate an equality that is smaller or equal to the edits on both + // sides of it. + if (lastEquality && (lastEquality.length <= + Math.max(length_insertions1, length_deletions1)) && + (lastEquality.length <= Math.max(length_insertions2, + length_deletions2))) { + // Duplicate record. + diffs.splice(equalities[equalitiesLength - 1], 0, + new diff_match_patch.Diff(DIFF_DELETE, lastEquality)); + // Change second copy to insert. + diffs[equalities[equalitiesLength - 1] + 1][0] = DIFF_INSERT; + // Throw away the equality we just deleted. + equalitiesLength--; + // Throw away the previous equality (it needs to be reevaluated). + equalitiesLength--; + pointer = equalitiesLength > 0 ? equalities[equalitiesLength - 1] : -1; + length_insertions1 = 0; // Reset the counters. + length_deletions1 = 0; + length_insertions2 = 0; + length_deletions2 = 0; + lastEquality = null; + changes = true; + } + } + pointer++; + } + + // Normalize the diff. + if (changes) { + this.diff_cleanupMerge(diffs); + } + this.diff_cleanupSemanticLossless(diffs); + + // Find any overlaps between deletions and insertions. + // e.g: abcxxxxxxdef + // -> abcxxxdef + // e.g: xxxabcdefxxx + // -> defxxxabc + // Only extract an overlap if it is as big as the edit ahead or behind it. + pointer = 1; + while (pointer < diffs.length) { + if (diffs[pointer - 1][0] == DIFF_DELETE && + diffs[pointer][0] == DIFF_INSERT) { + var deletion = diffs[pointer - 1][1]; + var insertion = diffs[pointer][1]; + var overlap_length1 = this.diff_commonOverlap_(deletion, insertion); + var overlap_length2 = this.diff_commonOverlap_(insertion, deletion); + if (overlap_length1 >= overlap_length2) { + if (overlap_length1 >= deletion.length / 2 || + overlap_length1 >= insertion.length / 2) { + // Overlap found. Insert an equality and trim the surrounding edits. + diffs.splice(pointer, 0, new diff_match_patch.Diff(DIFF_EQUAL, + insertion.substring(0, overlap_length1))); + diffs[pointer - 1][1] = + deletion.substring(0, deletion.length - overlap_length1); + diffs[pointer + 1][1] = insertion.substring(overlap_length1); + pointer++; + } + } else { + if (overlap_length2 >= deletion.length / 2 || + overlap_length2 >= insertion.length / 2) { + // Reverse overlap found. + // Insert an equality and swap and trim the surrounding edits. + diffs.splice(pointer, 0, new diff_match_patch.Diff(DIFF_EQUAL, + deletion.substring(0, overlap_length2))); + diffs[pointer - 1][0] = DIFF_INSERT; + diffs[pointer - 1][1] = + insertion.substring(0, insertion.length - overlap_length2); + diffs[pointer + 1][0] = DIFF_DELETE; + diffs[pointer + 1][1] = + deletion.substring(overlap_length2); + pointer++; + } + } + pointer++; + } + pointer++; + } +}; + + +/** + * Look for single edits surrounded on both sides by equalities + * which can be shifted sideways to align the edit to a word boundary. + * e.g: The cat came. -> The cat came. + * @param {!Array.} diffs Array of diff tuples. + */ +diff_match_patch.prototype.diff_cleanupSemanticLossless = function(diffs) { + /** + * Given two strings, compute a score representing whether the internal + * boundary falls on logical boundaries. + * Scores range from 6 (best) to 0 (worst). + * Closure, but does not reference any external variables. + * @param {string} one First string. + * @param {string} two Second string. + * @return {number} The score. + * @private + */ + function diff_cleanupSemanticScore_(one, two) { + if (!one || !two) { + // Edges are the best. + return 6; + } + + // Each port of this function behaves slightly differently due to + // subtle differences in each language's definition of things like + // 'whitespace'. Since this function's purpose is largely cosmetic, + // the choice has been made to use each language's native features + // rather than force total conformity. + var char1 = one.charAt(one.length - 1); + var char2 = two.charAt(0); + var nonAlphaNumeric1 = char1.match(diff_match_patch.nonAlphaNumericRegex_); + var nonAlphaNumeric2 = char2.match(diff_match_patch.nonAlphaNumericRegex_); + var whitespace1 = nonAlphaNumeric1 && + char1.match(diff_match_patch.whitespaceRegex_); + var whitespace2 = nonAlphaNumeric2 && + char2.match(diff_match_patch.whitespaceRegex_); + var lineBreak1 = whitespace1 && + char1.match(diff_match_patch.linebreakRegex_); + var lineBreak2 = whitespace2 && + char2.match(diff_match_patch.linebreakRegex_); + var blankLine1 = lineBreak1 && + one.match(diff_match_patch.blanklineEndRegex_); + var blankLine2 = lineBreak2 && + two.match(diff_match_patch.blanklineStartRegex_); + + if (blankLine1 || blankLine2) { + // Five points for blank lines. + return 5; + } else if (lineBreak1 || lineBreak2) { + // Four points for line breaks. + return 4; + } else if (nonAlphaNumeric1 && !whitespace1 && whitespace2) { + // Three points for end of sentences. + return 3; + } else if (whitespace1 || whitespace2) { + // Two points for whitespace. + return 2; + } else if (nonAlphaNumeric1 || nonAlphaNumeric2) { + // One point for non-alphanumeric. + return 1; + } + return 0; + } + + var pointer = 1; + // Intentionally ignore the first and last element (don't need checking). + while (pointer < diffs.length - 1) { + if (diffs[pointer - 1][0] == DIFF_EQUAL && + diffs[pointer + 1][0] == DIFF_EQUAL) { + // This is a single edit surrounded by equalities. + var equality1 = diffs[pointer - 1][1]; + var edit = diffs[pointer][1]; + var equality2 = diffs[pointer + 1][1]; + + // First, shift the edit as far left as possible. + var commonOffset = this.diff_commonSuffix(equality1, edit); + if (commonOffset) { + var commonString = edit.substring(edit.length - commonOffset); + equality1 = equality1.substring(0, equality1.length - commonOffset); + edit = commonString + edit.substring(0, edit.length - commonOffset); + equality2 = commonString + equality2; + } + + // Second, step character by character right, looking for the best fit. + var bestEquality1 = equality1; + var bestEdit = edit; + var bestEquality2 = equality2; + var bestScore = diff_cleanupSemanticScore_(equality1, edit) + + diff_cleanupSemanticScore_(edit, equality2); + while (edit.charAt(0) === equality2.charAt(0)) { + equality1 += edit.charAt(0); + edit = edit.substring(1) + equality2.charAt(0); + equality2 = equality2.substring(1); + var score = diff_cleanupSemanticScore_(equality1, edit) + + diff_cleanupSemanticScore_(edit, equality2); + // The >= encourages trailing rather than leading whitespace on edits. + if (score >= bestScore) { + bestScore = score; + bestEquality1 = equality1; + bestEdit = edit; + bestEquality2 = equality2; + } + } + + if (diffs[pointer - 1][1] != bestEquality1) { + // We have an improvement, save it back to the diff. + if (bestEquality1) { + diffs[pointer - 1][1] = bestEquality1; + } else { + diffs.splice(pointer - 1, 1); + pointer--; + } + diffs[pointer][1] = bestEdit; + if (bestEquality2) { + diffs[pointer + 1][1] = bestEquality2; + } else { + diffs.splice(pointer + 1, 1); + pointer--; + } + } + } + pointer++; + } +}; + +// Define some regex patterns for matching boundaries. +diff_match_patch.nonAlphaNumericRegex_ = /[^a-zA-Z0-9]/; +diff_match_patch.whitespaceRegex_ = /\s/; +diff_match_patch.linebreakRegex_ = /[\r\n]/; +diff_match_patch.blanklineEndRegex_ = /\n\r?\n$/; +diff_match_patch.blanklineStartRegex_ = /^\r?\n\r?\n/; + +/** + * Reduce the number of edits by eliminating operationally trivial equalities. + * @param {!Array.} diffs Array of diff tuples. + */ +diff_match_patch.prototype.diff_cleanupEfficiency = function(diffs) { + var changes = false; + var equalities = []; // Stack of indices where equalities are found. + var equalitiesLength = 0; // Keeping our own length var is faster in JS. + /** @type {?string} */ + var lastEquality = null; + // Always equal to diffs[equalities[equalitiesLength - 1]][1] + var pointer = 0; // Index of current position. + // Is there an insertion operation before the last equality. + var pre_ins = false; + // Is there a deletion operation before the last equality. + var pre_del = false; + // Is there an insertion operation after the last equality. + var post_ins = false; + // Is there a deletion operation after the last equality. + var post_del = false; + while (pointer < diffs.length) { + if (diffs[pointer][0] == DIFF_EQUAL) { // Equality found. + if (diffs[pointer][1].length < this.Diff_EditCost && + (post_ins || post_del)) { + // Candidate found. + equalities[equalitiesLength++] = pointer; + pre_ins = post_ins; + pre_del = post_del; + lastEquality = diffs[pointer][1]; + } else { + // Not a candidate, and can never become one. + equalitiesLength = 0; + lastEquality = null; + } + post_ins = post_del = false; + } else { // An insertion or deletion. + if (diffs[pointer][0] == DIFF_DELETE) { + post_del = true; + } else { + post_ins = true; + } + /* + * Five types to be split: + * ABXYCD + * AXCD + * ABXC + * AXCD + * ABXC + */ + if (lastEquality && ((pre_ins && pre_del && post_ins && post_del) || + ((lastEquality.length < this.Diff_EditCost / 2) && + (pre_ins + pre_del + post_ins + post_del) == 3))) { + // Duplicate record. + diffs.splice(equalities[equalitiesLength - 1], 0, + new diff_match_patch.Diff(DIFF_DELETE, lastEquality)); + // Change second copy to insert. + diffs[equalities[equalitiesLength - 1] + 1][0] = DIFF_INSERT; + equalitiesLength--; // Throw away the equality we just deleted; + lastEquality = null; + if (pre_ins && pre_del) { + // No changes made which could affect previous entry, keep going. + post_ins = post_del = true; + equalitiesLength = 0; + } else { + equalitiesLength--; // Throw away the previous equality. + pointer = equalitiesLength > 0 ? + equalities[equalitiesLength - 1] : -1; + post_ins = post_del = false; + } + changes = true; + } + } + pointer++; + } + + if (changes) { + this.diff_cleanupMerge(diffs); + } +}; + + +/** + * Reorder and merge like edit sections. Merge equalities. + * Any edit section can move as long as it doesn't cross an equality. + * @param {!Array.} diffs Array of diff tuples. + */ +diff_match_patch.prototype.diff_cleanupMerge = function(diffs) { + // Add a dummy entry at the end. + diffs.push(new diff_match_patch.Diff(DIFF_EQUAL, '')); + var pointer = 0; + var count_delete = 0; + var count_insert = 0; + var text_delete = ''; + var text_insert = ''; + var commonlength; + while (pointer < diffs.length) { + switch (diffs[pointer][0]) { + case DIFF_INSERT: + count_insert++; + text_insert += diffs[pointer][1]; + pointer++; + break; + case DIFF_DELETE: + count_delete++; + text_delete += diffs[pointer][1]; + pointer++; + break; + case DIFF_EQUAL: + // Upon reaching an equality, check for prior redundancies. + if (count_delete + count_insert > 1) { + if (count_delete !== 0 && count_insert !== 0) { + // Factor out any common prefixies. + commonlength = this.diff_commonPrefix(text_insert, text_delete); + if (commonlength !== 0) { + if ((pointer - count_delete - count_insert) > 0 && + diffs[pointer - count_delete - count_insert - 1][0] == + DIFF_EQUAL) { + diffs[pointer - count_delete - count_insert - 1][1] += + text_insert.substring(0, commonlength); + } else { + diffs.splice(0, 0, new diff_match_patch.Diff(DIFF_EQUAL, + text_insert.substring(0, commonlength))); + pointer++; + } + text_insert = text_insert.substring(commonlength); + text_delete = text_delete.substring(commonlength); + } + // Factor out any common suffixies. + commonlength = this.diff_commonSuffix(text_insert, text_delete); + if (commonlength !== 0) { + diffs[pointer][1] = text_insert.substring(text_insert.length - + commonlength) + diffs[pointer][1]; + text_insert = text_insert.substring(0, text_insert.length - + commonlength); + text_delete = text_delete.substring(0, text_delete.length - + commonlength); + } + } + // Delete the offending records and add the merged ones. + pointer -= count_delete + count_insert; + diffs.splice(pointer, count_delete + count_insert); + if (text_delete.length) { + diffs.splice(pointer, 0, + new diff_match_patch.Diff(DIFF_DELETE, text_delete)); + pointer++; + } + if (text_insert.length) { + diffs.splice(pointer, 0, + new diff_match_patch.Diff(DIFF_INSERT, text_insert)); + pointer++; + } + pointer++; + } else if (pointer !== 0 && diffs[pointer - 1][0] == DIFF_EQUAL) { + // Merge this equality with the previous one. + diffs[pointer - 1][1] += diffs[pointer][1]; + diffs.splice(pointer, 1); + } else { + pointer++; + } + count_insert = 0; + count_delete = 0; + text_delete = ''; + text_insert = ''; + break; + } + } + if (diffs[diffs.length - 1][1] === '') { + diffs.pop(); // Remove the dummy entry at the end. + } + + // Second pass: look for single edits surrounded on both sides by equalities + // which can be shifted sideways to eliminate an equality. + // e.g: ABAC -> ABAC + var changes = false; + pointer = 1; + // Intentionally ignore the first and last element (don't need checking). + while (pointer < diffs.length - 1) { + if (diffs[pointer - 1][0] == DIFF_EQUAL && + diffs[pointer + 1][0] == DIFF_EQUAL) { + // This is a single edit surrounded by equalities. + if (diffs[pointer][1].substring(diffs[pointer][1].length - + diffs[pointer - 1][1].length) == diffs[pointer - 1][1]) { + // Shift the edit over the previous equality. + diffs[pointer][1] = diffs[pointer - 1][1] + + diffs[pointer][1].substring(0, diffs[pointer][1].length - + diffs[pointer - 1][1].length); + diffs[pointer + 1][1] = diffs[pointer - 1][1] + diffs[pointer + 1][1]; + diffs.splice(pointer - 1, 1); + changes = true; + } else if (diffs[pointer][1].substring(0, diffs[pointer + 1][1].length) == + diffs[pointer + 1][1]) { + // Shift the edit over the next equality. + diffs[pointer - 1][1] += diffs[pointer + 1][1]; + diffs[pointer][1] = + diffs[pointer][1].substring(diffs[pointer + 1][1].length) + + diffs[pointer + 1][1]; + diffs.splice(pointer + 1, 1); + changes = true; + } + } + pointer++; + } + // If shifts were made, the diff needs reordering and another shift sweep. + if (changes) { + this.diff_cleanupMerge(diffs); + } +}; + + +/** + * loc is a location in text1, compute and return the equivalent location in + * text2. + * e.g. 'The cat' vs 'The big cat', 1->1, 5->8 + * @param {!Array.} diffs Array of diff tuples. + * @param {number} loc Location within text1. + * @return {number} Location within text2. + */ +diff_match_patch.prototype.diff_xIndex = function(diffs, loc) { + var chars1 = 0; + var chars2 = 0; + var last_chars1 = 0; + var last_chars2 = 0; + var x; + for (x = 0; x < diffs.length; x++) { + if (diffs[x][0] !== DIFF_INSERT) { // Equality or deletion. + chars1 += diffs[x][1].length; + } + if (diffs[x][0] !== DIFF_DELETE) { // Equality or insertion. + chars2 += diffs[x][1].length; + } + if (chars1 > loc) { // Overshot the location. + break; + } + last_chars1 = chars1; + last_chars2 = chars2; + } + // Was the location was deleted? + if (diffs.length != x && diffs[x][0] === DIFF_DELETE) { + return last_chars2; + } + // Add the remaining character length. + return last_chars2 + (loc - last_chars1); +}; + + +/** + * Convert a diff array into a pretty HTML report. + * @param {!Array.} diffs Array of diff tuples. + * @return {string} HTML representation. + */ +diff_match_patch.prototype.diff_prettyHtml = function(diffs) { + var html = []; + var pattern_amp = /&/g; + var pattern_lt = //g; + var pattern_para = /\n/g; + for (var x = 0; x < diffs.length; x++) { + var op = diffs[x][0]; // Operation (insert, delete, equal) + var data = diffs[x][1]; // Text of change. + var text = data.replace(pattern_amp, '&').replace(pattern_lt, '<') + .replace(pattern_gt, '>').replace(pattern_para, '¶
'); + switch (op) { + case DIFF_INSERT: + html[x] = '' + text + ''; + break; + case DIFF_DELETE: + html[x] = '' + text + ''; + break; + case DIFF_EQUAL: + html[x] = '' + text + ''; + break; + } + } + return html.join(''); +}; + + +/** + * Compute and return the source text (all equalities and deletions). + * @param {!Array.} diffs Array of diff tuples. + * @return {string} Source text. + */ +diff_match_patch.prototype.diff_text1 = function(diffs) { + var text = []; + for (var x = 0; x < diffs.length; x++) { + if (diffs[x][0] !== DIFF_INSERT) { + text[x] = diffs[x][1]; + } + } + return text.join(''); +}; + + +/** + * Compute and return the destination text (all equalities and insertions). + * @param {!Array.} diffs Array of diff tuples. + * @return {string} Destination text. + */ +diff_match_patch.prototype.diff_text2 = function(diffs) { + var text = []; + for (var x = 0; x < diffs.length; x++) { + if (diffs[x][0] !== DIFF_DELETE) { + text[x] = diffs[x][1]; + } + } + return text.join(''); +}; + + +/** + * Compute the Levenshtein distance; the number of inserted, deleted or + * substituted characters. + * @param {!Array.} diffs Array of diff tuples. + * @return {number} Number of changes. + */ +diff_match_patch.prototype.diff_levenshtein = function(diffs) { + var levenshtein = 0; + var insertions = 0; + var deletions = 0; + for (var x = 0; x < diffs.length; x++) { + var op = diffs[x][0]; + var data = diffs[x][1]; + switch (op) { + case DIFF_INSERT: + insertions += data.length; + break; + case DIFF_DELETE: + deletions += data.length; + break; + case DIFF_EQUAL: + // A deletion and an insertion is one substitution. + levenshtein += Math.max(insertions, deletions); + insertions = 0; + deletions = 0; + break; + } + } + levenshtein += Math.max(insertions, deletions); + return levenshtein; +}; + + +/** + * Crush the diff into an encoded string which describes the operations + * required to transform text1 into text2. + * E.g. =3\t-2\t+ing -> Keep 3 chars, delete 2 chars, insert 'ing'. + * Operations are tab-separated. Inserted text is escaped using %xx notation. + * @param {!Array.} diffs Array of diff tuples. + * @return {string} Delta text. + */ +diff_match_patch.prototype.diff_toDelta = function(diffs) { + var text = []; + for (var x = 0; x < diffs.length; x++) { + switch (diffs[x][0]) { + case DIFF_INSERT: + text[x] = '+' + encodeURI(diffs[x][1]); + break; + case DIFF_DELETE: + text[x] = '-' + diffs[x][1].length; + break; + case DIFF_EQUAL: + text[x] = '=' + diffs[x][1].length; + break; + } + } + return text.join('\t').replace(/%20/g, ' '); +}; + + +/** + * Given the original text1, and an encoded string which describes the + * operations required to transform text1 into text2, compute the full diff. + * @param {string} text1 Source string for the diff. + * @param {string} delta Delta text. + * @return {!Array.} Array of diff tuples. + * @throws {!Error} If invalid input. + */ +diff_match_patch.prototype.diff_fromDelta = function(text1, delta) { + var diffs = []; + var diffsLength = 0; // Keeping our own length var is faster in JS. + var pointer = 0; // Cursor in text1 + var tokens = delta.split(/\t/g); + for (var x = 0; x < tokens.length; x++) { + // Each token begins with a one character parameter which specifies the + // operation of this token (delete, insert, equality). + var param = tokens[x].substring(1); + switch (tokens[x].charAt(0)) { + case '+': + try { + diffs[diffsLength++] = + new diff_match_patch.Diff(DIFF_INSERT, decodeURI(param)); + } catch (ex) { + // Malformed URI sequence. + throw new Error('Illegal escape in diff_fromDelta: ' + param); + } + break; + case '-': + // Fall through. + case '=': + var n = parseInt(param, 10); + if (isNaN(n) || n < 0) { + throw new Error('Invalid number in diff_fromDelta: ' + param); + } + var text = text1.substring(pointer, pointer += n); + if (tokens[x].charAt(0) == '=') { + diffs[diffsLength++] = new diff_match_patch.Diff(DIFF_EQUAL, text); + } else { + diffs[diffsLength++] = new diff_match_patch.Diff(DIFF_DELETE, text); + } + break; + default: + // Blank tokens are ok (from a trailing \t). + // Anything else is an error. + if (tokens[x]) { + throw new Error('Invalid diff operation in diff_fromDelta: ' + + tokens[x]); + } + } + } + if (pointer != text1.length) { + throw new Error('Delta length (' + pointer + + ') does not equal source text length (' + text1.length + ').'); + } + return diffs; +}; + + +// MATCH FUNCTIONS + + +/** + * Locate the best instance of 'pattern' in 'text' near 'loc'. + * @param {string} text The text to search. + * @param {string} pattern The pattern to search for. + * @param {number} loc The location to search around. + * @return {number} Best match index or -1. + */ +diff_match_patch.prototype.match_main = function(text, pattern, loc) { + // Check for null inputs. + if (text == null || pattern == null || loc == null) { + throw new Error('Null input. (match_main)'); + } + + loc = Math.max(0, Math.min(loc, text.length)); + if (text == pattern) { + // Shortcut (potentially not guaranteed by the algorithm) + return 0; + } else if (!text.length) { + // Nothing to match. + return -1; + } else if (text.substring(loc, loc + pattern.length) == pattern) { + // Perfect match at the perfect spot! (Includes case of null pattern) + return loc; + } else { + // Do a fuzzy compare. + return this.match_bitap_(text, pattern, loc); + } +}; + + +/** + * Locate the best instance of 'pattern' in 'text' near 'loc' using the + * Bitap algorithm. + * @param {string} text The text to search. + * @param {string} pattern The pattern to search for. + * @param {number} loc The location to search around. + * @return {number} Best match index or -1. + * @private + */ +diff_match_patch.prototype.match_bitap_ = function(text, pattern, loc) { + if (pattern.length > this.Match_MaxBits) { + throw new Error('Pattern too long for this browser.'); + } + + // Initialise the alphabet. + var s = this.match_alphabet_(pattern); + + var dmp = this; // 'this' becomes 'window' in a closure. + + /** + * Compute and return the score for a match with e errors and x location. + * Accesses loc and pattern through being a closure. + * @param {number} e Number of errors in match. + * @param {number} x Location of match. + * @return {number} Overall score for match (0.0 = good, 1.0 = bad). + * @private + */ + function match_bitapScore_(e, x) { + var accuracy = e / pattern.length; + var proximity = Math.abs(loc - x); + if (!dmp.Match_Distance) { + // Dodge divide by zero error. + return proximity ? 1.0 : accuracy; + } + return accuracy + (proximity / dmp.Match_Distance); + } + + // Highest score beyond which we give up. + var score_threshold = this.Match_Threshold; + // Is there a nearby exact match? (speedup) + var best_loc = text.indexOf(pattern, loc); + if (best_loc != -1) { + score_threshold = Math.min(match_bitapScore_(0, best_loc), score_threshold); + // What about in the other direction? (speedup) + best_loc = text.lastIndexOf(pattern, loc + pattern.length); + if (best_loc != -1) { + score_threshold = + Math.min(match_bitapScore_(0, best_loc), score_threshold); + } + } + + // Initialise the bit arrays. + var matchmask = 1 << (pattern.length - 1); + best_loc = -1; + + var bin_min, bin_mid; + var bin_max = pattern.length + text.length; + var last_rd; + for (var d = 0; d < pattern.length; d++) { + // Scan for the best match; each iteration allows for one more error. + // Run a binary search to determine how far from 'loc' we can stray at this + // error level. + bin_min = 0; + bin_mid = bin_max; + while (bin_min < bin_mid) { + if (match_bitapScore_(d, loc + bin_mid) <= score_threshold) { + bin_min = bin_mid; + } else { + bin_max = bin_mid; + } + bin_mid = Math.floor((bin_max - bin_min) / 2 + bin_min); + } + // Use the result from this iteration as the maximum for the next. + bin_max = bin_mid; + var start = Math.max(1, loc - bin_mid + 1); + var finish = Math.min(loc + bin_mid, text.length) + pattern.length; + + var rd = Array(finish + 2); + rd[finish + 1] = (1 << d) - 1; + for (var j = finish; j >= start; j--) { + // The alphabet (s) is a sparse hash, so the following line generates + // warnings. + var charMatch = s[text.charAt(j - 1)]; + if (d === 0) { // First pass: exact match. + rd[j] = ((rd[j + 1] << 1) | 1) & charMatch; + } else { // Subsequent passes: fuzzy match. + rd[j] = (((rd[j + 1] << 1) | 1) & charMatch) | + (((last_rd[j + 1] | last_rd[j]) << 1) | 1) | + last_rd[j + 1]; + } + if (rd[j] & matchmask) { + var score = match_bitapScore_(d, j - 1); + // This match will almost certainly be better than any existing match. + // But check anyway. + if (score <= score_threshold) { + // Told you so. + score_threshold = score; + best_loc = j - 1; + if (best_loc > loc) { + // When passing loc, don't exceed our current distance from loc. + start = Math.max(1, 2 * loc - best_loc); + } else { + // Already passed loc, downhill from here on in. + break; + } + } + } + } + // No hope for a (better) match at greater error levels. + if (match_bitapScore_(d + 1, loc) > score_threshold) { + break; + } + last_rd = rd; + } + return best_loc; +}; + + +/** + * Initialise the alphabet for the Bitap algorithm. + * @param {string} pattern The text to encode. + * @return {!Object} Hash of character locations. + * @private + */ +diff_match_patch.prototype.match_alphabet_ = function(pattern) { + var s = {}; + for (var i = 0; i < pattern.length; i++) { + s[pattern.charAt(i)] = 0; + } + for (var i = 0; i < pattern.length; i++) { + s[pattern.charAt(i)] |= 1 << (pattern.length - i - 1); + } + return s; +}; + + +// PATCH FUNCTIONS + + +/** + * Increase the context until it is unique, + * but don't let the pattern expand beyond Match_MaxBits. + * @param {!diff_match_patch.patch_obj} patch The patch to grow. + * @param {string} text Source text. + * @private + */ +diff_match_patch.prototype.patch_addContext_ = function(patch, text) { + if (text.length == 0) { + return; + } + if (patch.start2 === null) { + throw Error('patch not initialized'); + } + var pattern = text.substring(patch.start2, patch.start2 + patch.length1); + var padding = 0; + + // Look for the first and last matches of pattern in text. If two different + // matches are found, increase the pattern length. + while (text.indexOf(pattern) != text.lastIndexOf(pattern) && + pattern.length < this.Match_MaxBits - this.Patch_Margin - + this.Patch_Margin) { + padding += this.Patch_Margin; + pattern = text.substring(patch.start2 - padding, + patch.start2 + patch.length1 + padding); + } + // Add one chunk for good luck. + padding += this.Patch_Margin; + + // Add the prefix. + var prefix = text.substring(patch.start2 - padding, patch.start2); + if (prefix) { + patch.diffs.unshift(new diff_match_patch.Diff(DIFF_EQUAL, prefix)); + } + // Add the suffix. + var suffix = text.substring(patch.start2 + patch.length1, + patch.start2 + patch.length1 + padding); + if (suffix) { + patch.diffs.push(new diff_match_patch.Diff(DIFF_EQUAL, suffix)); + } + + // Roll back the start points. + patch.start1 -= prefix.length; + patch.start2 -= prefix.length; + // Extend the lengths. + patch.length1 += prefix.length + suffix.length; + patch.length2 += prefix.length + suffix.length; +}; + + +/** + * Compute a list of patches to turn text1 into text2. + * Use diffs if provided, otherwise compute it ourselves. + * There are four ways to call this function, depending on what data is + * available to the caller: + * Method 1: + * a = text1, b = text2 + * Method 2: + * a = diffs + * Method 3 (optimal): + * a = text1, b = diffs + * Method 4 (deprecated, use method 3): + * a = text1, b = text2, c = diffs + * + * @param {string|!Array.} a text1 (methods 1,3,4) or + * Array of diff tuples for text1 to text2 (method 2). + * @param {string|!Array.=} opt_b text2 (methods 1,4) or + * Array of diff tuples for text1 to text2 (method 3) or undefined (method 2). + * @param {string|!Array.=} opt_c Array of diff tuples + * for text1 to text2 (method 4) or undefined (methods 1,2,3). + * @return {!Array.} Array of Patch objects. + */ +diff_match_patch.prototype.patch_make = function(a, opt_b, opt_c) { + var text1, diffs; + if (typeof a == 'string' && typeof opt_b == 'string' && + typeof opt_c == 'undefined') { + // Method 1: text1, text2 + // Compute diffs from text1 and text2. + text1 = /** @type {string} */(a); + diffs = this.diff_main(text1, /** @type {string} */(opt_b), true); + if (diffs.length > 2) { + this.diff_cleanupSemantic(diffs); + this.diff_cleanupEfficiency(diffs); + } + } else if (a && typeof a == 'object' && typeof opt_b == 'undefined' && + typeof opt_c == 'undefined') { + // Method 2: diffs + // Compute text1 from diffs. + diffs = /** @type {!Array.} */(a); + text1 = this.diff_text1(diffs); + } else if (typeof a == 'string' && opt_b && typeof opt_b == 'object' && + typeof opt_c == 'undefined') { + // Method 3: text1, diffs + text1 = /** @type {string} */(a); + diffs = /** @type {!Array.} */(opt_b); + } else if (typeof a == 'string' && typeof opt_b == 'string' && + opt_c && typeof opt_c == 'object') { + // Method 4: text1, text2, diffs + // text2 is not used. + text1 = /** @type {string} */(a); + diffs = /** @type {!Array.} */(opt_c); + } else { + throw new Error('Unknown call format to patch_make.'); + } + + if (diffs.length === 0) { + return []; // Get rid of the null case. + } + var patches = []; + var patch = new diff_match_patch.patch_obj(); + var patchDiffLength = 0; // Keeping our own length var is faster in JS. + var char_count1 = 0; // Number of characters into the text1 string. + var char_count2 = 0; // Number of characters into the text2 string. + // Start with text1 (prepatch_text) and apply the diffs until we arrive at + // text2 (postpatch_text). We recreate the patches one by one to determine + // context info. + var prepatch_text = text1; + var postpatch_text = text1; + for (var x = 0; x < diffs.length; x++) { + var diff_type = diffs[x][0]; + var diff_text = diffs[x][1]; + + if (!patchDiffLength && diff_type !== DIFF_EQUAL) { + // A new patch starts here. + patch.start1 = char_count1; + patch.start2 = char_count2; + } + + switch (diff_type) { + case DIFF_INSERT: + patch.diffs[patchDiffLength++] = diffs[x]; + patch.length2 += diff_text.length; + postpatch_text = postpatch_text.substring(0, char_count2) + diff_text + + postpatch_text.substring(char_count2); + break; + case DIFF_DELETE: + patch.length1 += diff_text.length; + patch.diffs[patchDiffLength++] = diffs[x]; + postpatch_text = postpatch_text.substring(0, char_count2) + + postpatch_text.substring(char_count2 + + diff_text.length); + break; + case DIFF_EQUAL: + if (diff_text.length <= 2 * this.Patch_Margin && + patchDiffLength && diffs.length != x + 1) { + // Small equality inside a patch. + patch.diffs[patchDiffLength++] = diffs[x]; + patch.length1 += diff_text.length; + patch.length2 += diff_text.length; + } else if (diff_text.length >= 2 * this.Patch_Margin) { + // Time for a new patch. + if (patchDiffLength) { + this.patch_addContext_(patch, prepatch_text); + patches.push(patch); + patch = new diff_match_patch.patch_obj(); + patchDiffLength = 0; + // Unlike Unidiff, our patch lists have a rolling context. + // https://github.com/google/diff-match-patch/wiki/Unidiff + // Update prepatch text & pos to reflect the application of the + // just completed patch. + prepatch_text = postpatch_text; + char_count1 = char_count2; + } + } + break; + } + + // Update the current character count. + if (diff_type !== DIFF_INSERT) { + char_count1 += diff_text.length; + } + if (diff_type !== DIFF_DELETE) { + char_count2 += diff_text.length; + } + } + // Pick up the leftover patch if not empty. + if (patchDiffLength) { + this.patch_addContext_(patch, prepatch_text); + patches.push(patch); + } + + return patches; +}; + + +/** + * Given an array of patches, return another array that is identical. + * @param {!Array.} patches Array of Patch objects. + * @return {!Array.} Array of Patch objects. + */ +diff_match_patch.prototype.patch_deepCopy = function(patches) { + // Making deep copies is hard in JavaScript. + var patchesCopy = []; + for (var x = 0; x < patches.length; x++) { + var patch = patches[x]; + var patchCopy = new diff_match_patch.patch_obj(); + patchCopy.diffs = []; + for (var y = 0; y < patch.diffs.length; y++) { + patchCopy.diffs[y] = + new diff_match_patch.Diff(patch.diffs[y][0], patch.diffs[y][1]); + } + patchCopy.start1 = patch.start1; + patchCopy.start2 = patch.start2; + patchCopy.length1 = patch.length1; + patchCopy.length2 = patch.length2; + patchesCopy[x] = patchCopy; + } + return patchesCopy; +}; + + +/** + * Merge a set of patches onto the text. Return a patched text, as well + * as a list of true/false values indicating which patches were applied. + * @param {!Array.} patches Array of Patch objects. + * @param {string} text Old text. + * @return {!Array.>} Two element Array, containing the + * new text and an array of boolean values. + */ +diff_match_patch.prototype.patch_apply = function(patches, text) { + if (patches.length == 0) { + return [text, []]; + } + + // Deep copy the patches so that no changes are made to originals. + patches = this.patch_deepCopy(patches); + + var nullPadding = this.patch_addPadding(patches); + text = nullPadding + text + nullPadding; + + this.patch_splitMax(patches); + // delta keeps track of the offset between the expected and actual location + // of the previous patch. If there are patches expected at positions 10 and + // 20, but the first patch was found at 12, delta is 2 and the second patch + // has an effective expected position of 22. + var delta = 0; + var results = []; + for (var x = 0; x < patches.length; x++) { + var expected_loc = patches[x].start2 + delta; + var text1 = this.diff_text1(patches[x].diffs); + var start_loc; + var end_loc = -1; + if (text1.length > this.Match_MaxBits) { + // patch_splitMax will only provide an oversized pattern in the case of + // a monster delete. + start_loc = this.match_main(text, text1.substring(0, this.Match_MaxBits), + expected_loc); + if (start_loc != -1) { + end_loc = this.match_main(text, + text1.substring(text1.length - this.Match_MaxBits), + expected_loc + text1.length - this.Match_MaxBits); + if (end_loc == -1 || start_loc >= end_loc) { + // Can't find valid trailing context. Drop this patch. + start_loc = -1; + } + } + } else { + start_loc = this.match_main(text, text1, expected_loc); + } + if (start_loc == -1) { + // No match found. :( + results[x] = false; + // Subtract the delta for this failed patch from subsequent patches. + delta -= patches[x].length2 - patches[x].length1; + } else { + // Found a match. :) + results[x] = true; + delta = start_loc - expected_loc; + var text2; + if (end_loc == -1) { + text2 = text.substring(start_loc, start_loc + text1.length); + } else { + text2 = text.substring(start_loc, end_loc + this.Match_MaxBits); + } + if (text1 == text2) { + // Perfect match, just shove the replacement text in. + text = text.substring(0, start_loc) + + this.diff_text2(patches[x].diffs) + + text.substring(start_loc + text1.length); + } else { + // Imperfect match. Run a diff to get a framework of equivalent + // indices. + var diffs = this.diff_main(text1, text2, false); + if (text1.length > this.Match_MaxBits && + this.diff_levenshtein(diffs) / text1.length > + this.Patch_DeleteThreshold) { + // The end points match, but the content is unacceptably bad. + results[x] = false; + } else { + this.diff_cleanupSemanticLossless(diffs); + var index1 = 0; + var index2; + for (var y = 0; y < patches[x].diffs.length; y++) { + var mod = patches[x].diffs[y]; + if (mod[0] !== DIFF_EQUAL) { + index2 = this.diff_xIndex(diffs, index1); + } + if (mod[0] === DIFF_INSERT) { // Insertion + text = text.substring(0, start_loc + index2) + mod[1] + + text.substring(start_loc + index2); + } else if (mod[0] === DIFF_DELETE) { // Deletion + text = text.substring(0, start_loc + index2) + + text.substring(start_loc + this.diff_xIndex(diffs, + index1 + mod[1].length)); + } + if (mod[0] !== DIFF_DELETE) { + index1 += mod[1].length; + } + } + } + } + } + } + // Strip the padding off. + text = text.substring(nullPadding.length, text.length - nullPadding.length); + return [text, results]; +}; + + +/** + * Add some padding on text start and end so that edges can match something. + * Intended to be called only from within patch_apply. + * @param {!Array.} patches Array of Patch objects. + * @return {string} The padding string added to each side. + */ +diff_match_patch.prototype.patch_addPadding = function(patches) { + var paddingLength = this.Patch_Margin; + var nullPadding = ''; + for (var x = 1; x <= paddingLength; x++) { + nullPadding += String.fromCharCode(x); + } + + // Bump all the patches forward. + for (var x = 0; x < patches.length; x++) { + patches[x].start1 += paddingLength; + patches[x].start2 += paddingLength; + } + + // Add some padding on start of first diff. + var patch = patches[0]; + var diffs = patch.diffs; + if (diffs.length == 0 || diffs[0][0] != DIFF_EQUAL) { + // Add nullPadding equality. + diffs.unshift(new diff_match_patch.Diff(DIFF_EQUAL, nullPadding)); + patch.start1 -= paddingLength; // Should be 0. + patch.start2 -= paddingLength; // Should be 0. + patch.length1 += paddingLength; + patch.length2 += paddingLength; + } else if (paddingLength > diffs[0][1].length) { + // Grow first equality. + var extraLength = paddingLength - diffs[0][1].length; + diffs[0][1] = nullPadding.substring(diffs[0][1].length) + diffs[0][1]; + patch.start1 -= extraLength; + patch.start2 -= extraLength; + patch.length1 += extraLength; + patch.length2 += extraLength; + } + + // Add some padding on end of last diff. + patch = patches[patches.length - 1]; + diffs = patch.diffs; + if (diffs.length == 0 || diffs[diffs.length - 1][0] != DIFF_EQUAL) { + // Add nullPadding equality. + diffs.push(new diff_match_patch.Diff(DIFF_EQUAL, nullPadding)); + patch.length1 += paddingLength; + patch.length2 += paddingLength; + } else if (paddingLength > diffs[diffs.length - 1][1].length) { + // Grow last equality. + var extraLength = paddingLength - diffs[diffs.length - 1][1].length; + diffs[diffs.length - 1][1] += nullPadding.substring(0, extraLength); + patch.length1 += extraLength; + patch.length2 += extraLength; + } + + return nullPadding; +}; + + +/** + * Look through the patches and break up any which are longer than the maximum + * limit of the match algorithm. + * Intended to be called only from within patch_apply. + * @param {!Array.} patches Array of Patch objects. + */ +diff_match_patch.prototype.patch_splitMax = function(patches) { + var patch_size = this.Match_MaxBits; + for (var x = 0; x < patches.length; x++) { + if (patches[x].length1 <= patch_size) { + continue; + } + var bigpatch = patches[x]; + // Remove the big old patch. + patches.splice(x--, 1); + var start1 = bigpatch.start1; + var start2 = bigpatch.start2; + var precontext = ''; + while (bigpatch.diffs.length !== 0) { + // Create one of several smaller patches. + var patch = new diff_match_patch.patch_obj(); + var empty = true; + patch.start1 = start1 - precontext.length; + patch.start2 = start2 - precontext.length; + if (precontext !== '') { + patch.length1 = patch.length2 = precontext.length; + patch.diffs.push(new diff_match_patch.Diff(DIFF_EQUAL, precontext)); + } + while (bigpatch.diffs.length !== 0 && + patch.length1 < patch_size - this.Patch_Margin) { + var diff_type = bigpatch.diffs[0][0]; + var diff_text = bigpatch.diffs[0][1]; + if (diff_type === DIFF_INSERT) { + // Insertions are harmless. + patch.length2 += diff_text.length; + start2 += diff_text.length; + patch.diffs.push(bigpatch.diffs.shift()); + empty = false; + } else if (diff_type === DIFF_DELETE && patch.diffs.length == 1 && + patch.diffs[0][0] == DIFF_EQUAL && + diff_text.length > 2 * patch_size) { + // This is a large deletion. Let it pass in one chunk. + patch.length1 += diff_text.length; + start1 += diff_text.length; + empty = false; + patch.diffs.push(new diff_match_patch.Diff(diff_type, diff_text)); + bigpatch.diffs.shift(); + } else { + // Deletion or equality. Only take as much as we can stomach. + diff_text = diff_text.substring(0, + patch_size - patch.length1 - this.Patch_Margin); + patch.length1 += diff_text.length; + start1 += diff_text.length; + if (diff_type === DIFF_EQUAL) { + patch.length2 += diff_text.length; + start2 += diff_text.length; + } else { + empty = false; + } + patch.diffs.push(new diff_match_patch.Diff(diff_type, diff_text)); + if (diff_text == bigpatch.diffs[0][1]) { + bigpatch.diffs.shift(); + } else { + bigpatch.diffs[0][1] = + bigpatch.diffs[0][1].substring(diff_text.length); + } + } + } + // Compute the head context for the next patch. + precontext = this.diff_text2(patch.diffs); + precontext = + precontext.substring(precontext.length - this.Patch_Margin); + // Append the end context for this patch. + var postcontext = this.diff_text1(bigpatch.diffs) + .substring(0, this.Patch_Margin); + if (postcontext !== '') { + patch.length1 += postcontext.length; + patch.length2 += postcontext.length; + if (patch.diffs.length !== 0 && + patch.diffs[patch.diffs.length - 1][0] === DIFF_EQUAL) { + patch.diffs[patch.diffs.length - 1][1] += postcontext; + } else { + patch.diffs.push(new diff_match_patch.Diff(DIFF_EQUAL, postcontext)); + } + } + if (!empty) { + patches.splice(++x, 0, patch); + } + } + } +}; + + +/** + * Take a list of patches and return a textual representation. + * @param {!Array.} patches Array of Patch objects. + * @return {string} Text representation of patches. + */ +diff_match_patch.prototype.patch_toText = function(patches) { + var text = []; + for (var x = 0; x < patches.length; x++) { + text[x] = patches[x]; + } + return text.join(''); +}; + + +/** + * Parse a textual representation of patches and return a list of Patch objects. + * @param {string} textline Text representation of patches. + * @return {!Array.} Array of Patch objects. + * @throws {!Error} If invalid input. + */ +diff_match_patch.prototype.patch_fromText = function(textline) { + var patches = []; + if (!textline) { + return patches; + } + var text = textline.split('\n'); + var textPointer = 0; + var patchHeader = /^@@ -(\d+),?(\d*) \+(\d+),?(\d*) @@$/; + while (textPointer < text.length) { + var m = text[textPointer].match(patchHeader); + if (!m) { + throw new Error('Invalid patch string: ' + text[textPointer]); + } + var patch = new diff_match_patch.patch_obj(); + patches.push(patch); + patch.start1 = parseInt(m[1], 10); + if (m[2] === '') { + patch.start1--; + patch.length1 = 1; + } else if (m[2] == '0') { + patch.length1 = 0; + } else { + patch.start1--; + patch.length1 = parseInt(m[2], 10); + } + + patch.start2 = parseInt(m[3], 10); + if (m[4] === '') { + patch.start2--; + patch.length2 = 1; + } else if (m[4] == '0') { + patch.length2 = 0; + } else { + patch.start2--; + patch.length2 = parseInt(m[4], 10); + } + textPointer++; + + while (textPointer < text.length) { + var sign = text[textPointer].charAt(0); + try { + var line = decodeURI(text[textPointer].substring(1)); + } catch (ex) { + // Malformed URI sequence. + throw new Error('Illegal escape in patch_fromText: ' + line); + } + if (sign == '-') { + // Deletion. + patch.diffs.push(new diff_match_patch.Diff(DIFF_DELETE, line)); + } else if (sign == '+') { + // Insertion. + patch.diffs.push(new diff_match_patch.Diff(DIFF_INSERT, line)); + } else if (sign == ' ') { + // Minor equality. + patch.diffs.push(new diff_match_patch.Diff(DIFF_EQUAL, line)); + } else if (sign == '@') { + // Start of next patch. + break; + } else if (sign === '') { + // Blank line? Whatever. + } else { + // WTF? + throw new Error('Invalid patch mode "' + sign + '" in: ' + line); + } + textPointer++; + } + } + return patches; +}; + + +/** + * Class representing one patch operation. + * @constructor + */ +diff_match_patch.patch_obj = function() { + /** @type {!Array.} */ + this.diffs = []; + /** @type {?number} */ + this.start1 = null; + /** @type {?number} */ + this.start2 = null; + /** @type {number} */ + this.length1 = 0; + /** @type {number} */ + this.length2 = 0; +}; + + +/** + * Emulate GNU diff's format. + * Header: @@ -382,8 +481,9 @@ + * Indices are printed as 1-based, not 0-based. + * @return {string} The GNU diff string. + */ +diff_match_patch.patch_obj.prototype.toString = function() { + var coords1, coords2; + if (this.length1 === 0) { + coords1 = this.start1 + ',0'; + } else if (this.length1 == 1) { + coords1 = this.start1 + 1; + } else { + coords1 = (this.start1 + 1) + ',' + this.length1; + } + if (this.length2 === 0) { + coords2 = this.start2 + ',0'; + } else if (this.length2 == 1) { + coords2 = this.start2 + 1; + } else { + coords2 = (this.start2 + 1) + ',' + this.length2; + } + var text = ['@@ -' + coords1 + ' +' + coords2 + ' @@\n']; + var op; + // Escape the body of the patch with %xx notation. + for (var x = 0; x < this.diffs.length; x++) { + switch (this.diffs[x][0]) { + case DIFF_INSERT: + op = '+'; + break; + case DIFF_DELETE: + op = '-'; + break; + case DIFF_EQUAL: + op = ' '; + break; + } + text[x + 1] = op + encodeURI(this.diffs[x][1]) + '\n'; + } + return text.join('').replace(/%20/g, ' '); +}; + +// CLOSURE:begin_strip +// Lines below here will not be included in the Closure-compatible library. + +// Export these global variables so that they survive Google's JS compiler. +// In a browser, 'this' will be 'window'. +// Users of node.js should 'require' the uncompressed version since Google's +// JS compiler may break the following exports for non-browser environments. +/** @suppress {globalThis} */ +this['diff_match_patch'] = diff_match_patch; +/** @suppress {globalThis} */ +this['DIFF_DELETE'] = DIFF_DELETE; +/** @suppress {globalThis} */ +this['DIFF_INSERT'] = DIFF_INSERT; +/** @suppress {globalThis} */ +this['DIFF_EQUAL'] = DIFF_EQUAL; diff --git a/resources/js/turndown.js b/resources/js/turndown.js new file mode 100644 index 0000000..cb9d04d --- /dev/null +++ b/resources/js/turndown.js @@ -0,0 +1,977 @@ +var TurndownService = (function () { + 'use strict'; + + function extend (destination) { + for (var i = 1; i < arguments.length; i++) { + var source = arguments[i]; + for (var key in source) { + if (source.hasOwnProperty(key)) destination[key] = source[key]; + } + } + return destination + } + + function repeat (character, count) { + return Array(count + 1).join(character) + } + + function trimLeadingNewlines (string) { + return string.replace(/^\n*/, '') + } + + function trimTrailingNewlines (string) { + // avoid match-at-end regexp bottleneck, see #370 + var indexEnd = string.length; + while (indexEnd > 0 && string[indexEnd - 1] === '\n') indexEnd--; + return string.substring(0, indexEnd) + } + + var blockElements = [ + 'ADDRESS', 'ARTICLE', 'ASIDE', 'AUDIO', 'BLOCKQUOTE', 'BODY', 'CANVAS', + 'CENTER', 'DD', 'DIR', 'DIV', 'DL', 'DT', 'FIELDSET', 'FIGCAPTION', 'FIGURE', + 'FOOTER', 'FORM', 'FRAMESET', 'H1', 'H2', 'H3', 'H4', 'H5', 'H6', 'HEADER', + 'HGROUP', 'HR', 'HTML', 'ISINDEX', 'LI', 'MAIN', 'MENU', 'NAV', 'NOFRAMES', + 'NOSCRIPT', 'OL', 'OUTPUT', 'P', 'PRE', 'SECTION', 'TABLE', 'TBODY', 'TD', + 'TFOOT', 'TH', 'THEAD', 'TR', 'UL' + ]; + + function isBlock (node) { + return is(node, blockElements) + } + + var voidElements = [ + 'AREA', 'BASE', 'BR', 'COL', 'COMMAND', 'EMBED', 'HR', 'IMG', 'INPUT', + 'KEYGEN', 'LINK', 'META', 'PARAM', 'SOURCE', 'TRACK', 'WBR' + ]; + + function isVoid (node) { + return is(node, voidElements) + } + + function hasVoid (node) { + return has(node, voidElements) + } + + var meaningfulWhenBlankElements = [ + 'A', 'TABLE', 'THEAD', 'TBODY', 'TFOOT', 'TH', 'TD', 'IFRAME', 'SCRIPT', + 'AUDIO', 'VIDEO' + ]; + + function isMeaningfulWhenBlank (node) { + return is(node, meaningfulWhenBlankElements) + } + + function hasMeaningfulWhenBlank (node) { + return has(node, meaningfulWhenBlankElements) + } + + function is (node, tagNames) { + return tagNames.indexOf(node.nodeName) >= 0 + } + + function has (node, tagNames) { + return ( + node.getElementsByTagName && + tagNames.some(function (tagName) { + return node.getElementsByTagName(tagName).length + }) + ) + } + + var rules = {}; + + rules.paragraph = { + filter: 'p', + + replacement: function (content) { + return '\n\n' + content + '\n\n' + } + }; + + rules.lineBreak = { + filter: 'br', + + replacement: function (content, node, options) { + return options.br + '\n' + } + }; + + rules.heading = { + filter: ['h1', 'h2', 'h3', 'h4', 'h5', 'h6'], + + replacement: function (content, node, options) { + var hLevel = Number(node.nodeName.charAt(1)); + + if (options.headingStyle === 'setext' && hLevel < 3) { + var underline = repeat((hLevel === 1 ? '=' : '-'), content.length); + return ( + '\n\n' + content + '\n' + underline + '\n\n' + ) + } else { + return '\n\n' + repeat('#', hLevel) + ' ' + content + '\n\n' + } + } + }; + + rules.blockquote = { + filter: 'blockquote', + + replacement: function (content) { + content = content.replace(/^\n+|\n+$/g, ''); + content = content.replace(/^/gm, '> '); + return '\n\n' + content + '\n\n' + } + }; + + rules.list = { + filter: ['ul', 'ol'], + + replacement: function (content, node) { + var parent = node.parentNode; + if (parent.nodeName === 'LI' && parent.lastElementChild === node) { + return '\n' + content + } else { + return '\n\n' + content + '\n\n' + } + } + }; + + rules.listItem = { + filter: 'li', + + replacement: function (content, node, options) { + content = content + .replace(/^\n+/, '') // remove leading newlines + .replace(/\n+$/, '\n') // replace trailing newlines with just a single one + .replace(/\n/gm, '\n '); // indent + var prefix = options.bulletListMarker + ' '; + var parent = node.parentNode; + if (parent.nodeName === 'OL') { + var start = parent.getAttribute('start'); + var index = Array.prototype.indexOf.call(parent.children, node); + prefix = (start ? Number(start) + index : index + 1) + '. '; + } + return ( + prefix + content + (node.nextSibling && !/\n$/.test(content) ? '\n' : '') + ) + } + }; + + rules.indentedCodeBlock = { + filter: function (node, options) { + return ( + options.codeBlockStyle === 'indented' && + node.nodeName === 'PRE' && + node.firstChild && + node.firstChild.nodeName === 'CODE' + ) + }, + + replacement: function (content, node, options) { + return ( + '\n\n ' + + node.firstChild.textContent.replace(/\n/g, '\n ') + + '\n\n' + ) + } + }; + + rules.fencedCodeBlock = { + filter: function (node, options) { + return ( + options.codeBlockStyle === 'fenced' && + node.nodeName === 'PRE' && + node.firstChild && + node.firstChild.nodeName === 'CODE' + ) + }, + + replacement: function (content, node, options) { + var className = node.firstChild.getAttribute('class') || ''; + var language = (className.match(/language-(\S+)/) || [null, ''])[1]; + var code = node.firstChild.textContent; + + var fenceChar = options.fence.charAt(0); + var fenceSize = 3; + var fenceInCodeRegex = new RegExp('^' + fenceChar + '{3,}', 'gm'); + + var match; + while ((match = fenceInCodeRegex.exec(code))) { + if (match[0].length >= fenceSize) { + fenceSize = match[0].length + 1; + } + } + + var fence = repeat(fenceChar, fenceSize); + + return ( + '\n\n' + fence + language + '\n' + + code.replace(/\n$/, '') + + '\n' + fence + '\n\n' + ) + } + }; + + rules.horizontalRule = { + filter: 'hr', + + replacement: function (content, node, options) { + return '\n\n' + options.hr + '\n\n' + } + }; + + rules.inlineLink = { + filter: function (node, options) { + return ( + options.linkStyle === 'inlined' && + node.nodeName === 'A' && + node.getAttribute('href') + ) + }, + + replacement: function (content, node) { + var href = node.getAttribute('href'); + if (href) href = href.replace(/([()])/g, '\\$1'); + var title = cleanAttribute(node.getAttribute('title')); + if (title) title = ' "' + title.replace(/"/g, '\\"') + '"'; + return '[' + content + '](' + href + title + ')' + } + }; + + rules.referenceLink = { + filter: function (node, options) { + return ( + options.linkStyle === 'referenced' && + node.nodeName === 'A' && + node.getAttribute('href') + ) + }, + + replacement: function (content, node, options) { + var href = node.getAttribute('href'); + var title = cleanAttribute(node.getAttribute('title')); + if (title) title = ' "' + title + '"'; + var replacement; + var reference; + + switch (options.linkReferenceStyle) { + case 'collapsed': + replacement = '[' + content + '][]'; + reference = '[' + content + ']: ' + href + title; + break + case 'shortcut': + replacement = '[' + content + ']'; + reference = '[' + content + ']: ' + href + title; + break + default: + var id = this.references.length + 1; + replacement = '[' + content + '][' + id + ']'; + reference = '[' + id + ']: ' + href + title; + } + + this.references.push(reference); + return replacement + }, + + references: [], + + append: function (options) { + var references = ''; + if (this.references.length) { + references = '\n\n' + this.references.join('\n') + '\n\n'; + this.references = []; // Reset references + } + return references + } + }; + + rules.emphasis = { + filter: ['em', 'i'], + + replacement: function (content, node, options) { + if (!content.trim()) return '' + return options.emDelimiter + content + options.emDelimiter + } + }; + + rules.strong = { + filter: ['strong', 'b'], + + replacement: function (content, node, options) { + if (!content.trim()) return '' + return options.strongDelimiter + content + options.strongDelimiter + } + }; + + rules.code = { + filter: function (node) { + var hasSiblings = node.previousSibling || node.nextSibling; + var isCodeBlock = node.parentNode.nodeName === 'PRE' && !hasSiblings; + + return node.nodeName === 'CODE' && !isCodeBlock + }, + + replacement: function (content) { + if (!content) return '' + content = content.replace(/\r?\n|\r/g, ' '); + + var extraSpace = /^`|^ .*?[^ ].* $|`$/.test(content) ? ' ' : ''; + var delimiter = '`'; + var matches = content.match(/`+/gm) || []; + while (matches.indexOf(delimiter) !== -1) delimiter = delimiter + '`'; + + return delimiter + extraSpace + content + extraSpace + delimiter + } + }; + + rules.image = { + filter: 'img', + + replacement: function (content, node) { + var alt = cleanAttribute(node.getAttribute('alt')); + var src = node.getAttribute('src') || ''; + var title = cleanAttribute(node.getAttribute('title')); + var titlePart = title ? ' "' + title + '"' : ''; + return src ? '![' + alt + ']' + '(' + src + titlePart + ')' : '' + } + }; + + function cleanAttribute (attribute) { + return attribute ? attribute.replace(/(\n+\s*)+/g, '\n') : '' + } + + /** + * Manages a collection of rules used to convert HTML to Markdown + */ + + function Rules (options) { + this.options = options; + this._keep = []; + this._remove = []; + + this.blankRule = { + replacement: options.blankReplacement + }; + + this.keepReplacement = options.keepReplacement; + + this.defaultRule = { + replacement: options.defaultReplacement + }; + + this.array = []; + for (var key in options.rules) this.array.push(options.rules[key]); + } + + Rules.prototype = { + add: function (key, rule) { + this.array.unshift(rule); + }, + + keep: function (filter) { + this._keep.unshift({ + filter: filter, + replacement: this.keepReplacement + }); + }, + + remove: function (filter) { + this._remove.unshift({ + filter: filter, + replacement: function () { + return '' + } + }); + }, + + forNode: function (node) { + if (node.isBlank) return this.blankRule + var rule; + + if ((rule = findRule(this.array, node, this.options))) return rule + if ((rule = findRule(this._keep, node, this.options))) return rule + if ((rule = findRule(this._remove, node, this.options))) return rule + + return this.defaultRule + }, + + forEach: function (fn) { + for (var i = 0; i < this.array.length; i++) fn(this.array[i], i); + } + }; + + function findRule (rules, node, options) { + for (var i = 0; i < rules.length; i++) { + var rule = rules[i]; + if (filterValue(rule, node, options)) return rule + } + return void 0 + } + + function filterValue (rule, node, options) { + var filter = rule.filter; + if (typeof filter === 'string') { + if (filter === node.nodeName.toLowerCase()) return true + } else if (Array.isArray(filter)) { + if (filter.indexOf(node.nodeName.toLowerCase()) > -1) return true + } else if (typeof filter === 'function') { + if (filter.call(rule, node, options)) return true + } else { + throw new TypeError('`filter` needs to be a string, array, or function') + } + } + + /** + * The collapseWhitespace function is adapted from collapse-whitespace + * by Luc Thevenard. + * + * The MIT License (MIT) + * + * Copyright (c) 2014 Luc Thevenard + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + + /** + * collapseWhitespace(options) removes extraneous whitespace from an the given element. + * + * @param {Object} options + */ + function collapseWhitespace (options) { + var element = options.element; + var isBlock = options.isBlock; + var isVoid = options.isVoid; + var isPre = options.isPre || function (node) { + return node.nodeName === 'PRE' + }; + + if (!element.firstChild || isPre(element)) return + + var prevText = null; + var keepLeadingWs = false; + + var prev = null; + var node = next(prev, element, isPre); + + while (node !== element) { + if (node.nodeType === 3 || node.nodeType === 4) { // Node.TEXT_NODE or Node.CDATA_SECTION_NODE + var text = node.data.replace(/[ \r\n\t]+/g, ' '); + + if ((!prevText || / $/.test(prevText.data)) && + !keepLeadingWs && text[0] === ' ') { + text = text.substr(1); + } + + // `text` might be empty at this point. + if (!text) { + node = remove(node); + continue + } + + node.data = text; + + prevText = node; + } else if (node.nodeType === 1) { // Node.ELEMENT_NODE + if (isBlock(node) || node.nodeName === 'BR') { + if (prevText) { + prevText.data = prevText.data.replace(/ $/, ''); + } + + prevText = null; + keepLeadingWs = false; + } else if (isVoid(node) || isPre(node)) { + // Avoid trimming space around non-block, non-BR void elements and inline PRE. + prevText = null; + keepLeadingWs = true; + } else if (prevText) { + // Drop protection if set previously. + keepLeadingWs = false; + } + } else { + node = remove(node); + continue + } + + var nextNode = next(prev, node, isPre); + prev = node; + node = nextNode; + } + + if (prevText) { + prevText.data = prevText.data.replace(/ $/, ''); + if (!prevText.data) { + remove(prevText); + } + } + } + + /** + * remove(node) removes the given node from the DOM and returns the + * next node in the sequence. + * + * @param {Node} node + * @return {Node} node + */ + function remove (node) { + var next = node.nextSibling || node.parentNode; + + node.parentNode.removeChild(node); + + return next + } + + /** + * next(prev, current, isPre) returns the next node in the sequence, given the + * current and previous nodes. + * + * @param {Node} prev + * @param {Node} current + * @param {Function} isPre + * @return {Node} + */ + function next (prev, current, isPre) { + if ((prev && prev.parentNode === current) || isPre(current)) { + return current.nextSibling || current.parentNode + } + + return current.firstChild || current.nextSibling || current.parentNode + } + + /* + * Set up window for Node.js + */ + + var root = (typeof window !== 'undefined' ? window : {}); + + /* + * Parsing HTML strings + */ + + function canParseHTMLNatively () { + var Parser = root.DOMParser; + var canParse = false; + + // Adapted from https://gist.github.com/1129031 + // Firefox/Opera/IE throw errors on unsupported types + try { + // WebKit returns null on unsupported types + if (new Parser().parseFromString('', 'text/html')) { + canParse = true; + } + } catch (e) {} + + return canParse + } + + function createHTMLParser () { + var Parser = function () {}; + + { + if (shouldUseActiveX()) { + Parser.prototype.parseFromString = function (string) { + var doc = new window.ActiveXObject('htmlfile'); + doc.designMode = 'on'; // disable on-page scripts + doc.open(); + doc.write(string); + doc.close(); + return doc + }; + } else { + Parser.prototype.parseFromString = function (string) { + var doc = document.implementation.createHTMLDocument(''); + doc.open(); + doc.write(string); + doc.close(); + return doc + }; + } + } + return Parser + } + + function shouldUseActiveX () { + var useActiveX = false; + try { + document.implementation.createHTMLDocument('').open(); + } catch (e) { + if (root.ActiveXObject) useActiveX = true; + } + return useActiveX + } + + var HTMLParser = canParseHTMLNatively() ? root.DOMParser : createHTMLParser(); + + function RootNode (input, options) { + var root; + if (typeof input === 'string') { + var doc = htmlParser().parseFromString( + // DOM parsers arrange elements in the and . + // Wrapping in a custom element ensures elements are reliably arranged in + // a single element. + '' + input + '', + 'text/html' + ); + root = doc.getElementById('turndown-root'); + } else { + root = input.cloneNode(true); + } + collapseWhitespace({ + element: root, + isBlock: isBlock, + isVoid: isVoid, + isPre: options.preformattedCode ? isPreOrCode : null + }); + + return root + } + + var _htmlParser; + function htmlParser () { + _htmlParser = _htmlParser || new HTMLParser(); + return _htmlParser + } + + function isPreOrCode (node) { + return node.nodeName === 'PRE' || node.nodeName === 'CODE' + } + + function Node (node, options) { + node.isBlock = isBlock(node); + node.isCode = node.nodeName === 'CODE' || node.parentNode.isCode; + node.isBlank = isBlank(node); + node.flankingWhitespace = flankingWhitespace(node, options); + return node + } + + function isBlank (node) { + return ( + !isVoid(node) && + !isMeaningfulWhenBlank(node) && + /^\s*$/i.test(node.textContent) && + !hasVoid(node) && + !hasMeaningfulWhenBlank(node) + ) + } + + function flankingWhitespace (node, options) { + if (node.isBlock || (options.preformattedCode && node.isCode)) { + return { leading: '', trailing: '' } + } + + var edges = edgeWhitespace(node.textContent); + + // abandon leading ASCII WS if left-flanked by ASCII WS + if (edges.leadingAscii && isFlankedByWhitespace('left', node, options)) { + edges.leading = edges.leadingNonAscii; + } + + // abandon trailing ASCII WS if right-flanked by ASCII WS + if (edges.trailingAscii && isFlankedByWhitespace('right', node, options)) { + edges.trailing = edges.trailingNonAscii; + } + + return { leading: edges.leading, trailing: edges.trailing } + } + + function edgeWhitespace (string) { + var m = string.match(/^(([ \t\r\n]*)(\s*))(?:(?=\S)[\s\S]*\S)?((\s*?)([ \t\r\n]*))$/); + return { + leading: m[1], // whole string for whitespace-only strings + leadingAscii: m[2], + leadingNonAscii: m[3], + trailing: m[4], // empty for whitespace-only strings + trailingNonAscii: m[5], + trailingAscii: m[6] + } + } + + function isFlankedByWhitespace (side, node, options) { + var sibling; + var regExp; + var isFlanked; + + if (side === 'left') { + sibling = node.previousSibling; + regExp = / $/; + } else { + sibling = node.nextSibling; + regExp = /^ /; + } + + if (sibling) { + if (sibling.nodeType === 3) { + isFlanked = regExp.test(sibling.nodeValue); + } else if (options.preformattedCode && sibling.nodeName === 'CODE') { + isFlanked = false; + } else if (sibling.nodeType === 1 && !isBlock(sibling)) { + isFlanked = regExp.test(sibling.textContent); + } + } + return isFlanked + } + + var reduce = Array.prototype.reduce; + var escapes = [ + [/\\/g, '\\\\'], + [/\*/g, '\\*'], + [/^-/g, '\\-'], + [/^\+ /g, '\\+ '], + [/^(=+)/g, '\\$1'], + [/^(#{1,6}) /g, '\\$1 '], + [/`/g, '\\`'], + [/^~~~/g, '\\~~~'], + [/\[/g, '\\['], + [/\]/g, '\\]'], + [/^>/g, '\\>'], + [/_/g, '\\_'], + [/^(\d+)\. /g, '$1\\. '] + ]; + + function TurndownService (options) { + if (!(this instanceof TurndownService)) return new TurndownService(options) + + var defaults = { + rules: rules, + headingStyle: 'setext', + hr: '* * *', + bulletListMarker: '*', + codeBlockStyle: 'indented', + fence: '```', + emDelimiter: '_', + strongDelimiter: '**', + linkStyle: 'inlined', + linkReferenceStyle: 'full', + br: ' ', + preformattedCode: false, + blankReplacement: function (content, node) { + return node.isBlock ? '\n\n' : '' + }, + keepReplacement: function (content, node) { + return node.isBlock ? '\n\n' + node.outerHTML + '\n\n' : node.outerHTML + }, + defaultReplacement: function (content, node) { + return node.isBlock ? '\n\n' + content + '\n\n' : content + } + }; + this.options = extend({}, defaults, options); + this.rules = new Rules(this.options); + } + + TurndownService.prototype = { + /** + * The entry point for converting a string or DOM node to Markdown + * @public + * @param {String|HTMLElement} input The string or DOM node to convert + * @returns A Markdown representation of the input + * @type String + */ + + turndown: function (input) { + if (!canConvert(input)) { + throw new TypeError( + input + ' is not a string, or an element/document/fragment node.' + ) + } + + if (input === '') return '' + + var output = process.call(this, new RootNode(input, this.options)); + return postProcess.call(this, output) + }, + + /** + * Add one or more plugins + * @public + * @param {Function|Array} plugin The plugin or array of plugins to add + * @returns The Turndown instance for chaining + * @type Object + */ + + use: function (plugin) { + if (Array.isArray(plugin)) { + for (var i = 0; i < plugin.length; i++) this.use(plugin[i]); + } else if (typeof plugin === 'function') { + plugin(this); + } else { + throw new TypeError('plugin must be a Function or an Array of Functions') + } + return this + }, + + /** + * Adds a rule + * @public + * @param {String} key The unique key of the rule + * @param {Object} rule The rule + * @returns The Turndown instance for chaining + * @type Object + */ + + addRule: function (key, rule) { + this.rules.add(key, rule); + return this + }, + + /** + * Keep a node (as HTML) that matches the filter + * @public + * @param {String|Array|Function} filter The unique key of the rule + * @returns The Turndown instance for chaining + * @type Object + */ + + keep: function (filter) { + this.rules.keep(filter); + return this + }, + + /** + * Remove a node that matches the filter + * @public + * @param {String|Array|Function} filter The unique key of the rule + * @returns The Turndown instance for chaining + * @type Object + */ + + remove: function (filter) { + this.rules.remove(filter); + return this + }, + + /** + * Escapes Markdown syntax + * @public + * @param {String} string The string to escape + * @returns A string with Markdown syntax escaped + * @type String + */ + + escape: function (string) { + return escapes.reduce(function (accumulator, escape) { + return accumulator.replace(escape[0], escape[1]) + }, string) + } + }; + + /** + * Reduces a DOM node down to its Markdown string equivalent + * @private + * @param {HTMLElement} parentNode The node to convert + * @returns A Markdown representation of the node + * @type String + */ + + function process (parentNode) { + var self = this; + return reduce.call(parentNode.childNodes, function (output, node) { + node = new Node(node, self.options); + + var replacement = ''; + if (node.nodeType === 3) { + replacement = node.isCode ? node.nodeValue : self.escape(node.nodeValue); + } else if (node.nodeType === 1) { + replacement = replacementForNode.call(self, node); + } + + return join(output, replacement) + }, '') + } + + /** + * Appends strings as each rule requires and trims the output + * @private + * @param {String} output The conversion output + * @returns A trimmed version of the ouput + * @type String + */ + + function postProcess (output) { + var self = this; + this.rules.forEach(function (rule) { + if (typeof rule.append === 'function') { + output = join(output, rule.append(self.options)); + } + }); + + return output.replace(/^[\t\r\n]+/, '').replace(/[\t\r\n\s]+$/, '') + } + + /** + * Converts an element node to its Markdown equivalent + * @private + * @param {HTMLElement} node The node to convert + * @returns A Markdown representation of the node + * @type String + */ + + function replacementForNode (node) { + var rule = this.rules.forNode(node); + var content = process.call(this, node); + var whitespace = node.flankingWhitespace; + if (whitespace.leading || whitespace.trailing) content = content.trim(); + return ( + whitespace.leading + + rule.replacement(content, node, this.options) + + whitespace.trailing + ) + } + + /** + * Joins replacement to the current output with appropriate number of new lines + * @private + * @param {String} output The current conversion output + * @param {String} replacement The string to append to the output + * @returns Joined output + * @type String + */ + + function join (output, replacement) { + var s1 = trimTrailingNewlines(output); + var s2 = trimLeadingNewlines(replacement); + var nls = Math.max(output.length - s1.length, replacement.length - s2.length); + var separator = '\n\n'.substring(0, nls); + + return s1 + separator + s2 + } + + /** + * Determines whether an input can be converted + * @private + * @param {String|HTMLElement} input Describe this parameter + * @returns Describe what it returns + * @type String|Object|Array|Boolean|Number + */ + + function canConvert (input) { + return ( + input != null && ( + typeof input === 'string' || + (input.nodeType && ( + input.nodeType === 1 || input.nodeType === 9 || input.nodeType === 11 + )) + ) + ) + } + + return TurndownService; + +}()); + +export { TurndownService }; +export default TurndownService; diff --git a/resources/svg/average.svg b/resources/svg/average.svg new file mode 100644 index 0000000..b427612 --- /dev/null +++ b/resources/svg/average.svg @@ -0,0 +1,3 @@ + + + diff --git a/resources/svg/check.svg b/resources/svg/check.svg new file mode 100644 index 0000000..71b981c --- /dev/null +++ b/resources/svg/check.svg @@ -0,0 +1,3 @@ + + + diff --git a/resources/svg/circle.svg b/resources/svg/circle.svg new file mode 100644 index 0000000..84f078d --- /dev/null +++ b/resources/svg/circle.svg @@ -0,0 +1,3 @@ + + + diff --git a/resources/svg/circledot.svg b/resources/svg/circledot.svg new file mode 100644 index 0000000..e380e09 --- /dev/null +++ b/resources/svg/circledot.svg @@ -0,0 +1,4 @@ + + + + diff --git a/resources/svg/circledots.svg b/resources/svg/circledots.svg new file mode 100644 index 0000000..6275a98 --- /dev/null +++ b/resources/svg/circledots.svg @@ -0,0 +1,4 @@ + + + + diff --git a/resources/svg/circleslash.svg b/resources/svg/circleslash.svg new file mode 100644 index 0000000..861352d --- /dev/null +++ b/resources/svg/circleslash.svg @@ -0,0 +1,3 @@ + + + diff --git a/resources/svg/clipboarddata.svg b/resources/svg/clipboarddata.svg new file mode 100644 index 0000000..aebd8f5 --- /dev/null +++ b/resources/svg/clipboarddata.svg @@ -0,0 +1,3 @@ + + + diff --git a/resources/svg/download.svg b/resources/svg/download.svg new file mode 100644 index 0000000..595f3a5 --- /dev/null +++ b/resources/svg/download.svg @@ -0,0 +1,4 @@ + + + + diff --git a/resources/svg/eye.svg b/resources/svg/eye.svg new file mode 100644 index 0000000..2f1f62f --- /dev/null +++ b/resources/svg/eye.svg @@ -0,0 +1,4 @@ + + + + diff --git a/resources/svg/flag.svg b/resources/svg/flag.svg new file mode 100644 index 0000000..ef18dc3 --- /dev/null +++ b/resources/svg/flag.svg @@ -0,0 +1,3 @@ + + + diff --git a/resources/svg/gear.svg b/resources/svg/gear.svg new file mode 100644 index 0000000..3827d6a --- /dev/null +++ b/resources/svg/gear.svg @@ -0,0 +1,11 @@ + + + + + + + + + + + diff --git a/resources/svg/plus.svg b/resources/svg/plus.svg new file mode 100644 index 0000000..e665b91 --- /dev/null +++ b/resources/svg/plus.svg @@ -0,0 +1,3 @@ + + + diff --git a/resources/svg/reply.svg b/resources/svg/reply.svg new file mode 100644 index 0000000..fdf9414 --- /dev/null +++ b/resources/svg/reply.svg @@ -0,0 +1,4 @@ + + + + diff --git a/resources/svg/settings.svg b/resources/svg/settings.svg new file mode 100644 index 0000000..ce5be79 --- /dev/null +++ b/resources/svg/settings.svg @@ -0,0 +1,3 @@ + + + diff --git a/resources/svg/trash.svg b/resources/svg/trash.svg new file mode 100644 index 0000000..c236be7 --- /dev/null +++ b/resources/svg/trash.svg @@ -0,0 +1,3 @@ + + + diff --git a/resources/svg/upload.svg b/resources/svg/upload.svg new file mode 100644 index 0000000..09e7ce4 --- /dev/null +++ b/resources/svg/upload.svg @@ -0,0 +1,4 @@ + + + + diff --git a/resources/svg/x.svg b/resources/svg/x.svg new file mode 100644 index 0000000..b209221 --- /dev/null +++ b/resources/svg/x.svg @@ -0,0 +1,3 @@ + + + diff --git a/resources/svg2img.ps1 b/resources/svg2img.ps1 new file mode 100644 index 0000000..d1a0760 --- /dev/null +++ b/resources/svg2img.ps1 @@ -0,0 +1,65 @@ +$svgDir = "./svg" +$outDir = "./img" +$sizes = @(16, 32, 64) +$themes = @{ + "light" = "#000000" + "dark" = "#ffffff" +} +$tempSvg = "temp.svg" + +# Ensure output directory exists +if (!(Test-Path -Path $outDir)) { + New-Item -ItemType Directory -Path $outDir | Out-Null +} + +# Check for Inkscape +if (-not (Get-Command "inkscape" -ErrorAction SilentlyContinue)) { + Write-Error "Inkscape CLI is not installed or not in PATH. Please install it from https://inkscape.org/" + exit 1 +} + +# Helper: inject color into tag +function Inject-Color { + param ($original, $color) + $content = Get-Content $original -Raw + + if ($content -match ']*>') { + # Inject color style + $patched = $content -replace ']*?)>', "" + Set-Content -Path $tempSvg -Value $patched + } + else { + throw "Couldn't find tag to patch." + } +} + +# Process each SVG file +Get-ChildItem -Path $svgDir -Filter *.svg | ForEach-Object { + $svgPath = $_.FullName + $baseName = $_.BaseName + + foreach ($theme in $themes.Keys) { + $color = $themes[$theme] + + # Create themed temp SVG + Inject-Color $svgPath $color + + foreach ($size in $sizes) { + $outFile = Join-Path $outDir "$baseName-$theme-$size.png" + Write-Host "Exporting $outFile (color $color)..." + & inkscape $tempSvg ` + --export-type=png ` + --export-filename="$outFile" ` + --export-width=$size ` + --export-height=$size ` + --actions="export-do" + } + } + + # Cleanup + if (Test-Path $tempSvg) { + Remove-Item $tempSvg -Force + } +} + +Write-Host "Done generating light/dark themed PNGs."