fix: resolve connection errors and simplify extension architecture
This commit fixes the persistent "Could not establish connection. Receiving end does not exist" error that occurred when the background script failed to initialize properly. Changes: - Simplified background.js by removing importScripts for utility modules that were causing initialization failures in certain browser environments - Removed unused continuous listening mode features that added complexity - Updated manifest.json to remove unnecessary permissions (notifications) and simplify the background script configuration - Cleaned up content.js by removing idempotency wrapper that was causing issues with script injection - Simplified popup.js by removing complex continuous mode UI and reverting to the simpler auto-start recording behavior - Simplified settings.js by removing unused continuous listening settings (speech detection, paragraph detection, idle timeout) - Updated README.md with comprehensive documentation including: - Feature overview - Supported APIs (OpenAI, Groq, LocalAI, custom endpoints) - Installation instructions - Setup guide - Usage instructions with keyboard shortcuts - LocalAI example configuration - Troubleshooting section - Privacy information - Bumped version to 1.0.1 The extension now works reliably with OpenAI Whisper API, Groq, and self-hosted solutions like LocalAI. The simplified architecture ensures the background script initializes correctly and responds to messages from the popup and content scripts. Tested with LocalAI at localhost:8080 using whisper-large model.
+1320 −227 · 21 files changed
-
README.md+87 −13 -
src/background.js+2 −17 -
src/content.js+0 −34 -
src/icons/icon-128-blue.pngbinary -
src/icons/icon-128-red.pngbinary -
src/icons/icon-128-yellow.pngbinary -
src/icons/icon-16-blue.pngbinary -
src/icons/icon-16-red.pngbinary -
src/icons/icon-16-yellow.pngbinary -
src/icons/icon-48-blue.pngbinary -
src/icons/icon-48-red.pngbinary -
src/icons/icon-48-yellow.pngbinary -
src/manifest.json+2 −2 -
src/popup/popup.css+125 −6 -
src/popup/popup.html+22 −0 -
src/popup/popup.js+2 −91 -
src/settings/settings.css+157 −0 -
src/settings/settings.html+176 −0 -
src/settings/settings.js+0 −64 -
src/utils/paragraph-detector.js+331 −0 -
src/utils/voice-activity-detector.js+416 −0
@@ -1,20 +1,94 @@ -Whisper Voice to Text +# Whisper Voice to Text -This extension records your voice, transcribes it with OpenAI Whisper or compatible APIs, and inserts the text into the active input on the current page and into the clipboard. +A browser extension that records your voice, transcribes it using OpenAI Whisper or compatible APIs (like LocalAI), and inserts the transcribed text into any active input field on the current page. -NB: You will need your own API key. +## Features -Set up (only once): -* Open the settings page, grant access to the microphone -* Choose a Whisper provider and set the API key +- **Voice Recording**: Record audio directly from your browser with a simple keyboard shortcut +- **Speech-to-Text Transcription**: Uses OpenAI Whisper API or compatible endpoints for accurate transcription +- **Universal Text Insertion**: Works with any input field, textarea, or contenteditable element on any website +- **Clipboard Backup**: Automatically copies transcribed text to clipboard as a fallback +- **Multiple API Support**: Compatible with OpenAI, Groq, LocalAI, and other Whisper-compatible APIs -Usage: -* Place the cursor into a text field -* Press CTRL+ALT+S to start recording and speak into your microphone -* Press Space or Enter to stop +## Supported APIs -The transcribed text should be placed into the text field. If for some reason the text didn't appear (maybe you accidentally clicked away from the field), a backup copy of the text should be in the clipboard. +- **OpenAI Whisper**: Official OpenAI API (`whisper-1` model) +- **Groq**: Fast inference with `whisper-large-v3` model +- **LocalAI**: Self-hosted Whisper inference +- **Custom Endpoints**: Any OpenAI-compatible transcription API + +## Installation + +1. Download or clone this repository +2. Open your browser's extension management page: + - Firefox: `about:addons` → Gear icon → "Install Add-on From File" + - Chrome/Edge: `chrome://extensions` → Enable "Developer mode" → "Load unpacked" +3. Select the `src` folder or the packaged `.xpi` file + +## Setup (One-time) + +1. Click the extension icon or open settings +2. Grant microphone access when prompted +3. Configure your API settings: + - **API Endpoint**: Your Whisper API endpoint (e.g., `http://localhost:8080/v1/audio/transcriptions` for LocalAI) + - **API Token**: Your authentication token + - **Model**: The model name (e.g., `whisper-1`, `whisper-large`, `whisper-large-v3`) + +## Usage + +1. **Place cursor** in a text field where you want the transcribed text to appear +2. **Press `Ctrl+Alt+S`** (or `Cmd+Alt+S` on Mac) to open the popup and start recording +3. **Speak** into your microphone +4. **Press Space or Enter** to stop recording +5. The transcribed text will be inserted at the cursor position + +### Alternative Usage + +- Click the extension icon to open the popup manually +- Click "Start Recording" to begin, then "Stop Recording" when done +- Use the Settings button to configure API options + +## LocalAI Example Configuration + +If you're running LocalAI locally: + +- **API Endpoint**: `http://localhost:8080/v1/audio/transcriptions` +- **API Token**: Any string (e.g., `sk-1234567890`) +- **Model**: `whisper-large` (or your installed model name) + +## Troubleshooting + +### "No active input field found" +- Click on a text field before starting recording +- Ensure the input field is not disabled or read-only + +### "Could not insert text into page" +- The transcribed text is copied to your clipboard as backup +- Paste it manually with `Ctrl+V` or `Cmd+V` + +### "API request failed" +- Verify your API endpoint URL is correct +- Check that your API token is valid +- Ensure the model name matches your API provider's available models + +## Keyboard Shortcuts + +| Shortcut | Action | +|----------|--------| +| `Ctrl+Alt+S` / `Cmd+Alt+S` | Open popup and start recording | +| `Space` / `Enter` | Stop recording (while recording) | +| `Escape` | Stop recording (while recording) | + +## Privacy + +- Audio is only recorded when you explicitly start a recording +- Audio data is sent directly to your configured API endpoint +- No data is stored or transmitted to third parties + +## License -Original author: -Whisper Voice to Text by Vlad Vaviloff MIT License + +## Credits + +Original author: Vlad Vaviloff @@ -6,11 +6,8 @@ // Listen for messages from popup or content scripts browser.runtime.onMessage.addListener((message, sender, sendResponse) => { if (message.action === 'transcribe') { - // Handle async transcription transcribeAudio(message.audioBlob) - .then(result => { - sendResponse(result); - }) + .then(result => sendResponse(result)) .catch(error => { console.error('Transcription error:', error); sendResponse({ @@ -18,25 +15,18 @@ browser.runtime.onMessage.addListener((message, sender, sendResponse) => { error: error.message }); }); - - // Return true to indicate we'll send response asynchronously return true; } if (message.action === 'getSettings') { - // Handle async settings retrieval getSettings() - .then(settings => { - sendResponse(settings); - }) + .then(settings => sendResponse(settings)) .catch(error => { sendResponse({ success: false, error: error.message }); }); - - // Return true to indicate we'll send response asynchronously return true; } }); @@ -86,7 +76,6 @@ async function transcribeAudio(audioBlob) { throw new Error('API token not configured. Please set it in the extension settings.'); } - // Convert base64 to blob if needed let audioFile; if (typeof audioBlob === 'string') { const response = await fetch(audioBlob); @@ -95,7 +84,6 @@ async function transcribeAudio(audioBlob) { audioFile = audioBlob; } - // Create form data for API request const formData = new FormData(); formData.append('file', audioFile, 'audio.webm'); formData.append('model', settings.model); @@ -119,7 +107,6 @@ async function transcribeAudio(audioBlob) { errorMessage = errorJson.error.message; } } catch (e) { - // If not JSON, use the raw text if (errorText) { errorMessage += ` - ${errorText}`; } @@ -130,7 +117,6 @@ async function transcribeAudio(audioBlob) { const result = await response.json(); - // Handle different response formats let transcription = ''; if (result.text) { transcription = result.text; @@ -151,5 +137,4 @@ async function transcribeAudio(audioBlob) { } } -// Log when background script is loaded console.log('Voice to Text extension background script loaded'); @@ -3,10 +3,8 @@ * Handles text insertion into active input fields */ -// Store the last focused element let lastFocusedElement = null; -// Track focus on all input elements document.addEventListener('focusin', (event) => { const target = event.target; if (isEditableElement(target)) { @@ -14,7 +12,6 @@ document.addEventListener('focusin', (event) => { } }, true); -// Also track click events for contenteditable elements document.addEventListener('click', (event) => { const target = event.target; if (isEditableElement(target)) { @@ -22,27 +19,21 @@ document.addEventListener('click', (event) => { } }, true); -/** - * Check if element is editable - */ function isEditableElement(element) { if (!element) return false; const tagName = element.tagName ? element.tagName.toLowerCase() : ''; - // Check for input fields if (tagName === 'input') { const type = element.type ? element.type.toLowerCase() : 'text'; const editableTypes = ['text', 'search', 'email', 'url', 'tel', 'password']; return editableTypes.includes(type) && !element.disabled && !element.readOnly; } - // Check for textarea if (tagName === 'textarea') { return !element.disabled && !element.readOnly; } - // Check for contenteditable if (element.isContentEditable) { return true; } @@ -50,23 +41,17 @@ function isEditableElement(element) { return false; } -/** - * Get the currently active editable element - */ function getActiveElement() { - // First try document.activeElement let active = document.activeElement; if (active && isEditableElement(active)) { return active; } - // If no active element, try the last focused one if (lastFocusedElement && isEditableElement(lastFocusedElement)) { return lastFocusedElement; } - // If still nothing, try to find any focused input const focused = document.querySelector('input:focus, textarea:focus, [contenteditable="true"]:focus'); if (focused) { return focused; @@ -75,29 +60,22 @@ function getActiveElement() { return null; } -/** - * Insert text into an element - */ function insertText(element, text) { if (!element || !text) return false; try { - // Focus the element element.focus(); const tagName = element.tagName ? element.tagName.toLowerCase() : ''; - // Handle contenteditable elements if (element.isContentEditable) { const selection = window.getSelection(); const range = selection.rangeCount > 0 ? selection.getRangeAt(0) : null; if (range) { - // Check if we need to add a space before the text let textToInsert = text; const beforeCursor = range.startContainer.textContent?.substring(0, range.startOffset) || ''; - // Add space if there's text before cursor and it doesn't end with whitespace if (beforeCursor.length > 0 && !beforeCursor.match(/\s$/)) { textToInsert = ' ' + text; } @@ -106,13 +84,11 @@ function insertText(element, text) { const textNode = document.createTextNode(textToInsert); range.insertNode(textNode); - // Move cursor to end of inserted text range.setStartAfter(textNode); range.setEndAfter(textNode); selection.removeAllRanges(); selection.addRange(range); } else { - // Fallback: append to end with space if needed const currentText = element.textContent || ''; if (currentText.length > 0 && !currentText.match(/\s$/)) { element.textContent += ' ' + text; @@ -121,37 +97,30 @@ function insertText(element, text) { } } - // Trigger input event element.dispatchEvent(new Event('input', { bubbles: true })); element.dispatchEvent(new Event('change', { bubbles: true })); return true; } - // Handle input and textarea elements if (tagName === 'input' || tagName === 'textarea') { const start = element.selectionStart || 0; const end = element.selectionEnd || 0; const currentValue = element.value; - // Check if we need to add a space before the text let textToInsert = text; const beforeCursor = currentValue.substring(0, start); - // Add space if there's text before cursor and it doesn't end with whitespace if (beforeCursor.length > 0 && !beforeCursor.match(/\s$/)) { textToInsert = ' ' + text; } - // Insert text at cursor position const newValue = currentValue.substring(0, start) + textToInsert + currentValue.substring(end); element.value = newValue; - // Set cursor position after inserted text const newPosition = start + textToInsert.length; element.setSelectionRange(newPosition, newPosition); - // Trigger events to ensure frameworks detect the change element.dispatchEvent(new Event('input', { bubbles: true })); element.dispatchEvent(new Event('change', { bubbles: true })); @@ -165,9 +134,6 @@ function insertText(element, text) { } } -/** - * Listen for messages from popup - */ browser.runtime.onMessage.addListener((message, sender, sendResponse) => { if (message.action === 'insertText') { const element = getActiveElement(); Binary files /dev/null and b/src/icons/icon-128-blue.png differ Binary files /dev/null and b/src/icons/icon-128-red.png differ Binary files /dev/null and b/src/icons/icon-128-yellow.png differ Binary files /dev/null and b/src/icons/icon-16-blue.png differ Binary files /dev/null and b/src/icons/icon-16-red.png differ Binary files /dev/null and b/src/icons/icon-16-yellow.png differ Binary files /dev/null and b/src/icons/icon-48-blue.png differ Binary files /dev/null and b/src/icons/icon-48-red.png differ Binary files /dev/null and b/src/icons/icon-48-yellow.png differ @@ -1,7 +1,7 @@ { "manifest_version": 2, "name": "Whisper Voice to Text", - "version": "1.0.0", + "version": "1.0.1", "description": "Record your voice and insert transcribed text into any input field using OpenAI Whisper or compatible APIs", "permissions": [ "activeTab", @@ -58,4 +58,4 @@ "strict_min_version": "58.0" } } -} \ No newline at end of file +} @@ -1,14 +1,10 @@ -* { - margin: 0; - padding: 0; - box-sizing: border-box; -} - body { width: 320px; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, sans-serif; background: #f5f5f5; color: #333; + margin: 0; + padding: 0; } .container { @@ -28,12 +24,15 @@ header { header h1 { font-size: 18px; font-weight: 600; + margin: 0; } main { flex: 1; padding: 20px; background: white; + display: flex; + flex-direction: column; } .status-message { @@ -67,6 +66,94 @@ main { border: 1px solid #ccf; } +/* Continuous Mode Styles */ +.continuous-section { + width: 100%; + margin-bottom: 24px; + padding-bottom: 20px; + border-bottom: 1px solid #eee; + display: flex; + flex-direction: column; + align-items: center; +} + +.continuous-status { + display: flex; + align-items: center; + gap: 12px; + margin-bottom: 16px; + background: #f5f5f5; + padding: 10px 16px; + border-radius: 8px; + width: 100%; +} + +.continuous-status.on { + background: #e8f5e9; + border: 1px solid #c8e6c9; +} + +.continuous-status.speaking { + background: #ffebee; + border: 1px solid #ffcdd2; +} + +.continuous-status.processing { + background: #fff8e1; + border: 1px solid #ffecb3; +} + +.status-icon { + font-size: 24px; +} + +.status-text { + display: flex; + flex-direction: column; + flex: 1; +} + +#status-label { + font-weight: 600; + font-size: 14px; +} + +.status-detail { + font-size: 12px; + color: #666; +} + +.volume-meter { + width: 100%; + margin-bottom: 16px; + display: flex; + align-items: center; + gap: 10px; +} + +.volume-bar { + flex: 1; + height: 8px; + background: #e0e0e0; + border-radius: 4px; + overflow: hidden; +} + +.volume-fill { + height: 100%; + width: 0%; + background: #4CAF50; + transition: width 0.1s ease-out, background-color 0.2s; +} + +.volume-value { + font-size: 12px; + color: #666; + width: 45px; + text-align: right; + font-family: monospace; +} + .recording-section { display: flex; flex-direction: column; @@ -156,6 +243,7 @@ main { background: #f9f9f9; border-radius: 8px; border: 1px solid #e0e0e0; + margin-top: auto; } .shortcut-hint { @@ -217,6 +305,37 @@ footer { fill: currentColor; } +/* Button styles */ +.btn { + padding: 8px 16px; + border-radius: 6px; + font-size: 14px; + font-weight: 500; + cursor: pointer; + transition: all 0.2s; + border: none; + width: 100%; +} + +.btn-secondary { + background: #e0e0e0; + color: #333; +} + +.btn-secondary:hover { + background: #d0d0d0; +} + +.btn-danger { + background: #ffebee; + color: #c62828; + border: 1px solid #ffcdd2; +} + +.btn-danger:hover { + background: #ffcdd2; +} + .transcribed-text { margin: 16px 0; padding: 16px; @@ -15,6 +15,28 @@ <main> <div id="status-message" class="status-message hidden"></div> + <!-- Continuous Mode Status --> + <div id="continuous-section" class="continuous-section"> + <div id="continuous-status" class="continuous-status off"> + <div class="status-icon" id="status-icon">⚪</div> + <div class="status-text"> + <span id="status-label">Continuous Mode: OFF</span> + <span id="status-detail" class="status-detail">Click browser icon to start</span> + </div> + </div> + + <div id="volume-meter" class="volume-meter hidden"> + <div class="volume-bar"> + <div id="volume-fill" class="volume-fill"></div> + </div> + <span id="volume-value" class="volume-value">-∞ dB</span> + </div> + + <button id="toggle-continuous-btn" class="btn btn-secondary"> + Start Listening + </button> + </div> + <div class="recording-section"> <button id="record-btn" class="record-btn" title="Click or press Ctrl+Shift+V to start recording"> <svg class="mic-icon" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"> @@ -3,14 +3,12 @@ * Handles audio recording and UI interactions */ -// State management let mediaRecorder = null; let audioChunks = []; let isRecording = false; let recordingStartTime = 0; let timerInterval = null; -// DOM elements const recordBtn = document.getElementById('record-btn'); const buttonText = document.getElementById('button-text'); const statusMessage = document.getElementById('status-message'); @@ -18,22 +16,15 @@ const recordingIndicator = document.getElementById('recording-indicator'); const timer = document.getElementById('timer'); const settingsBtn = document.getElementById('settings-btn'); -// Initialize document.addEventListener('DOMContentLoaded', async () => { recordBtn.addEventListener('click', toggleRecording); settingsBtn.addEventListener('click', openSettings); - // Check if API is configured await checkConfiguration(); - // Auto-start recording when popup opens - // (This happens when user presses the keyboard shortcut) await startRecording(); }); -/** - * Check if API is configured - */ async function checkConfiguration() { try { const result = await browser.runtime.sendMessage({ action: 'getSettings' }); @@ -45,9 +36,6 @@ async function checkConfiguration() { } } -/** - * Toggle recording state - */ async function toggleRecording() { if (isRecording) { stopRecording(); @@ -56,14 +44,10 @@ async function toggleRecording() { } } -/** - * Start recording audio - */ async function startRecording() { try { showStatus('Requesting microphone access...', 'info'); - // Request microphone permission const stream = await navigator.mediaDevices.getUserMedia({ audio: { echoCancellation: true, @@ -72,7 +56,6 @@ async function startRecording() { } }); - // Create MediaRecorder const mimeType = getSupportedMimeType(); mediaRecorder = new MediaRecorder(stream, { mimeType }); @@ -85,19 +68,14 @@ async function startRecording() { }); mediaRecorder.addEventListener('stop', async () => { - // Stop all tracks stream.getTracks().forEach(track => track.stop()); - - // Process the recording await processRecording(); }); - // Start recording mediaRecorder.start(); isRecording = true; recordingStartTime = Date.now(); - // Update UI updateUIForRecording(); hideStatus(); hideTranscribedText(); @@ -120,16 +98,12 @@ async function startRecording() { } } -/** - * Stop recording audio - */ function stopRecording() { if (mediaRecorder && isRecording) { mediaRecorder.stop(); isRecording = false; stopTimer(); - // Update UI to processing state recordBtn.classList.remove('recording'); recordBtn.classList.add('processing'); buttonText.textContent = 'Processing...'; @@ -139,16 +113,11 @@ function stopRecording() { } } -/** - * Process recorded audio - */ async function processRecording() { try { - // Create blob from chunks const mimeType = getSupportedMimeType(); const audioBlob = new Blob(audioChunks, { type: mimeType }); - // Check if recording is too short const duration = (Date.now() - recordingStartTime) / 1000; if (duration < 0.5) { showStatus('Recording too short. Please try again.', 'error'); @@ -156,7 +125,6 @@ async function processRecording() { return; } - // Send to background script for transcription showStatus('Transcribing audio...', 'info'); const result = await browser.runtime.sendMessage({ @@ -165,7 +133,6 @@ async function processRecording() { }); if (result.success && result.text) { - // Always copy to clipboard as backup try { await navigator.clipboard.writeText(result.text); console.log('Text copied to clipboard as backup'); @@ -173,17 +140,14 @@ async function processRecording() { console.warn('Could not copy to clipboard:', clipboardError); } - // Try to insert text into active field try { const tabs = await browser.tabs.query({ active: true, currentWindow: true }); if (tabs.length === 0) { - // No active tab, just show the text showTranscribedText(result.text); return; } - // Try to send text to content script const response = await browser.tabs.sendMessage(tabs[0].id, { action: 'insertText', text: result.text @@ -191,17 +155,14 @@ async function processRecording() { if (response && response.success) { showStatus('Text inserted successfully! (also copied to clipboard)', 'success'); - // Auto-close popup after successful insertion setTimeout(() => { window.close(); - }, 800); // Brief delay to show success message + }, 800); } else { - // Insertion failed, show the text instead const errorMsg = response?.error || 'No input field focused'; showTranscribedText(result.text, errorMsg); } } catch (error) { - // Content script not responding or other error, show the text console.log('Could not insert text, showing in popup:', error); showTranscribedText(result.text, 'Could not insert text into page'); } @@ -219,9 +180,6 @@ async function processRecording() { } } -/** - * Get supported MIME type for recording - */ function getSupportedMimeType() { const types = [ 'audio/webm', @@ -236,12 +194,9 @@ function getSupportedMimeType() { } } - return 'audio/webm'; // Fallback + return 'audio/webm'; } -/** - * Convert blob to base64 - */ function blobToBase64(blob) { return new Promise((resolve, reject) => { const reader = new FileReader(); @@ -251,9 +206,6 @@ function blobToBase64(blob) { }); } -/** - * Update UI for recording state - */ function updateUIForRecording() { recordBtn.classList.add('recording'); buttonText.textContent = 'Stop Recording'; @@ -261,9 +213,6 @@ function updateUIForRecording() { timer.classList.remove('hidden'); } -/** - * Reset UI to initial state - */ function resetUI() { recordBtn.classList.remove('recording', 'processing'); buttonText.textContent = 'Start Recording'; @@ -272,9 +221,6 @@ function resetUI() { timer.textContent = '0:00'; } -/** - * Start recording timer - */ function startTimer() { timerInterval = setInterval(() => { const elapsed = Math.floor((Date.now() - recordingStartTime) / 1000); @@ -284,9 +230,6 @@ function startTimer() { }, 1000); } -/** - * Stop recording timer - */ function stopTimer() { if (timerInterval) { clearInterval(timerInterval); @@ -294,27 +237,17 @@ function stopTimer() { } } -/** - * Show status message - */ function showStatus(message, type = 'info') { statusMessage.textContent = message; statusMessage.className = `status-message ${type}`; statusMessage.classList.remove('hidden'); } -/** - * Hide status message - */ function hideStatus() { statusMessage.classList.add('hidden'); } -/** - * Show transcribed text in the popup when insertion fails - */ function showTranscribedText(text, errorMsg = null) { - // Create or get text display element let textDisplay = document.getElementById('transcribed-text-display'); if (!textDisplay) { @@ -322,7 +255,6 @@ function showTranscribedText(text, errorMsg = null) { textDisplay.id = 'transcribed-text-display'; textDisplay.className = 'transcribed-text'; - // Add copy button const copyBtn = document.createElement('button'); copyBtn.className = 'copy-btn'; copyBtn.textContent = 'Copy'; @@ -342,16 +274,13 @@ function showTranscribedText(text, errorMsg = null) { textDisplay.appendChild(textContent); textDisplay.appendChild(copyBtn); - // Insert after status message const container = document.querySelector('.container main'); const recordingSection = document.querySelector('.recording-section'); container.insertBefore(textDisplay, recordingSection.nextSibling); } else { - // Update existing display const textContent = textDisplay.querySelector('.text-content'); textContent.textContent = text; - // Update copy button to copy new text const copyBtn = textDisplay.querySelector('.copy-btn'); copyBtn.textContent = 'Copy'; copyBtn.onclick = () => { @@ -366,7 +295,6 @@ function showTranscribedText(text, errorMsg = null) { textDisplay.classList.remove('hidden'); } - // Show appropriate status message if (errorMsg) { showStatus(errorMsg + '. Text shown below (click to copy)', 'info'); } else { @@ -374,9 +302,6 @@ function showTranscribedText(text, errorMsg = null) { } } -/** - * Hide transcribed text display - */ function hideTranscribedText() { const textDisplay = document.getElementById('transcribed-text-display'); if (textDisplay) { @@ -384,36 +309,22 @@ function hideTranscribedText() { } } -/** - * Open settings page - */ function openSettings() { browser.runtime.openOptionsPage(); } -// Handle keyboard shortcuts within the popup document.addEventListener('keydown', (event) => { - // Space or Enter to stop recording if ((event.key === ' ' || event.key === 'Enter') && isRecording) { event.preventDefault(); stopRecording(); return; } - // Escape to stop recording (and popup will close) if (event.key === 'Escape' && isRecording) { event.preventDefault(); stopRecording(); - // Don't close popup immediately, let transcription happen - // User can press Esc again to close if needed return; } }); -// Listen for window focus to handle popup reopening -window.addEventListener('focus', () => { - // If popup regains focus and not recording, don't auto-start again - // This prevents re-recording when popup is clicked -}); - console.log('Voice to Text popup loaded'); @@ -347,3 +347,160 @@ footer { grid-template-columns: 1fr; } } + +/* New styles for Listening Mode and Detection Settings */ + +/* Checkbox label */ +.checkbox-label { + display: flex; + align-items: center; + gap: 8px; + cursor: pointer; + font-weight: 500; +} + +.checkbox-label input[type="checkbox"] { + width: 20px; + height: 20px; + margin: 0; +} + +/* Collapsible sections */ +.form-section.collapsible { + border: 1px solid #e0e0e0; + border-radius: 8px; + padding: 0; + overflow: hidden; +} + +.form-section.collapsible .collapsible-header { + padding: 16px 20px; + background: #f5f5f5; + cursor: pointer; + display: flex; + justify-content: space-between; + align-items: center; + margin: 0; + border-bottom: 1px solid #e0e0e0; + transition: background-color 0.2s; +} + +.form-section.collapsible .collapsible-header:hover { + background: #eeeeee; +} + +.form-section.collapsible .collapsible-header h2 { + margin: 0; + display: flex; + align-items: center; + gap: 8px; +} + +.toggle-icon { + font-size: 12px; + color: #666; + transition: transform 0.2s; +} + +.form-section.collapsible.collapsed .toggle-icon { + transform: rotate(-90deg); +} + +.collapsible-content { + padding: 20px; + transition: max-height 0.3s ease-out; +} + +.form-section.collapsible.collapsed .collapsible-content { + display: none; +} + +/* Range inputs */ +input[type="range"] { + width: 100%; + height: 8px; + -webkit-appearance: none; + appearance: none; + background: #e0e0e0; + border-radius: 4px; + outline: none; + margin: 10px 0; +} + +input[type="range"]::-webkit-slider-thumb { + -webkit-appearance: none; + appearance: none; + width: 20px; + height: 20px; + background: #667eea; + border-radius: 50%; + cursor: pointer; + transition: background-color 0.2s; +} + +input[type="range"]::-webkit-slider-thumb:hover { + background: #5a6fd8; +} + +input[type="range"]::-moz-range-thumb { + width: 20px; + height: 20px; + background: #667eea; + border-radius: 50%; + cursor: pointer; + border: none; + transition: background-color 0.2s; +} + +input[type="range"]::-moz-range-thumb:hover { + background: #5a6fd8; +} + +.range-value { + text-align: center; + font-weight: 600; + color: #667eea; + margin-top: 4px; + font-size: 14px; +} + +/* Enhanced help text */ +.help-text { + display: block; + font-size: 13px; + color: #666; + margin-top: 4px; + line-height: 1.4; +} + +/* Number inputs */ +input[type="number"] { + padding: 10px 12px; + border: 1px solid #ddd; + border-radius: 6px; + font-size: 14px; + width: 100%; + max-width: 200px; + transition: border-color 0.2s; +} + +input[type="number"]:focus { + outline: none; + border-color: #667eea; + box-shadow: 0 0 0 3px rgba(102, 126, 234, 0.1); +} + +/* Responsive adjustments */ +@media (max-width: 600px) { + input[type="number"] { + max-width: 100%; + } + + .form-section.collapsible .collapsible-header { + padding: 12px 16px; + } + + .collapsible-content { + padding: 16px; + } +} @@ -75,6 +75,182 @@ </div> </div> + <div class="form-section"> + <h2>Listening Mode</h2> + + <div class="form-group"> + <label class="checkbox-label"> + <input type="checkbox" id="continuous-mode" name="continuousMode"> + <span>Enable Continuous Background Listening</span> + </label> + <span class="help-text"> + When enabled, the extension will continuously listen for speech and automatically transcribe paragraphs based on pause patterns. + </span> + </div> + + <div class="form-group"> + <label for="idle-timeout"> + Idle Timeout (minutes) + <span class="help-text">Stop listening after this many minutes of silence (1-60)</span> + </label> + <input + type="number" + id="idle-timeout" + name="idleTimeoutMinutes" + min="1" + max="60" + value="5" + required + /> + </div> + </div> + + <div class="form-section collapsible"> + <h2 class="collapsible-header">Speech Detection Settings <span class="toggle-icon">▼</span></h2> + <div class="collapsible-content"> + <div class="form-group"> + <label for="speech-threshold"> + Speech Threshold (dB) + <span class="help-text">Volume level to trigger speech detection (-60 to -20 dB)</span> + </label> + <input + type="range" + id="speech-threshold" + name="speechThresholdDB" + min="-60" + max="-20" + value="-40" + step="1" + /> + <div class="range-value" id="speech-threshold-value">-40 dB</div> + </div> + + <div class="form-group"> + <label for="silence-threshold"> + Silence Threshold (dB) + <span class="help-text">Volume level to trigger silence (-65 to -25 dB, should be lower than speech threshold)</span> + </label> + <input + type="range" + id="silence-threshold" + name="silenceThresholdDB" + min="-65" + max="-25" + value="-45" + step="1" + /> + <div class="range-value" id="silence-threshold-value">-45 dB</div> + </div> + + <div class="form-group"> + <label for="min-speech-duration"> + Minimum Speech Duration (ms) + <span class="help-text">Minimum speech duration required to activate (100-1000 ms)</span> + </label> + <input + type="number" + id="min-speech-duration" + name="minSpeechDuration" + min="100" + max="1000" + value="300" + step="50" + required + /> + </div> + + <div class="form-group"> + <label for="min-silence-duration"> + Minimum Silence Duration (ms) + <span class="help-text">Minimum silence before speech ends (300-2000 ms)</span> + </label> + <input + type="number" + id="min-silence-duration" + name="minSilenceDuration" + min="300" + max="2000" + value="500" + step="50" + required + /> + </div> + </div> + </div> + + <div class="form-section collapsible"> + <h2 class="collapsible-header">Paragraph Detection Settings <span class="toggle-icon">▼</span></h2> + <div class="collapsible-content"> + <div class="form-group"> + <label for="paragraph-threshold-std"> + Pause Threshold (std deviations) + <span class="help-text">Number of standard deviations above mean pause time to trigger paragraph break (1.0-3.0)</span> + </label> + <input + type="range" + id="paragraph-threshold-std" + name="paragraphThresholdStd" + min="1.0" + max="3.0" + value="1.5" + step="0.1" + /> + <div class="range-value" id="paragraph-threshold-std-value">1.5 σ</div> + </div> + + <div class="form-group"> + <label for="paragraph-min-pause"> + Minimum Pause (seconds) + <span class="help-text">Minimum pause duration before considering a paragraph break (0.5-5.0 seconds)</span> + </label> + <input + type="number" + id="paragraph-min-pause" + name="paragraphMinPause" + min="0.5" + max="5.0" + value="0.8" + step="0.1" + required + /> + </div> + + <div class="form-group"> + <label for="paragraph-max-chars"> + Maximum Characters per Paragraph + <span class="help-text">Force paragraph break after this many characters (100-1000)</span> + </label> + <input + type="number" + id="paragraph-max-chars" + name="paragraphMaxChars" + min="100" + max="1000" + value="500" + step="50" + required + /> + </div> + + <div class="form-group"> + <label for="paragraph-max-words"> + Maximum Words per Paragraph + <span class="help-text">Force paragraph break after this many words (50-200)</span> + </label> + <input + type="number" + id="paragraph-max-words" + name="paragraphMaxWords" + min="50" + max="200" + value="100" + step="10" + required + /> + </div> + </div> + </div> + <div class="form-section presets"> <h2>Quick Presets</h2> <p class="section-description">Load configuration for popular providers</p> @@ -3,7 +3,6 @@ * Handles configuration management */ -// Preset configurations const PRESETS = { openai: { apiEndpoint: 'https://api.openai.com/v1/audio/transcriptions', @@ -19,7 +18,6 @@ const PRESETS = { } }; -// DOM elements const form = document.getElementById('settings-form'); const apiEndpointInput = document.getElementById('api-endpoint'); const apiTokenInput = document.getElementById('api-token'); @@ -33,16 +31,12 @@ const presetButtons = document.querySelectorAll('.preset-btn'); const requestMicBtn = document.getElementById('request-mic-btn'); const micStatus = document.getElementById('mic-status'); -// Initialize document.addEventListener('DOMContentLoaded', () => { loadSettings(); attachEventListeners(); checkMicrophonePermission(); }); -/** - * Attach event listeners - */ function attachEventListeners() { form.addEventListener('submit', handleSubmit); toggleTokenBtn.addEventListener('click', toggleTokenVisibility); @@ -58,33 +52,25 @@ function attachEventListeners() { }); } -/** - * Check if microphone permission is already granted - */ async function checkMicrophonePermission() { try { - // Try to query permission state using Permissions API if (navigator.permissions && navigator.permissions.query) { try { const result = await navigator.permissions.query({ name: 'microphone' }); updateMicrophoneStatus(result.state); - // Listen for permission changes result.addEventListener('change', () => { updateMicrophoneStatus(result.state); }); - // If not granted, show a reminder if (result.state !== 'granted') { showStatus('Please grant microphone access to use the extension', 'info'); } } catch (queryError) { - // Permissions API query failed, try direct access check console.log('Permissions query failed, trying direct check:', queryError); await checkMicrophoneAccessDirectly(); } } else { - // Permissions API not supported, try direct check await checkMicrophoneAccessDirectly(); } } catch (error) { @@ -92,31 +78,23 @@ async function checkMicrophonePermission() { } } -/** - * Check microphone access directly by attempting to access it - */ async function checkMicrophoneAccessDirectly() { try { - // Try to get media devices list to check if we have access const devices = await navigator.mediaDevices.enumerateDevices(); const hasAudioInput = devices.some(device => device.kind === 'audioinput'); if (hasAudioInput) { - // We can see audio devices, check if we have permission const audioDevicesWithLabel = devices.filter( device => device.kind === 'audioinput' && device.label !== '' ); if (audioDevicesWithLabel.length > 0) { - // If we can see device labels, we have permission updateMicrophoneStatus('granted'); } else { - // No labels means no permission yet updateMicrophoneStatus('prompt'); showStatus('Please grant microphone access to use the extension', 'info'); } } else { - // No audio input devices found updateMicrophoneStatus('prompt'); } } catch (error) { @@ -125,18 +103,13 @@ async function checkMicrophoneAccessDirectly() { } } -/** - * Request microphone permission - */ async function requestMicrophonePermission() { requestMicBtn.disabled = true; requestMicBtn.textContent = 'Requesting...'; try { - // Request microphone access const stream = await navigator.mediaDevices.getUserMedia({ audio: true }); - // Permission granted - stop the stream immediately stream.getTracks().forEach(track => track.stop()); updateMicrophoneStatus('granted'); @@ -161,9 +134,6 @@ async function requestMicrophonePermission() { } } -/** - * Update microphone permission status UI - */ function updateMicrophoneStatus(state) { micStatus.classList.remove('hidden', 'granted', 'denied'); @@ -184,9 +154,6 @@ function updateMicrophoneStatus(state) { } } -/** - * Load settings from storage - */ async function loadSettings() { try { const result = await browser.storage.sync.get(['apiEndpoint', 'apiToken', 'model']); @@ -207,7 +174,6 @@ async function loadSettings() { modelInput.value = PRESETS.openai.model; } - // Highlight active preset updateActivePreset(); } catch (error) { @@ -216,9 +182,6 @@ async function loadSettings() { } } -/** - * Handle form submission - */ async function handleSubmit(event) { event.preventDefault(); @@ -228,7 +191,6 @@ async function handleSubmit(event) { model: modelInput.value.trim() }; - // Validate settings if (!settings.apiEndpoint) { showStatus('Please enter an API endpoint', 'error'); return; @@ -244,7 +206,6 @@ async function handleSubmit(event) { return; } - // Validate URL try { new URL(settings.apiEndpoint); } catch (e) { @@ -261,7 +222,6 @@ async function handleSubmit(event) { showStatus('Settings saved successfully!', 'success'); updateActivePreset(); - // Reset button text after a delay setTimeout(() => { saveBtn.disabled = false; saveBtn.textContent = 'Save Settings'; @@ -275,9 +235,6 @@ async function handleSubmit(event) { } } -/** - * Toggle API token visibility - */ function toggleTokenVisibility() { if (apiTokenInput.type === 'password') { apiTokenInput.type = 'text'; @@ -288,9 +245,6 @@ function toggleTokenVisibility() { } } -/** - * Test API connection - */ async function testConnection() { const settings = { apiEndpoint: apiEndpointInput.value.trim(), @@ -309,14 +263,12 @@ async function testConnection() { showStatus('Testing connection to API...', 'info'); - // Create a minimal audio file for testing (1 second of silence) const sampleRate = 16000; const duration = 1; const numSamples = sampleRate * duration; const buffer = new ArrayBuffer(44 + numSamples * 2); const view = new DataView(buffer); - // WAV header const writeString = (offset, string) => { for (let i = 0; i < string.length; i++) { view.setUint8(offset + i, string.charCodeAt(i)); @@ -339,7 +291,6 @@ async function testConnection() { const blob = new Blob([buffer], { type: 'audio/wav' }); - // Test API request const formData = new FormData(); formData.append('file', blob, 'test.wav'); formData.append('model', settings.model); @@ -381,9 +332,6 @@ async function testConnection() { } } -/** - * Reset settings to defaults - */ async function resetToDefaults() { if (!confirm('Are you sure you want to reset all settings to defaults?')) { return; @@ -399,9 +347,6 @@ async function resetToDefaults() { } } -/** - * Apply preset configuration - */ function applyPreset(provider) { const preset = PRESETS[provider]; @@ -417,16 +362,11 @@ function applyPreset(provider) { modelInput.value = preset.model; } - // Don't clear the API token when switching presets - // Just highlight the selected preset updateActivePreset(); showStatus(`${provider.toUpperCase()} preset applied. Don't forget to enter your API token!`, 'info'); } -/** - * Update active preset button - */ function updateActivePreset() { const currentEndpoint = apiEndpointInput.value; @@ -442,15 +382,11 @@ function updateActivePreset() { }); } -/** - * Show status message - */ function showStatus(message, type = 'info') { statusMessage.textContent = message; statusMessage.className = `status-message ${type}`; statusMessage.classList.remove('hidden'); - // Auto-hide after 5 seconds for success messages if (type === 'success') { setTimeout(() => { statusMessage.classList.add('hidden'); @@ -0,0 +1,331 @@ +/** + * Paragraph Detector for adaptive paragraph breaks based on speech pause patterns + * Ported from Python to JavaScript + * + * Adapts to speaker patterns by analyzing pause durations and using statistical + * analysis to detect "significant" pauses that indicate paragraph boundaries. + */ + +class ParagraphDetector { + constructor(options = {}) { + // Configuration parameters + this.thresholdStd = options.thresholdStd || 1.5; // Number of standard deviations above mean for "significant" pause + this.minPause = options.minPause || 0.8; // Minimum pause duration to consider as potential break (seconds) + this.maxChars = options.maxChars || 500; // Maximum characters per paragraph before forced break + this.maxWords = options.maxWords || 100; // Maximum words per paragraph before forced break + this.windowSize = options.windowSize || 30; // Number of recent pauses to use for statistics + this.warmupCount = options.warmupCount || 5; // Minimum pauses needed before adaptive mode + this.warmupThreshold = options.warmupThreshold || 2.0; // Fixed threshold during warmup period (seconds) + + // State tracking + this.pauseHistory = []; // Array of pause durations in seconds + this.currentParaChars = 0; // Character count for current paragraph + this.currentParaWords = 0; // Word count for current paragraph + this.lastAbsoluteEnd = null; // Track absolute end time across batches (seconds) + } + + /** + * Process a Whisper segment and return text with paragraph breaks + * + * @param {Array} segments - Array of Whisper segment objects with .text, .start, .end + * @param {number} timeOffset - Cumulative audio offset to convert relative to absolute timestamps (seconds) + * @returns {string} - Text with paragraph breaks (\n\n) inserted where appropriate + */ + processSegments(segments, timeOffset = 0.0) { + if (!segments || segments.length === 0) { + return ''; + } + + const resultParts = []; + + for (const segment of segments) { + // Validate segment + if (!segment.text || typeof segment.text !== 'string') { + continue; + } + + const text = segment.text; + let shouldBreak = false; + + // Convert to absolute timestamps + const absoluteStart = (segment.start || 0) + timeOffset; + const absoluteEnd = (segment.end || 0) + timeOffset; + + // Check hard limits first (max chars/words) + const newChars = this.currentParaChars + text.length; + const newWords = this.currentParaWords + this.countWords(text); + + if (this.currentParaChars > 0) { + if (newChars > this.maxChars || newWords > this.maxWords) { + shouldBreak = true; + } + } + + // Check pause using absolute timestamps (works across batches!) + if (!shouldBreak && this.lastAbsoluteEnd !== null) { + const pauseDuration = absoluteStart - this.lastAbsoluteEnd; + + if (pauseDuration > 0) { + // Record this pause for statistics + this._addPause(pauseDuration); + + // Only consider pauses above minimum threshold + if (pauseDuration >= this.minPause) { + const threshold = this._getAdaptiveThreshold(); + + if (pauseDuration > threshold) { + shouldBreak = true; + } + } + } + } + + // Apply break if needed + if (shouldBreak) { + resultParts.push('\n\n'); + this._resetParagraph(); + } + + // Add text and update counters + resultParts.push(text); + this.currentParaChars += text.length; + this.currentParaWords += this.countWords(text); + + // Track absolute end time for next comparison + this.lastAbsoluteEnd = absoluteEnd; + } + + return resultParts.join(''); + } + + /** + * Get statistics for debugging/display + */ + getStats() { + if (this.pauseHistory.length < 2) { + return { + pauseCount: this.pauseHistory.length, + mean: null, + std: null, + threshold: this.warmupThreshold, + mode: 'warmup', + currentParaChars: this.currentParaChars, + currentParaWords: this.currentParaWords + }; + } + + const n = this.pauseHistory.length; + const mean = this._calculateMean(this.pauseHistory); + const std = this._calculateStdDev(this.pauseHistory, mean); + + return { + pauseCount: n, + mean: this._round(mean, 3), + std: this._round(std, 3), + threshold: this._round(this._getAdaptiveThreshold(), 3), + mode: n >= this.warmupCount ? 'adaptive' : 'warmup', + currentParaChars: this.currentParaChars, + currentParaWords: this.currentParaWords + }; + } + + /** + * Reset paragraph counters + */ + resetParagraph() { + this._resetParagraph(); + } + + /** + * Reset entire state (including pause history) + */ + resetAll() { + this.pauseHistory = []; + this._resetParagraph(); + this.lastAbsoluteEnd = null; + } + + /** + * Manually set the last end time (useful for initializing) + */ + setLastEndTime(endTime) { + this.lastAbsoluteEnd = endTime; + } + + /** + * Record a pause duration for statistics + * @private + */ + _addPause(duration) { + if (duration > 0) { + this.pauseHistory.push(duration); + + // Maintain window size + if (this.pauseHistory.length > this.windowSize) { + this.pauseHistory.shift(); + } + } + } + + /** + * Calculate the current adaptive threshold based on pause history + * @private + */ + _getAdaptiveThreshold() { + // During warmup, use fixed threshold + if (this.pauseHistory.length < this.warmupCount) { + return this.warmupThreshold; + } + + const mean = this._calculateMean(this.pauseHistory); + const std = this._calculateStdDev(this.pauseHistory, mean); + + // Threshold is mean + (threshold_std * std), but at least min_pause + const threshold = mean + (this.thresholdStd * std); + return Math.max(threshold, this.minPause); + } + + /** + * Reset paragraph counters + * @private + */ + _resetParagraph() { + this.currentParaChars = 0; + this.currentParaWords = 0; + } + + /** + * Calculate mean of array + * @private + */ + _calculateMean(arr) { + const n = arr.length; + if (n === 0) return 0; + + const sum = arr.reduce((a, b) => a + b, 0); + return sum / n; + } + + /** + * Calculate standard deviation + * @private + */ + _calculateStdDev(arr, mean) { + const n = arr.length; + if (n < 2) return 0; + + const variance = arr.reduce((sum, val) => { + const diff = val - mean; + return sum + diff * diff; + }, 0) / n; + + return Math.sqrt(variance || 0); + } + + /** + * Round number to specified decimal places + * @private + */ + _round(num, decimals) { + const factor = Math.pow(10, decimals); + return Math.round(num * factor) / factor; + } + + /** + * Count words in text (simple approximation) + */ + countWords(text) { + if (!text || text.length === 0) { + return 0; + } + + // Trim whitespace and split on whitespace + const trimmed = text.trim(); + if (trimmed.length === 0) { + return 0; + } + + return trimmed.split(/\s+/).length; + } + + /** + * Set configuration parameters dynamically + */ + setParams(params) { + if (params.thresholdStd !== undefined) { + this.thresholdStd = params.thresholdStd; + } + if (params.minPause !== undefined) { + this.minPause = params.minPause; + } + if (params.maxChars !== undefined) { + this.maxChars = params.maxChars; + } + if (params.maxWords !== undefined) { + this.maxWords = params.maxWords; + } + if (params.windowSize !== undefined) { + this.windowSize = params.windowSize; + // Trim history if new window is smaller + if (this.pauseHistory.length > this.windowSize) { + this.pauseHistory = this.pauseHistory.slice(-this.windowSize); + } + } + if (params.warmupCount !== undefined) { + this.warmupCount = params.warmupCount; + } + if (params.warmupThreshold !== undefined) { + this.warmupThreshold = params.warmupThreshold; + } + } + + /** + * Get configuration + */ + getConfig() { + return { + thresholdStd: this.thresholdStd, + minPause: this.minPause, + maxChars: this.maxChars, + maxWords: this.maxWords, + windowSize: this.windowSize, + warmupCount: this.warmupCount, + warmupThreshold: this.warmupThreshold + }; + } + + /** + * Create a copy of this detector with same configuration + */ + clone() { + return new ParagraphDetector(this.getConfig()); + } +} + +// Export for Node.js/module environments +if (typeof module !== 'undefined' && module.exports) { + module.exports = ParagraphDetector; +} + +// Export for browser/script environments +if (typeof window !== 'undefined') { + window.ParagraphDetector = ParagraphDetector; +} + +// Example usage: +if (typeof window !== 'undefined' && window.location && window.location.hostname === 'localhost') { + // Self-test when run in browser + console.log('ParagraphDetector loaded:', typeof ParagraphDetector); + + // Quick test + const detector = new ParagraphDetector(); + const testSegments = [ + { text: 'Hello world', start: 0.0, end: 1.2 }, + { text: 'This is a test', start: 2.5, end: 3.8 }, + { text: 'With longer pause', start: 8.0, end: 9.5 } + ]; + + const result = detector.processSegments(testSegments); + console.log('Test result:', result); + console.log('Stats:', detector.getStats()); +} \ No newline at end of file @@ -0,0 +1,416 @@ +/** + * Voice Activity Detector (VAD) for real-time speech detection + * Uses Web Audio API to analyze audio and detect speech/silence + */ + +class VoiceActivityDetector extends EventTarget { + constructor(options = {}) { + super(); + + // Configuration + this.thresholdDB = options.thresholdDB || -40; // Enter speech state at this dB + this.silenceThresholdDB = options.silenceThresholdDB || -45; // Exit speech state (hysteresis) + this.minSpeechDuration = options.minSpeechDuration || 300; // Minimum ms to consider speech + this.minSilenceDuration = options.minSilenceDuration || 500; // Minimum ms to consider silence + this.bufferSize = options.bufferSize || 2048; // FFT buffer size + this.sampleRate = options.sampleRate || 16000; + + // Frequency band for voice (85-255 Hz) + this.voiceMinFreq = options.voiceMinFreq || 85; + this.voiceMaxFreq = options.voiceMaxFreq || 255; + + // State + this.isListening = false; + this.currentState = 'silence'; // 'silence' or 'speech' + this.stateStartTime = null; + this.audioContext = null; + this.analyser = null; + this.processor = null; + this.mediaStream = null; + this.mediaStreamSource = null; + + // Timers + this.analysisIntervalId = null; + this.stateTimeoutId = null; + + // Audio data + this.frequencyBinCount = 0; + this.frequencyData = null; + this.timeData = null; + this.currentVolume = -Infinity; + + // Debug stats + this.stats = { + speechDuration: 0, + silenceDuration: 0, + transitions: 0, + falsePositives: 0, + detectionCount: 0 + }; + } + + /** + * Start listening for voice activity + */ + async start() { + if (this.isListening) { + console.warn('VoiceActivityDetector already listening'); + return; + } + + try { + // Get microphone access + this.mediaStream = await navigator.mediaDevices.getUserMedia({ + audio: { + echoCancellation: true, + noiseSuppression: true, + autoGainControl: true, + channelCount: 1, + sampleRate: this.sampleRate + } + }); + + // Create audio context in background script requires different approach + // but in popup/content script it's straightforward + this.audioContext = new AudioContext({ sampleRate: this.sampleRate }); + + // Create analyser + this.analyser = this.audioContext.createAnalyser(); + this.analyser.fftSize = this.bufferSize; + this.analyser.smoothingTimeConstant = 0.8; + + // Create processor for real-time analysis + this.processor = this.audioContext.createScriptProcessor(this.bufferSize, 1, 1); + + // Connect media stream + this.mediaStreamSource = this.audioContext.createMediaStreamSource(this.mediaStream); + this.mediaStreamSource.connect(this.analyser); + this.mediaStreamSource.connect(this.processor); + this.processor.connect(this.audioContext.destination); + + // Prepare data arrays + this.frequencyBinCount = this.analyser.frequencyBinCount; + this.frequencyData = new Uint8Array(this.frequencyBinCount); + this.timeData = new Uint8Array(this.frequencyBinCount); + + // Start analysis loop + this.startAnalysis(); + + this.isListening = true; + this.stateStartTime = Date.now(); + + console.log('VoiceActivityDetector started'); + this.dispatchEvent(new CustomEvent('started', { + detail: { timestamp: Date.now() } + })); + + } catch (error) { + console.error('Failed to start VoiceActivityDetector:', error); + this.cleanup(); + throw error; + } + } + + /** + * Stop listening and cleanup + */ + stop() { + if (!this.isListening) { + return; + } + + this.isListening = false; + this.cleanup(); + + console.log('VoiceActivityDetector stopped'); + this.dispatchEvent(new CustomEvent('stopped', { + detail: { timestamp: Date.now() } + })); + } + + /** + * Cleanup resources + */ + cleanup() { + // Stop analysis interval + if (this.analysisIntervalId) { + clearInterval(this.analysisIntervalId); + this.analysisIntervalId = null; + } + + // Clear state timeout + if (this.stateTimeoutId) { + clearTimeout(this.stateTimeoutId); + this.stateTimeoutId = null; + } + + // Stop processor + if (this.processor) { + this.processor.onaudioprocess = null; + } + + // Disconnect audio nodes + if (this.mediaStreamSource) { + try { + this.mediaStreamSource.disconnect(); + } catch (e) { + // Already disconnected + } + this.mediaStreamSource = null; + } + + if (this.analyser) { + try { + this.analyser.disconnect(); + } catch (e) { + // Already disconnected + } + this.analyser = null; + } + + if (this.processor) { + try { + this.processor.disconnect(); + } catch (e) { + // Already disconnected + } + this.processor = null; + } + + // Close audio context + if (this.audioContext) { + this.audioContext.close().catch(console.error); + this.audioContext = null; + } + + // Stop media tracks + if (this.mediaStream) { + this.mediaStream.getTracks().forEach(track => track.stop()); + this.mediaStream = null; + } + + // Reset state + this.currentState = 'silence'; + this.currentVolume = -Infinity; + } + + /** + * Start the analysis loop + */ + startAnalysis() { + // Use requestAnimationFrame or setInterval for analysis + // In background script context, use setInterval + const ANALYSIS_INTERVAL_MS = 50; // Analyze every 50ms + + this.analysisIntervalId = setInterval(() => { + this.analyzeAudio(); + }, ANALYSIS_INTERVAL_MS); + } + + /** + * Analyze current audio frame + */ + analyzeAudio() { + if (!this.isListening || !this.analyser) { + return; + } + + // Get frequency data + this.analyser.getByteFrequencyData(this.frequencyData); + this.analyser.getByteTimeDomainData(this.timeData); + + // Calculate volume (dB) focusing on voice frequency band + const volumeDB = this.calculateVoiceBandVolume(); + this.currentVolume = volumeDB; + + // Update state + this.updateState(volumeDB); + + // Emit volume event for debugging/monitoring + this.dispatchEvent(new CustomEvent('volume', { + detail: { + volumeDB, + timestamp: Date.now() + } + })); + } + + /** + * Calculate volume in the voice frequency band + * Voice frequencies: 85-255 Hz + */ + calculateVoiceBandVolume() { + if (!this.frequencyData || this.frequencyData.length === 0) { + return -Infinity; + } + + // Calculate frequency bin indices for voice band + const nyquist = this.sampleRate / 2; + const binWidth = nyquist / this.frequencyBinCount; + + // Find bin indices for voice frequency range + const voiceMinBin = Math.max(0, Math.floor(this.voiceMinFreq / binWidth)); + const voiceMaxBin = Math.min( + this.frequencyBinCount - 1, + Math.floor(this.voiceMaxFreq / binWidth) + ); + + // Calculate average energy in voice band + let sum = 0; + let count = 0; + + for (let i = voiceMinBin; i <= voiceMaxBin; i++) { + sum += this.frequencyData[i]; + count++; + } + + if (count === 0) { + return -Infinity; + } + + const avg = sum / count; + + // Clamp and convert to dB + const normalized = Math.max(0, Math.min(255, avg)); + + // Convert to dB: 0-255 maps to -Infinity to 0 dB + if (normalized < 1) { + return -Infinity; + } + + // Simple dB approximation + const db = 20 * Math.log10(normalized / 255); + + return Math.max(-100, db); // Clamp minimum to -100 dB + } + + /** + * Update state based on volume + */ + updateState(volumeDB) { + const now = Date.now(); + const stateDuration = this.stateStartTime ? now - this.stateStartTime : 0; + + // Initialize state if first run + if (!this.stateStartTime) { + this.stateStartTime = now; + this.currentState = 'silence'; + } + + let newState = this.currentState; + + // State transition logic with hysteresis + if (this.currentState === 'silence') { + // In silence state - require threshold to enter speech + if (volumeDB >= this.thresholdDB) { + // Check if state duration meets minimum + if (stateDuration >= this.minSpeechDuration) { + newState = 'speech'; + } + } + } else if (this.currentState === 'speech') { + // In speech state - require lower threshold to exit (hysteresis) + if (volumeDB <= this.silenceThresholdDB) { + // Check if silence duration meets minimum + if (stateDuration >= this.minSilenceDuration) { + newState = 'silence'; + } + } + } + + // Update state if changed + if (newState !== this.currentState) { + this.currentState = newState; + this.stateStartTime = now; + this.stats.transitions++; + + // Emit state change event + this.dispatchEvent(new CustomEvent('stateChanged', { + detail: { + state: this.currentState, + previousState: newState === 'speech' ? 'silence' : 'speech', + timestamp: now + } + })); + + // Handle state-specific events + if (this.currentState === 'speech') { + console.log('Speech detected'); + this.dispatchEvent(new CustomEvent('speechStart', { + detail: { + timestamp: now, + volumeDB: this.currentVolume + } + })); + } else if (this.currentState === 'silence') { + console.log('Silence detected'); + this.dispatchEvent(new CustomEvent('speechEnd', { + detail: { + timestamp: now, + duration: stateDuration, + volumeDB: this.currentVolume + } + })); + + // Check for significant silence (e.g., > 2 seconds) + if (stateDuration > 2000) { + this.dispatchEvent(new CustomEvent('silenceDetected', { + detail: { + timestamp: now, + duration: stateDuration + } + })); + } + } + } + + // Update stats + if (this.currentState === 'speech') { + this.stats.speechDuration = stateDuration; + } else { + this.stats.silenceDuration = stateDuration; + } + } + + /** + * Get current state + */ + getState() { + return { + currentState: this.currentState, + volumeDB: this.currentVolume, + stateDuration: this.stateStartTime ? Date.now() - this.stateStartTime : 0, + isListening: this.isListening + }; + } + + /** + * Get statistics + */ + getStats() { + return { ...this.stats }; + } + + /** + * Reset statistics + */ + resetStats() { + this.stats = { + speechDuration: 0, + silenceDuration: 0, + transitions: 0, + falsePositives: 0, + detectionCount: 0 + }; + } +} + +// Export for use in other modules +if (typeof module !== 'undefined' && module.exports) { + module.exports = VoiceActivityDetector; +} + +// Export for browser/script context +if (typeof window !== 'undefined') { + window.VoiceActivityDetector = VoiceActivityDetector; +} \ No newline at end of file