Hardware Controller is a native macOS app that turns a VEC Infinity 3 USB foot controller into low-latency Actions. Each Control can run Local Dictation, Local AI Dictation, an exact keyboard shortcut, or No Action using Hold or Toggle behavior.

All configuration and speech content stay on the Mac. Local AI Dictation uses Apple's on-device model or a fixed-loopback Ollama service; it never uses cloud inference.

Requirements: Apple silicon, macOS 15 or later, Xcode 26 or a compatible Swift 6 toolchain, and rustup. The repository pins Rust 1.98.

git clone https://github.com/MarcusJRLee/hardware_controller.git
cd hardware_controller
scripts/run_demo.sh

Demo mode is deterministic and requires no foot controller, Apple signing identity, or privacy permission.

Run the same formatting, build, test, and release-script checks as GitHub:

scripts/check.sh

scripts/check_ios.sh additionally rejects network clients and network/cloud capabilities in iOS product sources before building or testing.

Configure the ignored .env.local signing values, connect and unlock an iPhone, then run:

scripts/install_ios.sh

The command asks which available iPhone and configuration to use, then builds, installs, and launches Voice Input. Development is the default Debug build. Local QA is an optimized Release build that remains Apple Development signed; it is not an App Store or production release. Automation may use --device <identifier-or-name> --config <development|local_qa>.

See the contributor guide for source ownership, test placement, Driver additions, and opt-in system checks.

The iOS app performs capture and inference locally and has no model-download path. To exercise its first speech-to-text adapter, prepare the pinned Whisper Tiny English package on the Mac, move that folder into Files on the iPhone, then use Voice Input → Local models → Import Model package → Use for speech to text. Runtime/model downloads occur only in repository build preparation; the installed app does not require a network connection.

The custom keyboard records through the containing app or its Control Center control; iOS does not permit microphone capture inside a keyboard extension. After local finalization, the keyboard makes one automatic insertion attempt. If no field update is confirmed, Recover… permits one explicit same-target retry or an on-device clipboard copy that expires after ten minutes and cannot cross Universal Clipboard. Any field, session, or process change falls back to Voice Input → History, where completed text remains copyable.

Voice Input → History → History storage configures recording age, total bytes, and count. The defaults are 90 days, 1 GiB, and 2,000 recordings. Pin important audio to exclude it from automatic cleanup; transcripts remain after audio expires. Low-disk maintenance restores a 1 GiB free-space reserve without discarding a committed capture. Installed Model packages use a separate budget and are removed only by an explicit user action.

For keyboard-free iPhone capture, add Voice Capture to Control Center, the Lock Screen, or the Action button, or use the bundled Siri/Shortcuts start and stop actions. The containing app records locally and shows a Live Activity with a stop action. Completed text is saved before it becomes available; copy or share it from Voice Input → History, or retrieve it from the keyboard later.

For a signed local build, create ignored private settings once:

cp .env.example .env.local
# Replace both placeholders in .env.local with your own signing values.
set -a
source .env.local
set +a

Build an app intended for /Applications only with that Apple Development identity:

security find-identity -v -p codesigning
scripts/build_app.sh
codesign --verify --deep --strict --verbose=2 \
  "dist/Hardware Controller.app"
codesign -dv --verbose=4 "dist/Hardware Controller.app" 2>&1 \
  | grep -E '^Authority='
codesign -dv --verbose=4 "dist/Hardware Controller.app" 2>&1 \
  | grep -E "^TeamIdentifier=${HC_EXPECTED_TEAM_ID}$"

Never commit .env.local. Quit the running app before replacing the single canonical /Applications/Hardware Controller.app, then launch that exact bundle. Preserve its marketing version and build number during routine local iterations. Never install an ad-hoc-signed build.

The application identifier is com.longdevity.hardwarecontroller. Profiles and preferences use only its matching Application Support directory. The pre-public identity transition is complete, so the public snapshot contains no predecessor personal namespace. macOS may require Accessibility, Microphone, Speech Recognition, and Launch at Login authorization again when the signed application identity changes.

Signal Bridge is the provisional public mark: three generic control nodes on a near-black surface, joined by one low-latency signal path with an active amber center. The menu-bar template uses the same three-node geometry. Neither mark copies a supported Device or manufacturer branding. The source raster is packaging/app_icon_source.png.

Across macOS and iOS, the interface is restrained and purpose-led: neutral surfaces, no decorative borders, system typography, and one strong action per workflow. Color is reserved for state or recovery that cannot be communicated as clearly through hierarchy and symbols.

A source change or version number is not release approval. Do not run scripts/build_release.sh, create a DMG, tag, GitHub Release, or release record without explicit approval for that exact version. The intentionally retained accepted-artifact evidence is in release validation.

1. Open the canonical Applications copy.
2. Allow Accessibility and Microphone when requested. macOS 15–25 also requests Speech Recognition.
3. Connect the VEC Infinity 3 and confirm each physical Control visibly changes state without producing a pointer click.
4. Configure each Control under Controller or its Profile's Device setup.
5. Focus an editable field, hold or toggle the Control, and speak.

The center Control defaults to Local Dictation in Hold mode. Left and right default to No Action. Any configured Control may receive an opt-in exact keyboard fallback for use while its Device is disconnected.

To dictate without a Device, open General → Voice capture shortcut, record an exact chord with at least two modifiers, then hold it while speaking. Two short presses latch capture; the next two finish. The chord uses Local AI Dictation and is disabled until you configure it.

See the user guide for Profiles, target behavior, recovery, and troubleshooting.

Voice History repairs app-owned partial, orphan, and interrupted-expiration audio at startup before applying storage limits. Recovered audio is marked, playable, and locally retranscribable without invented text; unpinned recovery audio expires after 24 hours while its History row remains searchable. History can also import a supported local recording, transcribe and format it on-device, and retain one app-owned copy without changing the original. V1 .voice_history archives move immutable transcript/audio evidence between installations through the Rust verifier linked into the Apple app, then bounded Swift restore logic. Import never delivers text.

Both Actions reuse the same microphone, recognition, target, permission, and delivery boundaries, but retain separate controllers and settings. One process-wide coordinator prevents simultaneous microphone ownership. Local AI model warm-up begins while the user speaks and never blocks the HID-to-Action path.

Local AI Dictation removes fillers, resolves clear self-corrections, corrects supported recognition errors, and applies the selected Natural, Casual Message, Formal, Technical, or Verbatim Style. It creates validated paragraph and list blocks, then preserves or flattens structure for the target. It also applies exact spoken commands such as scratch that, delete that sentence, new paragraph, start a bullet list, bullet, next item, and numbered-list boundaries before formatting; say literal immediately before a command phrase to keep the phrase. It normalizes conservative grocery, shopping, packing, task, explicit-marker, and sequential-ordinal list cues before formatting. Typed casing policy can retain the selected Style, lowercase prose while preserving source-signaled names, or enforce strict lowercase while protecting operational tokens. The formatter returns typed paragraph/list blocks; deterministic normalization restores protected token spelling and list boundaries when Edited text has confident cues. The same casing and spoken-list rules run on macOS and iOS. It validates protected numbers, URLs, email addresses, paths, code-like tokens, quotations, and dictionary terms. A provider error, invalid output, or three-second deadline delivers the deterministic Edited transcript once when the captured target is still safe.

General → Local AI Dictation names speech-to-text and formatting separately. Its active-pipeline evidence shows each provider and model, typed output, validation boundary, and deterministic fallback. History records whether each formatted result was validated or used the Edited fallback.

Apple On-Device refinement requires macOS 26, Apple Intelligence enabled, a supported locale, and installed model assets. Select it under General → Local AI Dictation, refresh status, then test the provider. The app uses SystemLanguageModel locally and does not enable Private Cloud Compute.

Install and start Ollama separately, then install the recommended model:

ollama pull qwen3.5:4b

Choose Ollama, select the installed model, refresh status, and run Test Selected Provider. The app connects only to http://127.0.0.1:11434, rejects redirects and proxy routing, excludes cloud model tags, and pins the selected local digest. The validated recommendation is qwen3.5:4b with digest:

2a654d98e6fba55d452b7043684e9b57a947e393bbffa62485a7aac05ee4eefd

The measured model file is 3.39 GB and its reference resident allocation is 5.74 GB, above the original 4 GB target. Model choice and measurements are in Local AI model evaluation.

Five-minute retention is the default. Until app quits is ownership-aware: the app unloads a model it started when settings change or the process exits, but preserves a model that another local Ollama client already had running.

• Local Dictation remains memory-only. Local AI Dictation records one local CAF plus immutable Raw, Edited, Formatted, Delivered, and corrected results under the app's Application Support directory. History can search, inspect, replay, correct, retranscribe, reformat, retry, export, pin, and delete those sessions. General → Voice History storage independently caps audio age, total bytes, and recording count; defaults are 90 days, 2 GiB, or 5,000 recordings, whichever is reached first. Unlimited and no-retained-audio choices are explicit. Cleanup keeps transcripts searchable and protects active, pinned, and sole recovery audio.
• Nearby context, prompts, and model payloads remain memory-only. Speech content is never written to logs.
• Apple speech recognition is required to run on-device.
• Ollama traffic is restricted to fixed numeric loopback; no user content is sent to a remote host.
• Nearby text is off by default. When enabled, the app reads at most a bounded caret window from an approved multiline, nonsecure target for the current session.
• The app never reads browser URLs, terminal contents, screenshots, whole documents, the pasteboard, or the global keyboard stream.
• Focus and caret ownership are revalidated before delivery. Secure fields are rejected.

• Exact-signature, exclusive IOKit HID input on a user-interactive serial queue.
• Independent per-Control Bindings, Hold/Toggle semantics, exact keyboard fallbacks scoped to the active Profile, and an independent opt-in hold/latch Voice chord.
• SpeechAnalyzer on macOS 26+ and on-device-required SFSpeechRecognizer on macOS 15–25.
• Adaptive Accessibility and guarded foreground text delivery for native, browser, and validated terminal targets.
• Apple Foundation Models and optional local Ollama refinement behind one typed provider contract.
• A dependency-free Rust retention policy plus bounded, digest-verifying Model- package and Voice History archive validators behind one versioned caller- owned C ABI.
• Atomic, schema-versioned local Profiles and application preferences with explicit migration, corruption recovery, and forward-schema protection.
• A native Controller, History, Profiles, and General shell with Dock and menu-bar presence, searchable local Voice evidence, direct permission recovery, and transient final-only transcript HUD.
• No accounts, analytics, telemetry, cloud APIs, remote storage, or third-party linked runtime dependencies.

These checks require the named local resource or foreground target:

HC_RUN_MICROPHONE_INTEGRATION=1 swift test \
  --filter capturesARealAuthorizedMicrophoneBuffer

HC_RUN_MICROPHONE_INTEGRATION=1 swift test \
  --filter explicitRealMicrophonePreservesTheSystemDefault

HC_RUN_MICROPHONE_ROUTE_INTEGRATION=1 swift test \
  --filter changesTheRealDefaultInputWithoutCrashing

say -v Samantha -o .build/local_transcription_test.aiff \
  "Hardware Controller local transcription test."
HC_SPEECH_AUDIO_FILE="$PWD/.build/local_transcription_test.aiff" \
  swift test --filter modernBackendRepeatedlyTranscribesLocalAudio

HC_RUN_TEXT_INSERTION_INTEGRATION=1 swift test \
  --filter insertsTextIntoTheFocusedRealTextField

HC_RUN_LOCAL_AI_MODEL_EVALUATION=1 swift test \
  --filter LocalAIModelEvaluationTest
HC_RUN_LOCAL_AI_END_TO_END_BENCHMARK=1 swift test \
  --filter measuresWarmReleaseToInsertionWithTheRecommendedModel

HC_RUN_IOS_ASR_PERFORMANCE=1 scripts/check_voice_whisper_bridge.sh

The web, terminal, foreground-event, and complete speech-to-field commands are documented in release validation.

The project is open source under the Apache License 2.0. Copyright remains with Marcus John Rice Lee; intentional contributions are accepted under Apache 2.0 as described in CONTRIBUTING.md. See NOTICE for attribution and BRANDING.md for canonical-project identification. Longdevity LLC formation and ownership transfer remain future work. Source licensing does not approve or publish a binary release.