{ Local Models & Offline Guide }

LOCAL MODELS & PRIVATE PRACTICE

● Last checked: October 2026

Run interview practice 100% offline or with zero cloud egress using in-browser WebGPU models or local server engines like Ollama and LM Studio.

> ZERO EGRESS GUARANTEE:
> In local mode, zero prompts, resumes, answers or transcripts leave your machine.
> In-browser models are cached in IndexedDB/OPFS; local servers run on loopback (127.0.0.1).
> Completely free and functional offline without an API key.

1. Device Requirements (PC Only, Text Only)

Running real language models locally requires desktop-class GPU or CPU compute and dedicated system memory. To protect device stability, Abhyas applies two strict rules:

PC Only (Phones Blocked): Local models are restricted to desktop and laptop PCs (Windows, macOS, Linux). Mobile phones and tablets are blocked (FR-022, ADR-013) due to thermal throttling, mobile OS aggressive background kills, and lack of full WebGPU compute shader pipelines. On phones, please use cloud providers (Gemini free tier) or practice in text mode.
Strictly Text-Only: Local model practice is 100% text-only (FR-022, ADR-013). No microphone prompt appears and no speech synthesis services are constructed. This reserves 100% of memory bandwidth and GPU compute for inference without audio chopping or dropped frames.
WebGPU Browser Support: For in-browser models, you need a WebGPU-enabled browser such as Google Chrome (113+), Microsoft Edge (113+), or Arc.

2. Hardware Tiers & Curated Models

Abhyas features an automated micro-benchmark that tests token decoding speed on your device to recommend an optimal model tier.

Hardware Tier	Minimum Specs	Curated Model	Download Size	Expected Speed
Basic	8 GB RAM, Integrated GPU / CPU	SmolLM2 1.7B / Qwen2.5 1.5B	~1.0 GB	< 15 tokens/sec
Standard	16 GB RAM, Modern M1/M2 or GPU	Llama 3.2 3B / Qwen2.5 3B	~2.0 GB	15 – 30 tokens/sec
Advanced	16+ GB RAM, Dedicated 6GB+ VRAM	Llama 3.1 8B / Qwen2.5 7B	~4.9 GB	30+ tokens/sec

3. In-Browser WebGPU vs. Local Servers

Abhyas gives you two ways to practice with local models:

Feature	In-Browser (WebGPU / WebLLM)	Local Server (Ollama / LM Studio)
Installation	Zero install: 1-click download in browser	Requires installing Ollama or LM Studio app
Model Formats	Pre-quantized GGUF / MLC weights	Any model pulled into Ollama or LM Studio
Execution Speed	WebGPU compute shaders	Native C++ / CUDA / Metal (often 20-40% faster)
Cross-Origin (CORS)	Not applicable (runs inside browser tab)	Requires setting `OLLAMA_ORIGINS="*"`

4. Ollama Setup & CORS Configuration

Ollama is a popular tool for running models locally. Because Abhyas runs as a web application, your browser enforces the Same-Origin Policy (CORS). By default, Ollama blocks web browser connections unless the OLLAMA_ORIGINS environment variable is configured.

Install Ollama

Download and install Ollama from ollama.com ↗.

Pull an interview model

Open your terminal or command prompt and pull a recommended model:

ollama pull llama3.2

Tip: For lower memory usage, pull qwen2.5:1.5b. For higher quality, pull llama3.1:8b.

Configure CORS (Cross-Origin Resource Sharing)

Configure Ollama to allow connections from your browser:

On Windows:

Option A: Set in terminal before starting:

set OLLAMA_ORIGINS=* && ollama serve

Option B (Permanent): Open System Properties > Environment Variables, add a User Variable named OLLAMA_ORIGINS with value *, then restart Ollama from the taskbar tray.

On macOS:

launchctl setenv OLLAMA_ORIGINS "*"

After running the command above, quit and reopen Ollama from your Applications folder.

On Linux (systemd):

sudo systemctl edit ollama.service

Add these lines under the [Service] section:

[Service] Environment="OLLAMA_ORIGINS=*"

Save and run sudo systemctl daemon-reload && sudo systemctl restart ollama.

Connect in Abhyas

Navigate to AI Setup in Abhyas, select "Ollama (local server)", verify the URL is http://localhost:11434, and click "Test connection". Your installed models will auto-populate in the model picker!

5. LM Studio & OpenAI-Compatible Local Servers

If you use LM Studio, LocalAI, or llama.cpp server, you can connect them via Abhyas's OpenAI-compatible adapter:

Open LM Studio, click the "Local Server" icon in the sidebar.
Load your preferred model and click "Start Server" (defaults to port 1234).
In the Server Settings pane, ensure the "Enable CORS" toggle is turned ON.
In Abhyas AI Setup, select "OpenAI-compatible", set the Base URL to http://localhost:1234/v1, leave the API key blank, and click "Test connection".

6. Honesty: What "Basic Feedback" Means

Abhyas is built on absolute honesty (FR-012, AI-014). Local models under 8 billion parameters excel at asking relevant technical questions and checking for keyword-level accuracy, but they cannot perform the subtle, multi-dimensional calibration of frontier cloud models.

Whenever a session is evaluated by a small local model, your scorecard is clearly labeled with a "Basic Feedback" badge. We never pretend a 3B model produces the same depth as a 70B model. It gives you immediate, private practice without misleading claims.