By the end of this lesson you will have the minimal production-shaped foundation every AI product needs and almost no tutorial shows: environment-based provider configuration, one thin LLM client with timeouts, retries, typed errors, and failover baked in, and request logging that turns "the AI is acting weird" from a mystery into a query. This is the skeleton every subsequent module in this course builds on.

Why a thin client, and why now

The naive path is seductive: install a provider's SDK, sprinkle openai.chat.completions.create(...) calls through your codebase, ship. Three weeks later you need a timeout (the SDK default hangs your request handler), then a retry (rate limits are a when, not an if), then a second provider (Lesson 3's fallback chain), and now you're refactoring forty call sites while users watch spinners.

The fix costs about eighty lines: every LLM call in your product goes through one module you own. Not a framework, not an abstraction layer with plugins — a single file with one exported function. Provider SDKs are fine, but the OpenAI-compatible HTTP shape has become the de facto protocol spoken by most providers and every local server (Ollama, vLLM, LM Studio), so a plain fetch-based client gives you failover across all of them for free.

Rule for the rest of this course: feature code never touches a provider API directly. It calls chat(). Everything operational — timeouts, retries, failover, logging, and later caching and evals — lives behind that seam.

Environment-based provider config

Hardcoded model names are how you end up doing a code deploy to respond to a provider outage. Configuration lives in the environment; code reads it once and validates it at startup:

```bash

.env.local — NEVER commit this file (add to .gitignore now, not later)

LLM_PRIMARY_BASE_URL=https://api.openai.com/v1
LLM_PRIMARY_API_KEY=sk-...
LLM_PRIMARY_MODEL=gpt-4o-mini

LLM_FALLBACK_BASE_URL=https://api.together.xyz/v1
LLM_FALLBACK_API_KEY=...
LLM_FALLBACK_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo

LLM_TIMEOUT_MS=30000
LLM_MAX_RETRIES=2
```

Two rules that repay their cost immediately. Fail fast on missing config: if a required variable is absent, crash at boot with a clear message — a missing key discovered at request time is a 3 a.m. page; discovered at deploy time it's a 30-second fix. Name variables by role, not vendor: LLM_PRIMARY_*, not OPENAI_*. The day you swap providers, it's an env edit and a restart — no rename sweep, no redeploy of code.

Timeouts, retries, and typed errors — the non-negotiables

Three behaviors separate a production client from a tutorial snippet:

Timeouts. LLM APIs occasionally hang — not error, hang. Without a timeout, that hang propagates to your user's browser tab. Every call gets an AbortController deadline. Choose it from your product, not from hope: an interactive feature might cap at 30s; a background batch job might allow 120s.

Retries — but only for retryable failures. A 429 (rate limit) or 500/503 (provider hiccup) deserves a retry with exponential backoff. A 401 (bad key) or 400 (malformed request) does not — retrying those just burns time and log noise on a failure that cannot succeed. The difference between the two categories is exactly what your error types must encode.

Typed errors. catch (e) { console.log(e) } tells your feature code nothing. Your client should throw one error class with a machine-readable kind — timeout, rate_limit, provider_error, bad_request, auth — so callers can decide: show "try again," fail silently, or page you. Errors are part of the module's API surface, and in an LLM product they fire daily.

Failover ties it together: if the primary provider exhausts its retries on a retryable failure, the same request is replayed against the fallback provider before the caller ever sees an error. That's Lesson 3's fallback chain, implemented in a for-loop.

Logging you'll thank yourself for

Here's the scenario this section prevents. A customer emails: "your AI gave me a completely wrong answer yesterday around 3pm." Without request logging you have nothing — no prompt, no output, no model name, no way to reproduce. With it, you have a JSON line you can grep in ten seconds.

Log one structured line per LLM request with: a request ID, timestamp, provider and model actually used (after failover — this matters), token counts in and out, latency, finish status, and error kind on failure. In development, log the prompt and response bodies too; in production, decide deliberately based on your privacy posture (log bodies with retention limits, hash them, or store a pointer to a redacted copy).

This one habit gives you, for free: per-feature cost tracking (sum the token counts — Lesson 3's model, but with real data), latency percentiles, failover frequency per provider, and reproducible bug reports. Every observability feature in the later modules builds on this line.

The client

The artifact for this lesson — a complete, self-contained module. Drop it in src/lib/llm-client.ts, set the env vars above, and every lesson from here on imports from it.

```typescript
// src/lib/llm-client.ts — fetch-based, OpenAI-compatible, failover-ready.
// Zero dependencies. Node 18+ (built-in fetch).

export type Role = "system" | "user" | "assistant";
export interface ChatMessage { role: Role; content: string; }

export interface ChatOptions {
temperature?: number;
maxTokens?: number;
timeoutMs?: number;
}

export interface ChatResult {
content: string;
model: string;
provider: string;
inputTokens: number;
outputTokens: number;
latencyMs: number;
}

export type LLMErrorKind = "timeout" | "rate_limit" | "provider_error" | "bad_request" | "auth";

export class LLMError extends Error {
constructor(public kind: LLMErrorKind, public provider: string, message: string) {
super([${provider}] ${kind}: ${message});
}
get retryable() { return this.kind === "timeout" || this.kind === "rate_limit" || this.kind === "provider_error"; }
}

interface Provider { name: string; baseUrl: string; apiKey: string; model: string; }

function env(key: string): string {
const v = process.env[key];
if (!v) throw new Error(Missing required env var: ${key}); // fail fast at boot
return v;
}

const PROVIDERS: Provider[] = [
{ name: "primary", baseUrl: env("LLM_PRIMARY_BASE_URL"), apiKey: env("LLM_PRIMARY_API_KEY"), model: env("LLM_PRIMARY_MODEL") },
...(process.env.LLM_FALLBACK_BASE_URL
? [{ name: "fallback", baseUrl: env("LLM_FALLBACK_BASE_URL"), apiKey: env("LLM_FALLBACK_API_KEY"), model: env("LLM_FALLBACK_MODEL") }]
: []),
];

const TIMEOUT_MS = Number(process.env.LLM_TIMEOUT_MS ?? 30_000);
const MAX_RETRIES = Number(process.env.LLM_MAX_RETRIES ?? 2);

async function callProvider(p: Provider, messages: ChatMessage[], opts: ChatOptions): Promise<ChatResult> {
const started = Date.now();
const ctrl = new AbortController();
const timer = setTimeout(() => ctrl.abort(), opts.timeoutMs ?? TIMEOUT_MS);
try {
const res = await fetch(${p.baseUrl}/chat/completions, {
method: "POST",
headers: { "Content-Type": "application/json", Authorization: Bearer ${p.apiKey} },
body: JSON.stringify({
model: p.model,
messages,
temperature: opts.temperature ?? 0,
...(opts.maxTokens ? { max_tokens: opts.maxTokens } : {}),
}),
signal: ctrl.signal,
});
if (!res.ok) {
const body = (await res.text()).slice(0, 500);
if (res.status === 401 || res.status === 403) throw new LLMError("auth", p.name, body);
if (res.status === 429) throw new LLMError("rate_limit", p.name, body);
if (res.status >= 500) throw new LLMError("provider_error", p.name, body);
throw new LLMError("bad_request", p.name, body);
}
const data = await res.json();
return {
content: data.choices?.[0]?.message?.content ?? "",
model: data.model ?? p.model,
provider: p.name,
inputTokens: data.usage?.prompt_tokens ?? 0,
outputTokens: data.usage?.completion_tokens ?? 0,
latencyMs: Date.now() - started,
};
} catch (e) {
if (e instanceof LLMError) throw e;
if ((e as Error).name === "AbortError") throw new LLMError("timeout", p.name, no response in ${opts.timeoutMs ?? TIMEOUT_MS}ms);
throw new LLMError("provider_error", p.name, (e as Error).message);
} finally {
clearTimeout(timer);
}
}

/** Every LLM call in the product goes through this function. */
export async function chat(messages: ChatMessage[], opts: ChatOptions = {}): Promise<ChatResult> {
const requestId = crypto.randomUUID();
let lastError: LLMError | undefined;

for (const provider of PROVIDERS) {
for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {
if (attempt > 0) await new Promise((r) => setTimeout(r, 500 * 2 ** (attempt - 1))); // backoff: 500ms, 1s
try {
const result = await callProvider(provider, messages, opts);
console.log(JSON.stringify({ evt: "llm_request", requestId, ok: true, provider: result.provider,
model: result.model, inputTokens: result.inputTokens, outputTokens: result.outputTokens,
latencyMs: result.latencyMs, attempt }));
return result;
} catch (e) {
lastError = e as LLMError;
console.log(JSON.stringify({ evt: "llm_request", requestId, ok: false, provider: provider.name,
kind: lastError.kind, attempt }));
if (!lastError.retryable) break; // auth/bad_request: retrying cannot succeed — stop attempts here
}
}
if (lastError?.kind === "bad_request") break; // malformed request fails on every provider; auth may differ, so fall through
}
throw lastError!;
}
```

A quick smoke test:

```typescript
// smoke.ts — npx tsx smoke.ts
import { chat } from "./src/lib/llm-client";
const r = await chat([{ role: "user", content: "Reply with exactly: client online" }]);
console.log(r.content, (${r.provider}, ${r.latencyMs}ms, ${r.inputTokens}+${r.outputTokens} tok));
```

Read the chat() loop once more before moving on — providers outer, retries inner, backoff only on retry, log every attempt. That ordering is the fallback chain from Lesson 3, and the JSON log lines are the raw material for the cost dashboard you'll build later.

Exercise

Stand it up for real: create a fresh project (npm init -y && npm i -D tsx typescript), add the client and .env.local with your primary provider, and run the smoke test. Then prove the failure paths: (1) set LLM_TIMEOUT_MS=1 and confirm you get a thrown LLMError with kind: "timeout" and a matching log line, not a hang; (2) break the API key and confirm kind: "auth" with no retry attempts in the logs; (3) if you have a second provider (a free local Ollama server counts), configure it as fallback, point the primary at a garbage URL, and confirm the request succeeds with provider: "fallback" in the result. Success looks like: all three failure drills behave exactly as described, and you can paste the three JSON log lines that prove it.

Your foundation is production-shaped from line one — in Module 2 we start building on it: structured outputs your code can trust, tool calling that lets the model drive real functions, and the retrieval layer that feeds it your data. That's where the product starts.