Production & efficiency
Engineering note · Part 15
Running AI Work in the Background with FastAPI and Persistent Jobs
Why a saved job lifecycle matters more than a spinner when AI work is slow, retryable and paid.
Read articleBROWSE BY THEME · 8 ARTICLES
Production & efficiency
Durable jobs, saved results, observability and task-specific model selection.
Production & efficiency
Engineering note · Part 16
Reducing Repeated AI Work with Versioned Indexes and Saved Results
How reuse, explicit invalidation and saved artifacts reduce latency without hiding stale evidence.
Read articleProduction & efficiency
Experiment design · Part 30
Choosing Smaller Models for Routine Tasks in an Evidence Application
A task-specific routing experiment that weighs quality, latency and cost without using model size as a safety rule.
Read articleParallel AI
Parallelising Embedding Generation for Large Document Collections
How to design embedding pipelines that scale with batching, partitioning, retries, idempotency and cost-aware concurrency.
Read articleHPC & ML Systems
What Slurm and HPC Scheduling Teach Us About Production ML
Lessons from scientific HPC that apply directly to batch inference, evaluation, document processing and AI pipelines.
Read articleProduction AI
Model Routing: Balancing AI Quality, Cost and Latency
How to route requests across models based on task complexity, risk and response-time requirements.
Read articleProduction Engineering
Structured Logging for Production AI Applications
How request IDs, event names and redaction make AI systems easier to debug and safer to operate.
Read articleAI Systems
Background Jobs for Long-Running AI Workloads
Why document processing, report generation and multi-step AI tasks should often move out of the request-response cycle.
Read article