ML Engineer

What an ML Engineer does · The ML lifecycle and MLOps maturity · From notebook to a structured, packaged ML codebase · Environments, dependencies and reproducibility · The platform and the DS-to-MLE handoff

From a pile of scripts to a DAG · Build a pipeline with targets · Caching, invalidation and incremental runs · Parameterized and dynamically-branched pipelines · Scheduling and orchestration · Pipeline testing and debugging

Data validation and contracts with pointblank · Schema enforcement and data-quality gates in the pipeline · Data versioning and lineage · Feature pipelines that do not leak · Columnar and larger-than-memory data · Batch, incremental and change-data-capture ingestion

Serializing models and objects · Bundling preprocessing with the model · Packaging ML code as an R package · Model versioning with vetiver · A model registry and promotion workflow · Model cards and metadata as first-class, queryable artifacts

The serving landscape · REST APIs with plumber · Serving a tidymodels / vetiver model end to end · Throughput and latency · Multi-model serving and version routing · Serving runtimes and when to leave R · Serverless, edge and on-device inference

Docker for R · Multi-stage builds and slim, secure images · Configuration and secrets management · Deploying · Health checks, readiness and graceful shutdown

Unit and integration testing for ML code (testthat) · Testing data · Testing models · Testing the serving layer · Evaluation sets and golden datasets · Regression testing for models · Slice-based and subgroup evaluation · Offline vs online evaluation, and eval-driven development

CI for ML · Continuous delivery · Continuous training (CT) · Model approval gates and automated rollback · Infrastructure as code, an intro

What to monitor · Data drift detection · Concept drift and silent performance decay · Alerts and thresholds without alert fatigue · Closing the loop

The three pillars · Distributed tracing across a prediction request · The prediction store and eval-in-production (shadow scoring, delayed-label backfill) · The full drift taxonomy · Multivariate and embedding drift · Retraining strategy · Online and incremental learning · Feedback loops and their dangers

Feature stores · Point-in-time correctness and time-travel joins · Feature freshness, the dual-write problem and serving latency · Feature versioning, lineage and reuse · The lakehouse and medallion architecture for ML · Streaming features and real-time pipelines

Shadow deployment · Canary and blue-green releases · Online A/B testing of models · Interleaving and multi-armed rollout · Progressive rollout, guardrail metrics and automated rollback · Multi-model serving, routing and champion/challenger

Parallel R · Out-of-core and chunked processing · arrow and duckdb at warehouse scale · Distributed compute with Spark via sparklyr · Distributed and multi-GPU training · Autoscaling the serving layer

The accuracy-latency-cost-throughput frontier · Quantization · Pruning, distillation and simpler-model substitution · Compilation and kernels · LLM-specific · Profiling R hot paths and the hot 5% · Latency budgets and caching · The economics of AI

SLOs, SLIs and error budgets for ML services · Graceful degradation and fallback models · Load, soak and chaos testing for ML · Incident response, on-call and blameless postmortems for ML · Capacity planning and the cost of reliability

Securing an ML/AI API · The ML attack surface · LLM security · Model and data supply chain · Privacy in production · Governance and compliance

The LLM serving stack · Calling and routing LLMs from R with ellmer · Streaming responses (SSE) and token-level UX · Throughput · Model routing and fallback · Self-hosted vs API

The RAG architecture · Chunking and the indexing pipeline · Embeddings and vector search · Hybrid search and re-ranking · Retrieval evaluation · Freshness, incremental indexing and cache invalidation · RAG observability and failure analysis

From chains to agents · Agent architectures · Orchestration, state and memory · Agent observability and tracing · Evaluating agents · Cost and latency control for multi-call agents · Guardrails and human-in-the-loop for agents

Why LLM eval is hard · Building eval sets and golden datasets for LLM tasks · LLM-as-judge · Reference-based and reference-free metrics · Human eval pipelines and inter-rater reliability · Online eval and regression suites in CI · Red-teaming and adversarial evaluation

Guardrails · Content safety, toxicity and PII redaction · Hallucination mitigation and grounding · Prompt management · Fine-tuning and adapter ops · Semantic caching and cost/latency control for LLM apps · Monitoring LLMs in prod

The system-design framework · Latency, scale and cost trade-offs · Case · Case · Case · Case · Case · The ML/AI system design interview