SERIES·6 parts · ~35 min total

Learning AI Engineering by Building

I catalogued every AI decision across three production systems and found a consistent pattern — along with five gaps I'd been working around instead of solving.

1.
Auditing My AI Systems: Patterns, Tradeoffs, and Gaps I Was Working Around
I catalogued every AI decision across three production systems and found a consistent pattern — along with five gaps I'd been working around instead of solving.
Mar 23, 20266 min read ai-engineering architecture python
2.
My LLM Pipeline Passed Every Manual Check — Then 36 Tests Proved Otherwise
Five manual runs looked fine. Then 36 automated tests exposed non-deterministic sourcing, biased scoring, and a confidence threshold that fired randomly.
Mar 24, 20265 min read ai-engineering testing python
3.
When Embeddings Fail: Why Vector Search Can't Judge Capability
I added vector search to the screening pipeline and watched it rank a junior frontend developer above a Principal Engineer who processed 1B+ events/day. The embedding model matched vocabulary, not capability.
Mar 25, 20266 min read ai-engineering embeddings rag
4.
Letting the Model Pick Its Own Tools: How Tool Use Inverts Control Flow
The model autonomously combined keyword search and vector search in the optimal sequence — without being told to. Then I ran experiments to measure what vague descriptions, over-calling, and temperature actually do to tool selection.
Mar 26, 20265 min read ai-engineering tool-use python
5.
How ReAct Agents Recover from Their Own Mistakes
ReAct agents recover from their own mistakes — not because the model is clever, but because of how tools return errors and how the loop is structured. Here's what that looks like in practice.
Mar 27, 20267 min read ai-engineering agents python
6.
I Evaluated Fine-Tuning Across 3 Projects — None of Them Needed It
Three projects, three evaluations, zero cases where fine-tuning was justified. Here's the decision framework, the cost math, and why simpler approaches won every time.
Mar 28, 20266 min read ai-engineering fine-tuning python

Start Reading →

Learning AI Engineering by Building

Auditing My AI Systems: Patterns, Tradeoffs, and Gaps I Was Working Around

My LLM Pipeline Passed Every Manual Check — Then 36 Tests Proved Otherwise

When Embeddings Fail: Why Vector Search Can't Judge Capability

Letting the Model Pick Its Own Tools: How Tool Use Inverts Control Flow

How ReAct Agents Recover from Their Own Mistakes

I Evaluated Fine-Tuning Across 3 Projects — None of Them Needed It

Learning AI Engineering by Building

Auditing My AI Systems: Patterns, Tradeoffs, and Gaps I Was Working Around

My LLM Pipeline Passed Every Manual Check — Then 36 Tests Proved Otherwise

When Embeddings Fail: Why Vector Search Can't Judge Capability

Letting the Model Pick Its Own Tools: How Tool Use Inverts Control Flow

How ReAct Agents Recover from Their Own Mistakes

I Evaluated Fine-Tuning Across 3 Projects — None of Them Needed It