# Mo Khaled ## Pages - [About](https://mokh.tech/about.md): About a software builder with ten years’ experience in production software and content platforms, now developing AI products and their documentation. ## Lessons - [Indirect prompt injection: the under-defended attack](https://mokh.tech/lessons/indirect-prompt-injection-the-under-defended-attack.md): Explains indirect prompt injection through poisoned documents and why retrieval pipelines need validators to detect malicious instructions. - [Cost: per task, not per token](https://mokh.tech/lessons/cost-per-task-not-per-token.md): Calculates AI model cost per task by accounting for token prices, retries, and self-hosted GPU costs before deployment. - [Observability: the trace that explains a decision](https://mokh.tech/lessons/observability-the-trace-that-explains-a-decision.md): Lesson 15 explains production observability traces that capture why a RAG model made a decision, including retrieval, responses, and audit fields. - [The twelve components: an LLM system is mostly not the LLM](https://mokh.tech/lessons/the-twelve-components-an-llm-system-is-mostly-not-the-llm.md): The lesson maps twelve components of production AI systems, with the LLM as component three and the other eleven accounting for 80–90% of project cost and... - [Diagnosis: retrieval, chunking, or generation failure](https://mokh.tech/lessons/diagnosis-retrieval-chunking-generation-failure.md): Diagnoses wrong RAG answers by distinguishing retrieval, chunking, and generation failures and matching each mode to fixes. - [Parent-child retrieval: retrieve small, serve big](https://mokh.tech/lessons/parent-child-retrieve-small-serve-big.md): Parent-child retrieval separates small chunks for precise retrieval from larger parent chunks served to the LLM in RAG systems. - [Reranking: precision purchased with latency](https://mokh.tech/lessons/reranking-precision-purchased-with-latency.md): Reranking moves relevant chunks from hybrid retrieval’s top twenty into the top three, using a second encoder at measurable latency cost. - [Retrieval: BM25 vs dense vs hybrid](https://mokh.tech/lessons/retrieval-bm25-vs-dense-vs-hybrid.md): Explains BM25, dense, and hybrid retrieval for RAG, including BM25’s lexical scoring, failure modes, and why production systems combine retrievers. - [Chunking: the decision a retriever cannot undo](https://mokh.tech/lessons/chunking-the-decision-a-retriever-cannot-undo.md): Explains how chunking determines what information a RAG retriever can find and cannot recover after documents are split. - [KV cache: the serving bottleneck](https://mokh.tech/lessons/kv-cache-the-serving-bottleneck.md): Explains how KV caching reduces repeated attention computation in LLM serving by storing keys and values, while making memory the main cost. - [Sampling: temperature, top-k, top-p](https://mokh.tech/lessons/sampling-temperature-top-k-top-p.md) - [Forward pass: what actually happens when an LLM emits one token](https://mokh.tech/lessons/forward-pass-what-actually-happens-when-an-llm-emits-one-token.md): Explains the seven-stage pipeline involved when a large language model emits one token. - [Tokenisation: the cost surface of LLMs](https://mokh.tech/lessons/tokenisation-the-cost-surface-of-llms.md): Explains how tokenisation, rather than word counts alone, affects estimating the cost of deploying LLM-based customer support agents. - [Agent Safety Audit: The Layer 10 Eval Gate](https://mokh.tech/lessons/agent-safety-audit-the-layer-10-eval-gate.md): Explains a Layer 10 evaluation gate requiring a bounded agent contract, failure-mode coverage, and trace-replay evidence for deployment approval. - [Agent Failure Mode Heatmap](https://mokh.tech/lessons/agent-failure-mode-heatmap.md): Lesson 36 presents a heatmap of predictable agent failure modes, including fabricated facts, semantically wrong tool parameters, and ignored constraints. - [Bounded Agent Trace Playground](https://mokh.tech/lessons/bounded-agent-trace-playground.md): Lesson on bounded agents, budget limits, web-search tool traces, and auditable agent behavior. - [Workflow vs Agent Decision Matrix](https://mokh.tech/lessons/workflow-vs-agent-decision-matrix.md): Explains the distinction between workflows and agents using a customer-service system with fixed steps as a workflow example. ## Insights - [The Web Is Getting a Second Reader](https://mokh.tech/insights/the-web-is-getting-a-second-reader.md): The article discusses designing web pages for AI agents that read sites to answer questions, compare options, and complete tasks. - [AI Compliance in Europe Is an Architecture Spec](https://mokh.tech/insights/ai-compliance-in-europe-is-an-architecture-spec.md): Explains how the EU AI Act’s phased obligations and enforcement authorities make compliance an architecture requirement for AI systems serving European org... - [The Eval Set Is the Alignment Document](https://mokh.tech/insights/the-eval-set-is-the-alignment-document.md): The article argues that an evaluation set aligns AI projects by making differing goals explicit and testable. - [The Model Is Not the System](https://mokh.tech/insights/the-model-is-not-the-system.md): AI engineering failures often stem from retrieval, chunking, validation, observability, and evaluation rather than model selection. - [What AI Is Doing to My Head](https://mokh.tech/insights/what-ai-is-doing-to-my-head.md): An engineer examines how daily use of AI coding agents increased productivity while weakening engineering skills during an aggressive automation experiment. - [Deterministic Habits, Non-Deterministic Software](https://mokh.tech/insights/deterministic-habits-non-deterministic-software.md): Explains how LLM systems replace deterministic input-output behavior with probabilistic outputs and why software development practices must adapt. ## Products - [LLM Doctor](https://mokh.tech/products/llm-doctor.md): LLM Doctor sends a controlled OpenTelemetry span to PostHog, verifies it through the API, and identifies one of ten pipeline failure modes. - [TriageFast](https://mokh.tech/products/triagefast.md): API-first feedback triage that analyzes input from email, APIs, and support tools, then creates structured work in GitHub Issues, Linear, Notion, or webhooks. - [Mokhai — Agent Readiness Kit](https://mokh.tech/products/mokhai-agent-readiness-kit.md): Mokhai provides clean Markdown versions of WordPress URLs and a curated /llms.txt index for AI agents. - [evalspec](https://mokh.tech/products/evalspec.md): evalspec measures agent behavior against gold labels for answers, abstentions, clarifications, or tool calls, - [RAG Probe](https://mokh.tech/products/rag-probe.md): RAG Probe benchmarks RAG pipelines end to end, identifying failures in retrieval, chunking, or generation using a configurable experiment matrix. - [PHP AI Client](https://mokh.tech/products/php-ai-client.md): PHP AI Client is a PHP SDK providing one interface for calling multiple LLM API providers, developed with the WordPress Core AI Team.