Core AI, LLMs, Prompt Engineering & Agentic Loops (30 Q&As + Code)

Prompt/KV caches hash prefixes sequentially starting from token 0. If even 1 character changes near the top of the prompt, the cache miss cascades through all 6,500 static instructions and tool JSON schemas below it, forcing full uncached prefill billing on every turn. Always freeze static rules + tool schemas first and inject timestamps/user context at the bottom of the user message.

// BAD: breaks 90% prefix cache hits on every call
const badPrompt = `Time: ${new Date().toISOString()}\n${STATIC_SYSTEM_RULES}`;

// GOOD: identical token prefix shares KV cache across all users
const messages = [
  { role: 'system', content: STATIC_SYSTEM_RULES },
  { role: 'user', content: `[Time: ${new Date().toISOString()}]\n${userQuery}` }
];

MCP Stdio transport sends newline-delimited JSON-RPC 2.0 frames strictly over stdout. Writing human log lines into stdout corrupts JSON framing and crashes the host parser. Always route debug logs to stderr (console.error or sys.stderr).

import sys, json
# BAD: print("Starting MCP server...") -> corrupts stdout JSON-RPC
print("MCP server booting...", file=sys.stderr, flush=True)
sys.stdout.write(json.dumps({"jsonrpc": "2.0", "id": 1, "result": {}}) + "\n")
sys.stdout.flush()