You just got paged at 2am because the model is hallucinating
Your ML model trained on 5 years of labeled customer data is confidently generating support tickets that never happened. Your team’s trained on classifier debugging — check the confusion matrix, retrain on edge cases, deploy. But that playbook doesn’t work here. The model didn’t misclassify; it invented facts.
This is the problem GenAI creates, and it’s not the same problem AI has always been.
The 60-year context, compressed
AI as a field started with symbolic reasoning — hand-coded rules, expert systems, search over explicit knowledge representations. It was interpretable and provably correct within its domain, and it collapsed the moment reality got messier than the rules anticipated.
Machine learning shifted the burden from “hand-write the rules” to “learn the rules from data” — huge win for pattern recognition, still mostly narrow: one model, one task, one dataset. You built a classifier for spam detection, it detected spam. You trained it on your data, you knew what it would do.
Deep learning scaled that further with representation learning — the model learns its own features instead of you engineering them. Better results, same game: input → fixed prediction.
What GenAI adds isn’t just “bigger deep learning.” It’s three things that changed the shape of the engineering problem:
1. Self-supervised pretraining at scale
Instead of needing labeled data for every task, the model learns from the structure of raw text/image/audio itself — predict the next token, reconstruct the masked patch. This is the unlock that let one model absorb enough general knowledge to be useful across tasks it was never explicitly trained on.
Task-specific ML needed a dataset and a training run per task. GenAI needs a prompt.
The training signal is almost trivially simple. For language models, it’s “given the first N tokens, predict token N+1.” No labels. No curated dataset. Just scale up the pretraining on public text, and emergent capabilities fall out.
2. Emergent capability, not just accuracy
A model trained purely to predict the next token turns out to do arithmetic, translate languages, write code, and reason step-by-step, without being explicitly taught any of those as separate objectives.
That was not the design goal — it fell out of scale.
This is why GenAI conversations sound different from ML conversations: you’re not asking “did the model converge,” you’re asking “what can it do that we didn’t train it to do?”
If you’ve worked with traditional ML, this is foreign territory. A classifier doesn’t suddenly learn to do tasks it wasn’t trained on. A language model at sufficient scale does — and in ways that surprise even the teams that trained it.
3. Instruction-following as an interface
RLHF (Reinforcement Learning from Human Feedback) and instruction-tuning turned a raw next-token predictor into something you can talk to — a system with a natural language API instead of a fixed schema.
This is the part that actually changed engineering practice. The interface to the model is now prompts, not feature vectors. That changes everything downstream:
- How you version behavior (it’s a prompt, not a model artifact)
- How you test it (what does sensible mean?)
- How you debug failures (looks like misunderstanding, not a bug)
Before instruction-tuning, you got raw model outputs. After, you got something that understood intent and conversational context. That’s a usability cliff.
Where “just AI” undersells it
If you’ve worked in ML — training classifiers, tuning hyperparameters, managing feature pipelines — GenAI can feel like the same job with an LLM API swapped in for a model.pkl file.
It isn’t, for one structural reason: the failure modes are different.
A classifier fails predictably:
- Wrong class ✗
- Confidence too low ✗
- Distribution shift ✗
An LLM fails in ways that look like reasoning errors, hallucinated facts, or inconsistent behavior across near-identical prompts. Debugging a GenAI system looks more like debugging a very smart, very literal, occasionally overconfident new hire than debugging a numerical model.
That’s a different discipline, even though the underlying math (gradient descent, attention, backprop) hasn’t changed.
When you train an ML model and deploy it, you’re shipping a fixed function with known input/output shapes. When you deploy a GenAI system, you’re shipping an interface to a stochastic reasoning process. The testing and debugging move from “did the model converge” to “does it understand what I’m asking and give sensible answers.”
Where the skeptics have a point
The “it’s just autocomplete” critique isn’t wrong about the mechanism — GPT-style models are trained to predict the next token, full stop.
Where the critique falls short is treating the training objective as the ceiling on what the system can do. Next-token prediction is the training signal, not a description of the system’s capability once trained at scale. Ignoring emergent behavior because the training objective looks simple is like dismissing a chess engine’s play because it’s “just minimax” — technically true, doesn’t explain why it beats grandmasters.
The “it hallucinates, so it’s not real AI” critique also misses the point. Yes, language models confidently generate false facts. That’s a real failure mode you have to design around (hence RAG, grounding, retrieval). But it’s not evidence that it’s not doing reasoning — it’s evidence that it’s reasoning under uncertainty without access to ground truth, which is exactly what a statistical model does.
The hallucination problem is a design problem, not a fundamental limitation.
Why this matters for what you build next
Once you internalize that the shift is (a) self-supervised scale, (b) emergent capability, and (c) instruction-following as an interface, the rest of the modern stack makes more sense:
RAG exists because instruction-following models are fluent but not grounded — they need retrieval to anchor answers in facts they weren’t trained on.
Agents exist because instruction-following plus tool use turns a text generator into something that can act, not just answer.
Fine-tuning still matters but is now a narrower tool — you’re steering an already-general model toward a domain, not building capability from zero.
Vector databases and knowledge graphs matter because they’re the external memory that grounds the model in your specific facts.
Reasoning engines (logic programming, constraint solvers) matter because LLMs are probabilistic and sometimes you need guarantees. “This is definitely consistent with policy” is not a job for an LLM; it’s a job for a rule engine.
This is why the next posts in this series go where they go. Each one is downstream of understanding this shift, not a separate topic.
The takeaway
GenAI isn’t “AI, but with better PR.” It’s what happens when you combine self-supervised pretraining at scale with an instruction-following interface, and get emergent capability as a side effect nobody explicitly engineered.
That’s a real discontinuity in how you build systems — worth taking seriously whether you’re the one hyping it or the one rolling your eyes at the hype.
Next in this series: Building LLMs from Scratch: Decoder-Only Transformers & DeepSeek R1