Understanding Large Language Models

If you are new to this topic, start with Getting Started with AI for a broad overview before diving in here.

How LLMs Work

Large language models are neural networks trained to predict the next token in a sequence. A token is roughly a word fragment — "understanding" might be split into "under" + "standing". Models learn from hundreds of billions of tokens drawn from web pages, books, and code.

At inference time the model takes an input sequence (the prompt) and generates output one token at a time, with each token conditioned on everything that came before it.

Context Windows

Every model has a context window — the maximum number of tokens it can hold in memory at once. Modern models range from 8 K to 1 M+ tokens. Larger windows allow you to feed in entire codebases, long documents, or extended conversation history.

Training vs Inference

| Phase | What Happens | Who Pays | |-------|-------------|----------| | Pre-training | Model learns from raw data | AI lab (very expensive) | | Fine-tuning | Model is adapted to a task | Lab or developer | | Inference | Model generates output for a prompt | Developer / user |

Prompting Techniques

How you phrase a prompt dramatically affects output quality.

• Zero-shot — give the task only, no examples: "Summarize this article."
• Few-shot — include 2–5 examples before the task to guide the format
• Chain-of-thought — ask the model to reason step by step: "Think through this step by step before answering."
• System prompts — set the model's role and constraints at the start of a conversation

Model Comparison

| Model | Provider | Strengths | |-------|----------|-----------| | GPT-4o | OpenAI | Multimodal, broad capability | | Claude 3 Opus | Anthropic | Long context, nuanced reasoning | | Gemini 1.5 Pro | Google | 1 M token context, multimodal | | Llama 3 | Meta | Open weights, runs locally |

What to Read Next

Once you understand the models, Building AI-Powered Applications shows how to connect these models to real APIs and build production pipelines.