Almost everything about using AI at work, from speed to quality to cost, comes back to tokens. They are the unit models actually read, write, and bill in, and once you understand them, pricing pages and context-window limits stop being mysterious. This is the practical version: what tokens are, why the same sentence can cost different amounts depending on wording, and where the hidden costs actually accumulate.
What a token actually is
Models don't read words; they read tokens, which are chunks of text produced by a tokenizer before the model ever sees your prompt. A token is roughly four characters of English, or about three-quarters of a word, though the exact split depends on the specific tokenizer a model uses. Common whole words like "the" or "model" are usually a single token. Longer, rarer, or compound words get broken into pieces: a word like "tokenization" might become two or three tokens, and an unusual proper noun or a string of numbers can split into even more. Spaces, punctuation, and line breaks all count as tokens too, which is part of why formatting-heavy text (tables, code, JSON) tends to use more tokens per visible character than plain prose. As a rule of thumb, 1,000 tokens is about 750 words, but that ratio shifts with the content: dense technical text and non-English languages often tokenize less efficiently than everyday English writing.
Why the same word isn't always the same token
Tokenizers work off a fixed vocabulary of common fragments, and how a word breaks down depends on what surrounds it, its capitalization, and whether a space precedes it. "Cat" at the start of a sentence and " cat" mid-sentence can tokenize differently because the tokenizer treats the leading space as part of the token. This is also why translated text, code identifiers, or product names invented by a company often cost more tokens than they look like they should: the tokenizer has never seen that exact fragment as a common unit, so it falls back to smaller, less efficient pieces. None of this is visible in the text you type or read; it only shows up in the token count and the bill.
Input tokens vs. output tokens
You pay for both directions, and providers typically price them differently. Everything you send counts as input: your question, any system instructions, pasted documents or spreadsheet data, and the entire prior conversation the model needs to stay coherent. Everything the model returns counts as output, and providers usually price output tokens several times higher than input tokens, since generating each token is more computationally expensive than reading one. That asymmetry matters in practice: a short, simple-looking prompt that triggers a long, detailed answer, like "write a full project plan" or "draft three versions of this email", can cost far more than a much longer prompt that produces a short answer.
Context windows, and why long chats get more expensive
A model can only attend to a fixed number of tokens at once, and that ceiling is its context window. Crucially, most chat interfaces don't give the model persistent memory between turns; instead, they resend the entire conversation history, plus any system prompt, as input on every single message. That means a thread that started as a quick question can quietly turn into an expensive one by message twenty, because each new turn is billed for re-reading everything that came before it, not just the new question. The same is true for documents: pasting a large spreadsheet or contract into a chat and then asking five follow-up questions means paying to resend that document five times, once per turn, unless the tool is specifically built to avoid that.
Why tokens equal cost, and what quietly inflates them
Providers bill per million tokens, and the rate varies enormously between models, sometimes by an order of magnitude for functionally similar tasks. A frontier model can cost many times what a smaller, faster model does for the exact same request, which means an identical task can be cheap or expensive purely based on which model happens to run it, independent of the prompt itself. A handful of everyday habits are responsible for most of the overspend teams don't notice:
- Pasting entire documents or spreadsheets into a prompt when a relevant excerpt or summary would answer the question just as well.
- Long, static system prompts and instructions that get resent, unchanged, on every single request.
- Conversation histories that grow turn after turn without ever being trimmed or summarized.
- Asking for exhaustive, verbose output (long explanations, multiple drafts, restated context) when a concise answer would serve the same purpose.
- Using a frontier-tier model by default for routine tasks like formatting, simple lookups, or short rewrites that a smaller model handles just as well.
How Switchboard helps
Switchboard makes tokens visible and manageable instead of something that only shows up as a surprise on the invoice. It shows the actual token count and cost for every request and every model, attributes that spend to the person and team who generated it, and routes routine work to smaller, cheaper models automatically so you aren't paying frontier-model rates for tasks that don't need frontier-model reasoning. Understanding tokens is step one; Switchboard is where you actually see them, attribute them, and keep them under control as usage scales across your organization.