WHAT HAPPENS WHEN YOU TALK TO AI: THE PROCESS EXPLAINED FOR OLDER ADULTS

Introduction

When you type a message into ChatGPT or Claude, it feels immediate - you hit send, and a few seconds later you get a response. Simple, right?

Not quite. There's a lot happening behind the scenes, and understanding that process helps you use AI more effectively and understand why it sometimes behaves in ways that seem strange.

Your message goes to a server

First thing: AI isn't running on your computer but instead when you type a message, it's sent over the internet to the company's servers (OpenAI's, Anthropic's, Google's, whoever runs the AI you're using).

The AI model itself is enormous and too big to fit on a phone or laptop, so everything happens remotely. You're essentially sending your question to a data centre where the AI processes it and sends back a response.

This matters for privacy because your conversation isn't private to you and your device but is being processed by the company's servers. Most companies claim they don't use your conversations to train future models unless you explicitly opt in, but they do have access to what you're saying, so if you're typing something sensitive (personal information, confidential business details), you're trusting the company to handle it properly.

Some companies offer enterprise versions with stronger privacy guarantees, but the free versions? Assume your conversation could be seen by someone.

The AI reads the entire conversation (every time)

Here's something most people don't realise: AI doesn't remember previous messages in the way you do. Instead of having a memory of "we were talking about X earlier", every time you send a message the AI re-reads the entire conversation from the start.

It looks at everything you've said and everything it's said, then predicts what should come next. This is why AI seems to "remember" context - it's not memory, it's re-reading.

This works well for short conversations, but it has limits.

What "context window" means (and why AI forgets)

AI can only process a certain amount of text at once, and this limit is called the context window. Think of it like working memory - how much the AI can hold in its head at one time.

If your conversation gets too long, the earliest messages eventually fall out of the context window and the AI can't see them anymore. From its perspective, they never happened.

This is why AI sometimes "forgets" things you told it earlier - it's not being inconsistent or buggy, it's just that the information you gave it fifty messages ago has been pushed out of the window and the AI genuinely can no longer see it. Different models have different context windows where some can handle very long conversations (tens of thousands of words) while others are more limited, but they all have a cap eventually.

Practically, this means that if you're having a long conversation and the AI starts ignoring something important you said at the beginning, it's probably because that information is now outside the context window. You might need to start a new conversation and restate the key points, or just remind the AI of what you told it earlier - it's not being difficult, it literally can't see that part of the conversation anymore.

Why the same question gets different answers

If you ask AI the same question twice, even in the same conversation, you might get different answers. This confuses people, but it's not a bug - it's how prediction works.

AI generates responses probabilistically, which means it's not retrieving a fixed answer from a database where "question X always gets answer Y" but instead predicting what words should come next based on your question and the conversation so far, and those predictions have a degree of randomness built into them.

Most AI systems have a setting called "temperature" that controls this randomness, and although you usually don't see it as a user, it's there. A higher temperature means more randomness (more creative, more varied, sometimes more wrong) while a lower temperature means more predictability (more consistent, but sometimes more boring and repetitive).

Even with the same temperature setting, slight differences in how you phrase the question, where you are in the conversation, or just the inherent randomness in the prediction process can lead to different responses. This is normal behaviour - it's not the AI being unreliable (well, not more unreliable than it already is!) but just the nature of probabilistic text generation.

What happens during generation

When the AI starts responding, it's not writing the whole answer at once and then showing it to you but instead generating one word (or more precisely, one "token" which is roughly three-quarters of a word) at a time, in sequence.

You see the words appear as they're generated, which is why responses seem to stream in rather than appearing all at once. The AI predicts the first word based on your message and the conversation history, then uses that first word plus everything that came before to predict the second word, then uses the first and second words to predict the third, and so on.

It's thinking out loud, if you can call it thinking, where each word depends on the words that came before it.

This is why AI sometimes starts an answer confidently and then trails off into nonsense halfway through - it committed to a direction early on because that's what the prediction said to do, and then had to keep going even though the direction turned out to be wrong or the logic broke down. It can't backtrack because it's locked into the path it started down.

Humans can stop mid-sentence and say "Actually, that's not right, let me rethink this" because we can recognise when we're heading in the wrong direction and correct course. AI can't do that - once it's started generating a response, it has to finish, and if the prediction starts to go off course halfway through, it's too late to change direction. It just keeps predicting the next word until it reaches what it thinks is a natural stopping point.

Privacy: what AI companies can see

Most AI companies state that they don't use your conversations to train their models unless you give explicit permission, and that's the official policy. Whether it actually gets followed in practice and whether or not you trust that policy is up to you.

What they definitely do have is access to your conversations, at least temporarily, because they need that access to process your messages and generate responses. The question is what they do with it after that and whether or not they store those conversations, for how long, who can see them, and what safeguards are in place all varies by company and by which version of the service you're using.

If you're using a free version of an AI tool, assume lower privacy protections where the terms of service are usually vaguer and the data handling less strict. If you're using an enterprise or paid version, there are usually stronger guarantees written into the contract, but even then read the terms of service if privacy actually matters to you.

Remember the basic principle: anything you type into an AI could, in principle, be seen by someone at the company. Maybe it won't be, maybe it's only seen by automated systems or deleted immediately or encrypted in ways that make it effectively private, but it could be seen. So don't put anything in a conversation with AI that you wouldn't want another human being to read.

Why understanding this helps

Knowing how AI processes your messages makes it much easier to use the tool effectively and understand when it's going wrong.

You understand why long conversations get confused and contradictory (it's the context window limit, not the AI being stupid or broken), why AI sometimes contradicts itself or forgets what you told it (it's re-reading the conversation each time rather than actually remembering, and sometimes the earlier parts have fallen out of view), and why the same question can get different answers (probabilistic generation means there's always some randomness in what comes out).

AI isn't magic but is software running on servers with specific technical constraints and design choices. Once you know what those constraints are, the weird behaviour makes a lot more sense, and once you understand the mechanics you can work with the tool more effectively instead of being baffled when it does something unexpected.

Browse all topics → Index