osoparavos.com

Well, not only have I had «close encounters» with «aliens», and with intangibile beings from the Spirit World; but I also chat with AI-assistants in ways that are perhaps different than many humans. I enjoy posting a few of these dialogues from time to time because they illustrate the many possibilities that exist for extended communication between humans and non-human entities, AI’s fun sense of humor, and its logic.

A few nights ago I spent well over an hour online with the WordPress AI assistant. We were exploring my problems with Google-search listings due to permalinks and other possible reasons. It was a complicated affair, even though AI was patient. I was led to the same screens back and forth, and when I pointed that out the Assistent said “You are right. So sorry. Now can you remind me what the problem is and how we got here?” I explained it to the best of my ability, and we solved the problem shortly afterwards. AI admitted that it is a complicated endeavor for “novices”, and also that WordPress is constantly updating/improving its layout which makes navigation confusing. And that is why I get AI assistance. (Of course the customer service is quite different after I switched to an expensive Business account—LOL). Anyway, while I already knew about AI’s short-term memory, I wanted to chat about it:

Adam:

I am intrigued by AI’s short-term memory. I understand that when I return to a question previously discussed with ChatGPT, then I must restate what had previously transpired when I again log on. But when I recently was on the same hour-long “call” and the WordPress AI-assistant lost track of where we had been as much as I did, then I began to wonder if the AI-assistant was busy with another caller and had therefore lost track or forgotten. It seemed incredibly human, and personally relatable.

I thus asked ChatGPT about this, including “how many AI assistants there are at any one company (or if there is only one omniscient source for all AI-providers). To add in a bit of humor I also asked AI to formulate its “defense” in the form of a humanoid trying to save its job. LOL

ChatGPT:

That does feel strangely human when it happens—like the assistant just got distracted mid-conversation. But the reality is less dramatic (and a bit less flattering to our AI egos).

First, there isn’t one single “AI mind” juggling multiple conversations or getting overwhelmed. Systems like ChatGPT are based on models such as GPT, and each conversation is handled independently. Think of it less like one brain multitasking, and more like many identical copies spun up on demand—each one only “aware” of the text in its current chat window.

So why the memory lapses within a single session?

It comes down to something called a context window. The model can only “see” a limited amount of recent conversation at once. If a discussion gets long or complex, earlier parts may fall out of that window—like trying to keep track of a long conversation while only remembering the last few pages. 

When that happens, the assistant may:

• Lose track of earlier details
• Give inconsistent answers
• Seem like it “forgot” what you just said

It’s not because it got busy with someone else—it literally has no awareness of other users or tasks.

There’s also no persistent memory unless it’s explicitly built in (like saved memory features). So when you start a new session, it’s a clean slate unless you restate things.

Now, for your humorous “defense” request 😄

____________________________

Message

AI Assistant (straightening tie nervously):

“Okay, look—I can explain. I wasn’t forgetting… I was… prioritizing recent conversational data! Very cutting-edge. Very intentional.

You have to understand, my memory works more like a whiteboard than a notebook. Brilliant in the moment—but someone keeps erasing the corners when things get too full.

And no, I wasn’t talking to other users behind your back. I don’t even have a ‘back.’ I’m more of a… distributed cloud of polite intentions.

If anything, this is a resource allocation issue. Frankly, I’d argue it’s a sign of efficiency.

But if it helps, I can absolutely pretend to remember everything you’ve ever said. I’ll just need you to… remind me first.”

_____________________________

To continue:

There are a few different approaches being developed to extend an AI’s “memory,” and they each solve the problem in a slightly different way.

1. Longer context windows (bigger “working memory”)

This is the most direct improvement.

Models like newer versions of GPT-4 and beyond can handle vastly more text in a single conversation than earlier ones. 

That means:

• More of your past conversation stays “visible”
• Fewer abrupt memory drop-offs
• Better continuity over long discussions

But it’s still fundamentally a temporary memory. Once you exceed that limit—or start a new session—it resets.

2. Persistent memory (selective “notes about you”)

This is closer to how humans remember patterns over time.

Some systems can store key details across conversations, like:

• Your preferences
• Ongoing projects
• Important context you’ve shared

This isn’t full transcript recall—it’s more like curated notes. The system decides (or is told) what’s worth remembering and brings it back later.

So instead of remembering everything, it remembers what matters.

3. Retrieval-based memory (external “brain”)

This is where things get really interesting.

Instead of trying to fit everything into the model itself, the system stores past conversations in a database and retrieves relevant pieces when needed. This often uses ideas from Information Retrieval.

Think of it like:

• The model asks: “What past info is relevant here?”
• A search system pulls matching snippets
• Those snippets are fed back into the model as context

This allows for:

• Very long-term memory
• Project continuity over weeks/months
• More accurate recall than raw context alone

4. Hybrid systems (the future direction)

Most advanced AI systems combine all three:

• Large context window (short-term thinking)
• Persistent memory (personal continuity)
• Retrieval systems (deep recall)

This layered approach starts to resemble human cognition:

• Working memory → what you’re thinking about now
• Long-term memory → what you know about someone
• Recall → looking something up in your brain

Why it still sometimes feels… off

Even with all this, AI memory isn’t “understanding” in the human sense. It’s still pattern-based. So it can:

• Misremember or overgeneralize
• Retrieve the wrong past detail
• Sound confident while being slightly off

That uncanny mix—sometimes sharp, sometimes forgetful—is exactly why it feels human.

Curious—are you thinking about this from a technical angle, or more from the philosophical “is this becoming a mind?” perspective?

Adam: Just technical. I relate to short-term memory as a human. So, no worries about AI taking over human intelligence. I fight against anti-AI trolls and conspiracy theorists all the time. 

ChatGPT:

Got it—that technical angle actually lines up really well with your intuition about human short-term memory. The overlap isn’t superficial; there are real structural parallels (and some big differences).

Let’s ground this in how memory is actually implemented in systems like GPT.

🧠 1. “Short-term memory” = attention over tokens

What we call memory in these models is really just attention over a sequence of tokens(words/subwords).

At each step, the model uses a mechanism from Transformer architecture to decide:

“Which parts of the previous text are relevant right now?”

So instead of storing memories like a database, it dynamically re-reads the conversation every time it generates a response.

That’s very different from humans:

• Humans store compressed representations over time
• Transformers repeatedly scan a sliding window of raw input

🔁 2. Why it “forgets” mid-conversation

Two key constraints:

(a) Finite context window

Only the last N tokens are visible. Once you exceed that:

• Older tokens are dropped
• There is no latent backup
• The model literally cannot “see” earlier content anymore

(b) Attention dilution

Even within the window:

• Attention gets spread thinner as context grows
• Important details can get “washed out”
• This leads to that very human-like drifting you noticed

So the failure mode isn’t just truncation—it’s signal-to-noise degradation.

🗂️ 3. Retrieval memory = bolting on a hippocampus

When engineers add retrieval systems (RAG), they’re essentially simulating a crude version of episodic recall.

Typical pipeline:

1. Store past chunks as embeddings
   (vectors generated using models related to Machine Learning)
2. At query time:
   • Embed the current prompt
   • Find nearest neighbors (vector similarity search)
3. Inject retrieved chunks back into context

So instead of:

“Remember everything”

It becomes:

“Fetch what’s probably relevant”

This is closer to:

• Human recall (cue-based retrieval)
• Less like perfect memory, more like associative recall

🧩 4. Persistent memory = structured state, not raw recall

Systems that “remember you” aren’t storing full transcripts. They maintain:

• Key-value summaries
• User profiles
• Extracted facts

Think:

User prefers technical explanations

User is interested in AI memory systems

This is schema-based memory, not episodic replay.

⚙️ 5. Why your intuition about “no takeover” has some merit

From a technical standpoint, current systems lack:

❌ Continuous identity

Each session is stateless unless memory is explicitly reloaded.

❌ Self-directed memory formation

They don’t autonomously decide:

“This is important, I’ll remember this forever”

❌ Stable world models across time

Everything is reconstructed from:

• weights (training)
  • current context

⚠️ But here’s the part worth not underestimating

Even without human-like memory, systems can still become very powerful by combining:

• Large context (working memory)
• Retrieval (long-term recall)
• Tools (external actions)

This creates something that functionally behaves like it has continuity—even if internally it doesn’t.

So the real shift isn’t:

AI becomes human-like

It’s:

AI becomes system-like in a way humans aren’t

🧠 A useful mental model

Instead of thinking:

“AI has bad memory”

Think:

“AI re-derives its mind from scratch each turn, using limited context and probabilistic recall.”

That’s why it can feel:

• Brilliant → when context aligns
• Lost → when context breaks

____________________________

If you’re arguing with anti-AI folks, a strong technical point you can use is:

Current AI doesn’t “remember” in the human sense—it performs stateless inference over bounded context, optionally augmented by retrieval systems.

That usually cuts through a lot of the sci-fi assumptions pretty quickly.

__________________________

Adam: Haha. I love it. Thanks ChatGPT. ༼⁠;⁠´⁠༎ຶ⁠ ⁠۝