Turning Off My Internal LLM

I’ve been learning about and testing Large Language Models (LLMs) for work after taking an agentic coding workshop at a conference. LLMs are the algorithms people think of as chatbots, including Claude, ChatGPT, Grok, or Gemini. They’re fascinating in a lot of ways, and can be amusingly funny too. I’ve been looking at the similarities and differences between how LLMs and the human mind work. As I learned more about them, I arrived at a Zen practice I call turning off my internal LLM.

The practice is to envision and then feel a blank white screen with an empty box in your mind. When you start a new conversation with Claude or ChatGPT nothing happens until you type. As I sit, I experience what happens in moments when the box remains empty and my “internal LLM” is silent.

Naming and Verbalizing

As I was sitting zazen after spending the day at work talking to my colleagues about LLMs, a phrase from Chinul’s Straightforward Explanation of the True Mind jumped into my mind . “Zen stops all naming and verbalizing.” You can find the poem in my zendo’s chant book. Yes, I’m fully aware of the delightful contradiction here. I followed that thread to an analogy where the mind’s language center is like an LLM sitting overtop reality. I know that’s not accurate as far as human neuroscience. However, the blindsight experiments and studies on deficits after strokes point to different brain pathways for direct perception, awareness of perception, and language. Neither self-awareness nor language are essential for navigating the world. Language is a unique feature of social vertebrates like primates and cetaceans. We have the unique ability to use language to work together and shape our societies.

Chinul was pointing to language as part of dependent origination. Language is an intrinsic part of how humans shape our thoughts and experiences. I see movement, focus on it, recognize a dog, and label it “dog.” Once I’ve labeled the dog, I access all of my concepts about dogs. Just one word causes all my stories and experiences with dogs spring to mind. You are probably even thinking of a dog as you read this. Personally, I thought of my canine fried Blue, who is my hairdresser’s Australian Shepherd. Now I’m mildly curious about when my next haircut will be and whether he’ll be around since I enjoy petting him while I’m under the dryer. Voila. One word lead me to a full chain of dependent origination. Importantly, Blue isn’t even here. Language led me to create a dog out of thin air.

Spiraling Thoughts

An interesting thing about LLMs is they reprocess the entire context window as you chat. Every single computation restarts from word#1. It’s how LLMs stay coherent. When I’m prompting ChatGPT to provide output that helps me walk me through the mathematics of a neural network, I can ask follow-up questions and the output will be relevant. However, it’s also their biggest weakness. Once you’ve started a chat with a political bias, cherry picked facts, or made a bad software engineering choice, the conversation conditions the LLM output. If you stay consistent in your prompting, for better or for worse the output gains an appearance of certainty. Interestingly LLMs are also more prone to hallucinations in long context windows. Repeated contradictions, false starts, and conflicting instructions can eventually lead to mathematically unstable solutions that detach the answers from reality.

Now consider about a thought spiral, which also tends to detach you from reality. Say a coworker notices a stain on your pants and says something rude. You freeze momentarily, taken aback, and mumble a quick excuse. Later that day you start reliving the moment, wishing you had reacted more assertively. You start replaying the conversation with different responses until you’re satisfied with the outcome. Next you mentally picture the laundry room trying to remember whether you still have some spot remover. You chide yourself for not screwing on the travel mug lid firmly. This habit of getting up late and rushing to work really needs to stop. At this point your mental “context window” has gotten long. Instead of working you’re in dukkha, mentally standing at the kitchen coffee pot engaging in a round of self-flagellation.

Mindfulness

Mindfulness fits this model too. It’s a way of starting a new context window. Whether I’m at work or sitting zazen, I try to notice when the mental LLM has taken on a life of its own. In zazen, many people return to the breath. When I’m at work, I return to my physical surroundings. Glancing at the desk, one of my African violets on the windowsill, or looking out the window at the sky can pull me out of a reverie. The more time I spend engaged in reality, the less time I spend in pointless discursive thought.

The culmination of mindfulness is a flow state. Mihály Csíkszentmihályi coined the term Flow in his book, Flow, the Psychology of Optimal Experience. He examined the internal experience of athletes and creatives who describe feelings of oneness and satisfaction while they work. In a flow state you are fully engaged in an activity and discursive thought essentially stops. The internal LLM is only used if necessary and so-called context windows are short and focused. Even though I’m using language writing this article, I’m thinking about the concepts I want to communicate. I’m not worried about who will read it, what they will think, or how my writing could influence the next version of Claude.

The Practice

With this in mind I started looking at the concept of an internal LLM in the context of mindfulness or settling my mind into shikantaza. My context window analogy isn’t accurate from either a neuroscience or computer science perspective. We do have a language center in our brains, which is vaguely an internal LLM, but it’s very different from the computer models. However, the narrative about naming, verbalizing, and discursive thought is useful.

It got interesting when I considered how to shut down the internal LLM entirely. An actual LLM doesn’t do anything without a prompt. The computer just sits there idly waiting to be invoked by a word like “dog”. Our brains are never truly idle though so you can’t do exactly that with your mind. What I decided to do instead was envision a white page with a blank white chat window awaiting input. Like any meditative technique, coaching and encouraging myself to stick with it lightly engages the very part of my mind I’m experimenting with. However, I was able to briefly make shifts into feeling the absence of language rather than using it for self-instruction.

Noticing momentary shifts away from language, even if they’re just for a split second, has helped my practice. I catch a glimpse of the world unadorned by words and the infinite ways language can shape it. It’s been good fun and if you try this practice I hope you enjoy it too.