A rant about AI writing co-pilots

I had an idea for a new product the other day in the shower, right between the end of shampooing and start of deciding whether today would be a conditioner or a no-conditioner day. “Oh my god, that’s revolutionary”, I thought, and rewarded my hair with conditioner as appreciation for sitting atop the head that had this incredible idea. I soaked in the idea for the next few minutes in the hot shower and jumped out, ready to pitch my vision to the team. At the meeting, I jumped right in. “Ok guys, check this out” and I got started, first with some context, because of course you’ve got to set context and then a series of “revolutionary” thoughts one after another.

At the 4 minute mark, Came the first “But Neil, what about….” And I said, “Sure we have to iron out some of the details, but the main idea is…”. Then came the next, “but what about…”, and the next, and the more I tried to explain what was in my head in words, the more I realized that the revolutionary thoughts in my head, were just that - thoughts. The act of trying to articulate my thoughts into coherent sentences had exposed how nebulous and poorly thought out they were.

Another thing happened as I tried to put the idea into words. It started to feel less exciting up close than it did at a distance. The act of explaining had uncovered freckles and warts in its skin, and ultimately a feeling of disillusionment took over. Perhaps it wasn’t a good idea after all.

Explaining something to someone tends to be a very strict test of the thoroughness of your thoughts. Even if you’re talking to a wall, or worse, an uninterested audience, just the mere act of thinking about your ideas from the perspective of a listener, makes you a sales person, trying to convince someone of something you believe in. You have to give context, explain the current state of things, prove that your idea is different enough, so they get as excited about it as you are. Doing this, as it turns out, is a very useful tool to help you turn nebulous thoughts into actually useful ones.

There’s another, even stricter test - writing.

When you write about something, you’re effectively doing the same thing, but with a new handicap. You don’t have body language as a crutch to lean on. Your words must stand on their own two feet, an entity outside yourself, doing the convincing for you. Writing is also slower, it lets you stop, reflect, and revise, a luxury you don’t have in conversations, at least if you don’t want to look like a psychopath.

There’s this notion that writing is a way to just communicate your thoughts. You have this fully formed idea in your head and then you just write it all out for the world to understand. Except that couldn’t be further from the truth. Anyone who has ever written about anything in depth, knows that writing about something, actually changes how you think about it. that process of articulating thoughts, reflecting and revising clarifies your thinking, helps you discover holes, resolve the salvageable ideas and discard the terrible ones. In doing so, writing changes your thoughts for the better.

Most of this will probably sound fairly obvious. It did to me as well, until I read Paul Graham’s essay “Putting ides into words”. In a somewhat reverse effect, reading a piece of well thought writing makes you rethink something seemingly obvious, in a new way, and that tickles some new corner of your brain, enough to make you go, “Hmm, I never thought of it that way”. In this particular case, Paul Graham argued that the top experts in any field, also tend to be the top writers in the field, and this wasn’t a coincidence, because writing unequivocally makes you a better thinker.

So writing is great…what am I getting at? First off, this was the impetus I needed to start blogging again, but more importantly, in the context of the current LLM revolution, that had me wondering, if writing is such an important tool for human thought, how does AI generated writing fit into this equation?

The internet is ablaze with Generative AI optimism right now, and for good reason. Every major publication is talking about how AI is going to replace some or the other profession. Including writers. There’s been a goldrush of products helping you turn single sentence prompts into fully formed blog posts, stories, emails, and product descriptions. Hell, the Writers Guild of America even went on strike, afraid of losing their job to an AI and never getting paid what they’re worth.

But while it’s conceivable for AI generated writing to replace menial writing like product descriptions, training manuals, and other forms of relatively innocuous corporate drivel, it’s suffice to say that writing as tool for independent thought is not going away.

This is what bothers me about the current crop of LLM based writing products. The products seem to be designed with the intention of generated written works faster, with minimal human effort, to the point of replacing the human entirely. For example, writing a paragraph (or entire essay) from a single prompt, turning bullet points into prose or vice versa, summarizing text, or changing the tone of some pre-existing text. They seem to all be designed with the intention of expediting the process of communicating what is already in your head so the world can better understand you, and not so that you can better understand yourself. In effect it’s like having someone else write that they think you’re thinking about and then reading it back to you, reaffirming your thoughts. You’re, at best, a passive observer of your own thoughts.

I’d like to see writing products designed as tools for thought, that help you get into the flow of exploring and reflecting on your own thoughts rather than breaking you out of it. Typically, when you start writing, you spend the first hour or more, staring at a blank page or writing a few nonsensical sentences that you inevitably discard but it’s writing a few shitty first drafts that helps you figure out what you’re not writing, which in turn helps you narrow in on what you are. When that clicks, it feels like you’re in flow and words start coming to you easier.

Conversely, consider the scenario where you type in a prompt describing what you want to write and an entire sequence of paragraphs is magically created for you. There are certainly situations where I welcome this, but if I were writing to think, not laboring through articulating myself, actually makes me a worse writer and a worse thinker.

This is really part of a broader question about human machine interaction. When machines can do part of the thinking, what is the level of abstraction at which humans should interact with machines? In this case, the question is, how do you design AI-driven writing tools that help you accelerate your writing, without taking away from the benefits of putting your ideas into words.

To think through how we might design a tool like this, let’s define some of the bottlenecks in a typical writing exercise.

Background Research: finding reference material to back up your points. Blank page syndrome: feeling overwhelmed, not knowing where to start, or brooding about the perfect introduction. Scaffolding: Creating a high level structure for your writing. Brainstorming: Testing alternative ways to write the same thing. Proof reading Each of these bottlenecks creates speed bumps between your ideas and your words and slows down the feedback loop that helps you discover your voice and clarify your thoughts. So where can AI help? I see AI as a speed highway between your thoughts and your words. The basic premise is that an AI language model has read a lot more than you, and therefore knows more than you, and can coach you, but can’t get inside your head. Like a good coach, it should encourage you and push you to do your best work, and then get out of your way.

To design a tool that accomplishes these goals then, we might start by trying to tackle some of those bottlenecks.

For example, one possible design might be a tool that prescribes ways to tackle common writing problems. Writing a blog article? Enter a prompt and out comes a short introduction, plus a scaffolding that helps you get over writers block and start writing. As you write, you get suggestions for 2-3 directions you could take, based on where it thinks you’re going, sort of like a higher level “autocomplete” that completes sentences at a conceptual level rather than a grammatical level.

The actual design is perhaps the topic of another post, but I think there is something to be said about this new approach to UX design, where rather than writing specific instructions to control a machine, you work together in tandem, like a jockey working with a race horse. We’re currently trying to replace the jockey all together by building a horse than can waltz in and win a race on its own, but there is an opportunity here to make the jockey a super hero, by redefining what it means to hold a rein.