Seven observations from playing with AI

In which I try Claude Code

For about a month now I have been using Claude Code for both work and personal admin. I have 30 active days, according to the desktop version, with my longest streak at 20 days. I did not have much opportunity to see what the fuss was about when it first came, out my son had just been born and then I moved to India, so some of this might seem a little hackneyed. I’ve had a bit more time recently so I thought it was important to try this technology for myself and see how good it is. Here are seven observations I have taken from the experiment:

1. It is fun. Using it is what it used to be like playing with the computer, in the early do-it-yourself days of the internet. You want to do something or have an idea about how something would go and you keep tinkering until you crack it. Using Claude Code in the terminal feels like stepping out of the walled gardens of apps and back to a more scrappy version of information technology. I have written a few different python scripts, scraped data and tried and failed to make an app.

2. Most of the stuff I have automated is incredibly low value-added. I will give you an example. I keep a database of the sources I speak to. Often I am given business cards. Often a lot of business cards at once. They used to stack up on my desk for a rainy day but using Claude or ChatGPT I can take a photo of them, upload a photo and it will turn them into a table I can then just add to my list of sources. At the task level this is a big increase in productivity, what might have taken an afternoon now takes a minute or so, but I don’t think it adds much to my output. The business cards used to stack up because it was rarely a good use of my time to sort them out. Maybe there would be something serendipitous but often not.

3. The AI is sycophantic, forgetful and hallucinates constantly. It needs babysitting. In a way the sycophancy is the most dangerous because it is insidious and easy to fall for someone telling you how great you are. The others just require you to check and double check constantly. Memory has been my big problem with Claude. ChatGPT seems to be much better at remembering (though it is breathless in its sycophancy). Claude can also become obsessed with one point in particular and will just not let it go. It both fails to remember and fails to forget in the right ways.

4. The upfront time investment is massive both to learn how to use it and to give it all the context it needs. I now have a pretty great workflow with Claude. But it needed teaching about exactly what it is I want. I have definitely spent more time than I have saved. Perhaps I am doing it wrong, given what is happening to enterprise software stocks, but I have been far happier to pay for productivity software but only if it works well with Claude. I think of this software in two categories: one giving it a memory, so I am using Obsidian as machine readable note taking; the other useful category is giving it ways of seeing, so various subscriptions that output CSVs that it can then process in Python. This changes what I value within software. I don’t much care about well-designed UIs, instead I want text files that can be saved locally and manipulated with code.

5. My job is not productive enough to make a personal assistant worth it. Claude is better than nothing in helping me keep track of my diary, my progress on stories, doing my expenses and so on. But a person would be far better. A person armed with Claude would be even better. You can get much more out of the AI with subject specific knowledge and expertise in what it is doing. It also seems to change the value of different character traits in the workplace. It would be very dangerous for someone who is lazy and I wonder if it will give an advantage to the conscientious, the ones who make sure things are right and aren’t seduced by sycophancy, over the lazy but brilliant.

6. The AI cannot do my job. It cannot write, it cannot fact check, it does not really understand what is interesting or exciting about a particular subject. ChatGPT is a bit better at writing than Claude and I expect both are better than most people (who probably have not written anything since school) but the style is bad and flat. Even if it ever became good, recreating the same style at an industrial quantity would mean readers swiftly got tired of it.

7. Human bottlenecks limit how much productivity I can get out of Claude. My editor’s time is scarce, as are my sources. Driving across Mumbai still takes hours. Readers have a limited attention span. They do not actually want me to write tens of articles a week. The improvement in my output is likely an improvement in quality more than anything else, but even then I am not sure. It is making my job easier in lots of ways and I am spending my time on different parts of my job but I am not convinced it is making me more productive in a way that economic measurement would pick up.

The next thing I would like to do is try one of the open source models installed locally on my machine. I have a feeling the result of this experiment will be that my laptop is not really powerful enough to do much with it but for my use cases having more control over my own data, ensuring it is not trained on what I upload, is probably more valuable than using the latest frontier model.