This is not really a blog post proper, more of an update to an older set of posts made a year ago. Since that time, newer, bigger, and better models have come out. I gave a little talk about what it’s like to work with the AI and where my frustrations with it lie, particularly with regards to creative work, mentioning in specific that coherence seemed to be a real problem.
One of the suggestions from the crowd was to use Claude 3 Opus, since most of the work I’d been doing had been with an earlier version of Claude and my experiments with Claude 3 Sonnet hadn’t shown much difference.
Since I am a man of science, I tried generating the first chapter again. I have a few caveats here: first, this was done in chatbot mode, since I’m on my netbook for this convention and won’t be home to my usual setup for another two weeks. It’s always better to use the LLMs in their native mode, which means using an API. Second, this was not very carefully prompted, I’m using the old prompts that I had on hand and hoping for the best. Third, this is the first generation (though for a full novel I cannot imagine that you would want to generate a bunch of scenes and manually cherry-pick the best).
With that out of the way, the chapter is … fine? Serviceable? There’s some definite clunk, and perhaps some of this is due to the prompting, but it is better: possibly good enough that someone would actually read it if it were freely available on the internet. The moss wolf stuff is awkward, but might have been less awkward with a different prompt that attempted to foreshadow, or a helper that asked “did you foreshadow?” There’s a lot of telling instead of showing, in spite of my requests for it to do otherwise.
Do I think that the LLMs are at point of being able to write a serviceable novel yet? Something that’s readable and maybe would be good enough for one in a thousand people to recommend it? Something that people would read?
Maybe?
I’ll investigate more when I get home. My hunch is that there’s too much that’s weird and strange, which still needs to be sanded away, and too many things that are simply unexplained in spite of the fact that the outline wants them explained. It says “During the time of day her stepmother is gone” in the prompt and then “With her stepmother gone for the day, Adelia seized her chance.” in the response, and this is awkward because surely we need to explain where the stepmother has gone, don’t we? Where is this stepmother going? I don’t need much, just “on Thursdays, her stepmother went to a bridge club” or something.
I think reading it twice, there are lots of off notes that I’m picking up on, but I don’t know if they’re “off” enough to deter readers. I’ll plan to write a part four to this, which will be a fuller examination of Claude 3 Opus. I’m particularly interested if it’s any better at writing the scene descriptions from chapter descriptions itself, which is a bigger test of the approach. I’ll also try to build up some of the “helpers” to do various passes for prose quality, which never got completed the first time I was on this project.
If you have found a spelling error, please, notify us by selecting that text and pressing Ctrl+Enter.