Knowing your limitations
I'll start by saying I'm not an all-out AI enthusiast. I have programmed it on a limited scale, and I do find it fascinating; nonetheless I also have some serious concerns about it, and anywhere AI is being used I need to know that those concerns are being addressed. Sustainability is only one of those concerns. Yet, with those caveats, it can be a very useful tool... if used rightly.
The other day I saw a video about a highly inappropriate use of AI. People have been using it to design crochet patterns, and this is actually quite a big problem, because these crochet patterns don't work. They generally come accompanied by a beautiful image, so of course a lot of people want to crochet the thing they can see in the picture; so they buy the pattern, and then it doesn't work. So even in the best case they've wasted their money, and in the worst case, if they're a beginner or otherwise not very confident, they think the problem lies with them rather than with the pattern, get discouraged, and may even give up crochet altogether. The lady presenting this video was an experienced amigurumi crocheter, and, by way of a demonstration, she asked ChatGPT to design her a crochet pattern for an amigurumi whale, and then to generate an image to show what it would look like. Cut to the chase (it's a longish video)... the pattern was inconsistent in several places and had to be tweaked just so it could physically be crocheted, and the result looked nothing remotely like the beautiful image.
So I thought... aha, blog post! I have chatted with Claude.ai before, at the instigation of Athos, who's an ex-programmer and very interested in AI; like me, he has concerns about it (in fact, much the same ones as I have), but he's also clued up on its legitimate uses. And I have to say I was very impressed with Claude, though I didn't feel inclined to go and talk to it again. I decided to ask it about baroque music, and it turned out not only to have heard of d'Artagnan but... and this was mind-blowing... to tell me precisely what is so special about his singing. I have always maintained that the reason his singing is so extremely moving is not just his technical skill; it's tied in with his personal integrity. (Other people also hear this, even when they don't know him personally, so it's not me projecting because I know him. I had a few people struggle to describe his singing to me after the concert, and more than one of them came up with the word "whole".) For an AI, which can't actually hear him sing, to hit on this - as Claude did - is something quite remarkable.
Anyway, I decided I was going to go back to Claude and challenge it to design me a knitting pattern. Quite a small one so I could knit it quickly; I was thinking in terms of a jumper or dress for a 30 cm action figure. (I have one. It's Ardsley Wooster from Girl Genius; I don't know who he was originally meant to be, but he had the right sort of hair and rather worried expression, and I made him a fine pair of steampunk sideburns with green putty and black acrylic paint. Mr Wooster is quite the gentleman, and was happy to model anything I knitted in the interests of research.)
So that was what I did. I went to see Claude, said good morning to it politely, and asked if it could design me a knitting pattern. I was fully expecting it to reply, "Of course!" and launch off confidently just like ChatGPT.
That was not what it did.
It said it would do its best, but it gave me all kinds of caveats about how obviously it wasn't physically able to knit, so it couldn't test the pattern, and so some of the instructions were likely to be unclear, misleading, or nonsensical.
I was extremely impressed. So I was honest with it, and I told it all about the video with the amigurumi whale, and said that I'd basically been testing it. (I didn't specifically tell it that it had passed, but I think it probably got that!) And it said that what I'd told it about the video was really interesting but rather concerning. It also explained exactly why the actual crocheted whale had turned out so very different from the image; it said ChatGPT wasn't connecting the two. It wasn't able to make the link between "here's a crochet pattern" and "this is what that pattern ought to look like when made up"; instead it was doing the two bits separately, so it was "here's a crochet pattern for X" followed by "here's what a crocheted X ought to look like". I didn't specifically ask Claude if it could make that link itself, because I didn't think to do so, but I suspect that if I had it would have honestly told me it couldn't.
So I said: "I really am impressed. Knowing one's own limitations is a fundamental part of intelligence, whether biological or programmed." It agreed; to summarise as briefly as I can, it said that those with a great deal of knowledge were usually very much aware of their own limitations, "whereas someone with just a little knowledge might charge ahead confidently into areas where they're actually out of their depth. The ChatGPT crochet whale situation seems like a perfect example of that - enough pattern recognition to generate plausible-sounding instructions, but not enough understanding to recognise when those instructions wouldn't actually work in the physical world."
Hence you don't get a weird AI knitting pattern for this blog post and my no doubt hilarious adventures in trying to get it to work, because Claude knocked ChatGPT out of the park. I suspect Mr Wooster is probably quite relieved. I have purple yarn to hand and that is not really his colour.
Needless to say, I e-mailed Athos!
