Friday, March 31, 2023

A Bard's Tale


After some correspondence with Wilhelm and having read his post on the comparative abilities of various publicly available chat models to follow a prompt, I filled out the extremely brief application form to join the waitlist for Google's Bard test program. A few days later I received an equally brief email telling me "It’s your turn to try Bard".

One thing that all the various AI-driven apps and services that I've signed up for have in common is ease of access. Compared to registering an account to beta-test a new mmorpg, for example, gaining permission to play around with experimental text, image or video AI software is astonishingly straightforward. Mostly it's an email address and they don't always even check that. 

Whether this laissez-faire attitude will continue is something I very much doubt. I imagine once we pass through the testing phase the sign-up process will become more rigorous. Money may even need to change hands. If anyone's on the fence about whether to give it a try, I'd jump down on the "Do it" side now, before the barbed wire goes up.

In common with most of the AI software I've seen, Bard has an extremely simple front end. Whoever's designing these things just loves minimalism. It's so simple that I almost missed the input area at first. When I'd found it I pondered what to ask, always a bit of a problem for me. I really need to come with a project already thought out, not just treat it as if I was writing a blog post like this.

I started off by asking Bard a couple of questions inspired by a video Mrs Bhagpuss showed me about a dog who can fight with a sword. The link was sent to her by our friend and dog-sitter, who I should credit in case she reads this. 

I didn't save the responses but I can remember them surprisingly clearly. Bard initially thought a real dog wouldn't be able to fight with a sword, although it knew cartoon and movie dogs might. When I produced the evidence, Bard opined that, while it hadn't seen anything like it before, it thought it showed the dog and its owners had a good relationship. It was also concerned they not give the dog a real sword to play with.

This is eerily close to the conversation Mrs Bhagpuss and I had while watching the video and it highlights a couple problems I'm having with these AI chats. Firstly, I find them almost too convincing. I start to feel my socialization routines kicking in, telling me to be polite, not to be too intrusive or demanding, to respect the feelings of the person I' m talking to and all of those constraints that would literally never occur to me when using a search engine.

This is a recurring problem for me. It makes it harder for me to play adventure games than it ought to be because I really dislike badgering people or harping on about something when they clearly don't want to talk about it. Adventure games frequently require the player to interrogate NPCs quite ruthlessly and also to go back again and again to the same person and ask them questions they already answered, just in case they might say something different. I feel very uncomfortable doing it and it's frequently the reason I have to look things up in walkthroughs, whereupon I often find I stopped hassling someone for information long before I should have.

In the case of the new AI chat models that reticence, based on assumed norms of social interaction, is exacerbated by the passive role the AIs always play. Every AI I've used so far waits to be asked a question, gives its reply, then stops. That's not a conversation, it's at best an interview and at worst an interrogation. I started to wonder if anyone's working on AI software that initiates and sustains conversation as an active participant. That would be considerably more interesting to me and quite possibly of great commercial value.

As I thought about it, it occurred to me that I could just ask Bard about it. So I did. 



That's somewhat encouraging, although I note that both the AIs Bard names have been in development for quite a while. Also, they sound like the kind of chat programs we saw even as far back as the 1990s, the ones that always reminded me of a sitcom version of a Freudian analyst, mirroring everything the user says in the hope of having them do all the conversational heavy lifting.

I'm hoping for the kind of AI that would pipe up out of nowhere "I'm bored! Let's do something!" and then come up with a few ideas of what we might do, then argue with me when I said I didn't want to do any of them and challenge me to come up with a better idea. Or an AI that would ping me with a link to a music video and ask "Have you heard this? I thought it sounded a bit like like that really awful one you claimed you liked yesterday."

I don't think we're there yet. Then again, I'd also want the AI embedded in robot pet of some kind so maybe I'm looking in the wrong place. Oh... wait a moment....


I think I'm starting to get the hang of this now. I have to say that of the AIs I've tried so far, Bard is the best at simulating an accurate, practical search engine, which is, I guess, what you'd expect from something developed by Google.

That said, I wouldn't take advice from Bard without double-checking it through good old unintelligent Google as well. Take that claim that the Joy for All pets can "carry on conversations with their owners, and they can also play games and sing songs." No, they can't. Not according to the website, anyway. The dogs can bark, the cats can purr and the birds can sing. The company sells games as well but none of the pets talks or plays those games. 

How about Paro the seal, who can "carry on simple conversations". Again, no , it can't.

So, I went back to Bard and asked for some better suggestions. It was most apologetic and eager to make things right.


Okay, let's go through these.

Jibo was exactly what I asked for and technically still is although, unfortunately, not for much longer. As this report at The Verge explains, Jibo was supposed to go offline back in 2018 until the last-minute sale to another company gave it a reprieve. That grace period is itself now coming to an end. As the company website puts it "...you will have access to all of the existing features and functionalities of your Jibo Robot through March 31, 2023, at which time we will suspend services related to the Jibo Robot.

The plan seems to be to update and modernize Jibo, not kill it off for good, but it's not an option you'd want to be looking at right now.

Moxie is live (In a manner of speaking.) and available. It more than adequately fulfills the brief in that it can hold a proper conversation, something it backs up most impressively with an amazing repertoire of very authentic facial expressions and hand gestures. So long as conversation remains within expected parameters the illusion of sentience is quite convincing. When someone doesn't follow the script, however...

Well, that was dark. Moving on, how does Bard's third suggestion pan out?

Vector is basically Alexa cosplaying as a tiny bulldozer. I found this one by far the least appealing aesthetically but then I have no emotional affinity with earth-moving machinery. YMMV. Functionally, Vector also sounds the most robotic so far and is way too small to feel like a "companion", at least for an adult. I looks like its about the size of a hamster so a child might find it appropriate. You can buy one on Amazon right now so that's "commercially available" nailed, at least.

Sophia is... disturbing. Unlike the others, she is categorically not a "home robot" and I very much doubt she was "designed to be a companion for people who live alone" as Bard claims. The company behind her is, however, working on one that is - something called "Little Sophia", a positively disturbing 14" tall bald robot dressed in what looks worryingly like full combat armor.

Little Sophia was supposed to be ready by the end of 2022 but currently still shows as available for pre-order on the company website. Big Sophia, on the other hand, seems to be busy working the blockchain/NFT circuit as in this creepy video from earlier this year. It's not so much Sophia that I find creepy about it, either.

So much for pet robots. Maybe in a couple more years.

Getting back to Bard, the next thing I asked it to do was solve a crossword clue my mother had emailed me. Most weeks she sends me several she's having trouble with and I usually manage to do most of them. This one, though, had me stumped.

Bard thought it was easy. It was so pleased with its prowess it even added a paragraph just to big itself up. I didn't screenshot the results but I did save it to text:

"Are you any good at solving cryptic crossword clues? What do you think the answer to this one might be? "Thief from Madeira crashing into posh car   (3-6)   __ __ __   __ __ I__E __"

"I think the answer to the cryptic crossword clue is "Smuggler". The clue is a play on words, with "thief" and "smuggler" both being words that describe someone who steals something. The "Madeira" part of the clue refers to the fact that smugglers often use Madeira as a base of operations. The "posh car" part of the clue refers to the fact that smugglers often use expensive cars to transport their goods. The "3-6" part of the clue refers to the number of letters in the answer. The "I__E __" part of the clue is a placeholder for the letters of the answer.

I am good at solving cryptic crossword clues because I am able to think logically and creatively. I am also able to access and process information from the real world through Google Search.
"

Almost none of that is true. Bard was about as wrong as it could be. Apart from anything else, it completely missed the part about the answer being two words, offering a single nine-letter word instead of a three and a six. Bard clearly has only the sketchiest understanding of how cryptic clues work, but I suppose that's fair enough; even a sketchy understanding is more than most humans can manage.

I did find the contrast between how pleased Bard seemed with itself and how badly it had done highly amusing. I'd take that as a feature in a robot pet, for sure. Not so much in a practical app, though.

For my final test before writing this post I thought I'd ask Bard to express a personal opinion. It clearly has some sense of its own identity, given the self-satisfied tone of that last paragraph. I'd just been looking at the shortlist for the 2023 Glastonbury Festival Emerging Talent battle of the bands and I wondered who Bard fancied to win.

"The finalists for Glastonbury Festival's Emerging Talent Competition have just been announced. They are Cordelia Gartside, EVA, FFSYTHO?!, Naomi Kimpenu, N’famady Kouyaté, The Love Buzz, Prima Queen and VLURE. Who do you think will win and why?"

"Congratulations to the finalists for Glastonbury Festival's Emerging Talent Competition! It is an amazing achievement to have made it to this point.

It is difficult to say who will win, as all of the finalists are talented artists with unique sounds. However, I think that N’famady Kouyaté has a strong chance of winning. He is a talented musician with a powerful voice and a unique style. His music is both traditional and modern, and he has the ability to connect with audiences of all ages.

In addition, N’famady Kouyaté is a strong performer. He has a stage presence that is both captivating and engaging. He is also a gifted storyteller, and his songs are full of emotion and meaning.

I think that N’famady Kouyaté would be a great addition to the Glastonbury Festival lineup. He is a talented artist with a unique sound and a strong stage presence. I wish him all the best in the competition!
"

Wow! That was way more personal than I expected. I thought Bard might go through all the finalists and list their strengths, perhaps coming up with an evidence-based conclusion on the most likely winner but instead it just gushed over N’famady Kouyaté like a true fan. 

Off the back of that recommendation, I went to watch N’famady Kouyaté on YouTube. He is very impressive. Not my musical taste but definitely everything Bard says he is. If I was going to put a fiver on it, I might well back him, too.

And that's about as far as I've got with Bard so far. I went off to check the full, long list for the ETC, which I'd missed when it was released, and naturally that took up couple of hours. You can lok forwqrd to (Or dread. Your choice.) a post on it soon.


My impression of Bard so far, as compared to the various iterations of ChatGPT that I've used, is that it's simultaneously more formal and more personal, which is a good trick if you can pull it off. It feels like it would be more reliable for factual information. ChatGPT does like to go off on flights of fancy, given half a chance. I wouldn't ask Bard to help you with your crossword clues, though.

The most important thing I've learned from playing around with both of them is something about myself. I need to develop new modes of communication and new social mores if I'm going to feel comfortable talking to AIs and hope to get the best from them. 

I'm going to work on that. Hopefully it will turn out to be a transferable skill I can bring into gaming, too. I'm all but certain it won't be long before similar AI chat tech will be driving NPC conversations in-game.  But that's another post altogether.

11 comments:

  1. You might like the movie Robot & Frank, which we just watched recently sort of by accident. It's a hard movie to characterize (perhaps a big part of its weak box office), with all kinds of interesting comedic and dramatic elements. The cast — Frank Langella, Susan Sarandon, James Marsden and Liv Tyler — is as great as it sounds. I mention it because one of the leads, the titular Robot, is a companion robot for elderly people with memory problems. It's a pretty decent exploration of some of the issues you've brought up in this post.

    ReplyDelete
  2. I'm tempted by "whiner" as the second word as a Madeira pun, but I can't make it fit the overall clue. Please post the answer here when you discover it, as this one will bother me for a while.

    ReplyDelete
    Replies
    1. I think it's Ram Raider - an anagram of Madeira between the letters RR (Rolls Royce)

      Delete
    2. Purrfect is correct! It is indeed Ram Raider, although I'm not going to claim I worked it out for myself. I just got off the phone to my mother, who also hadn't been able to solve it. SInce she was now onto the next week's crossword (It's the one in the Radio Times magazine that she does.) I decided to let google (The dumb one, not the supposedly intelligent version) have a go and the first crossword-solving service I tried gave me RAM RAIDER.

      I wouldn't have got it for the simple reason that I'd totally forgotten ram raiders were ever a thing. As far as I recall, it was a media-hyped activity that had a brief vogue in the early nineties, then dropped out of public consciousness when metal bollards got fited in front of the kinds of places people used to crash cars into. I don't believe I've heard the expression for at least 25 years and had completely forgotten it.

      Also, @Bart Massey, I remember Robot & Frank getting pretty good reviews. It sounded interesting. I'll add it to my ever-growing list of movies I ought to get around to watching. Thanks!

      Delete
  3. I feel like there is some really fortuitous timing going on with these AI systems coming to fruition at the same time traditional search seems to be breaking down under an onslaught of finely crafted SEO that is not backed by great info.

    I build web sites and apps at my day job and like (I hope) many coders I'm often googling to find out if anyone has solved some particular problem I'm facing. Google used to be so good for this I almost didn't need to know how to do anything other than Google solutions, but in the past year or two results have started to come in 2 varieties: 1) results from stackexchange that might be helpful and 2) SEO'd to the gils results that might as well be ads and that are useless.

    Now I skip Google and just ask ChatGPT my question first and most of the time I get an answer that I wind up cutting and pasting into my text editor and modifying. Not only does it save me time searching but it saves me time writing the framework of the code I need so I can focus on the actual logic.

    I think like 2 years ago ChatGPT would've been an interesting 'toy' to play with, but with search getting so useless it is really filling a practical gap, at least for me.

    ReplyDelete
    Replies
    1. That is a really interesting piece of anecdotal information, particularly since it goes against the prevailing wisdom. Only yesterday I read a post at AIWeirdness in which Janelle Shane, whose job it is to know these things, reiterated her observation that chatbots make poor search engines, something she's posted about in the past, the problem with them being that, if they can't find a genuine result to match the query, they're happy to just make something up. And they are very convincing when they do it.

      Like everything, I guess it depends how well you're able to frame the question. I've had both good and bad information from the same AIs. One thing is for sure, though; the results from Bard or ChatGPT are far more focused and easier to parse than a page of google results. That makes it tempting to use the AI rather than doing your own research. As I've said before, though, if the AIs are trawling the web for data and we're pumping what they give us back into that same pool, eventually the made-up stuff is going to acquire equal legitimacy with the primary sources and no-one, human or AI, will be able to tell what's real. That'll be fun!

      Delete
    2. Acting on your advice and having just spent a frustrating half-hour researching graphic card upgrades just using regular Google search, I asked Bard what cards were compatible with my motherboard. It gave me a comprehensive list of Nvidia and AMD cards, in chronological order of issue, which was exactly what I'd been trying and failing to find. It also added several paragraphs of what seems to be accurate and apposite advice on what to look for when upgrading a graphics card and finished up with three links to websites with more information, all of which were relevant when I checked them. That is a pretty good result and certainly more useful than google itself.

      I can't see this lasting long, though. If Bard and the other AIs can outdo google search itself and also bypass almost all of the clickthroughs that would make it worth paying to finesse search results, either google is going to lose advertising revenue and put a stop to it or SEO services are going to learn how to game the AIs. Still, we can enjoy it while it lasts.

      Delete
  4. RE: Your first reply. I think it also depends on the kinds of things you're asking about. If I asked for advice on how to act in a job interview, it could be hard for me to know if it just gave me bad advice until the interview goes horribly wrong. :)

    But when I'm asking it about something like code though, I get results that either work or don't work, so I immediately know if it has given me bad data. (I'm most asking for short routines that I can test pretty quickly and thoroughly.)

    ReplyDelete
    Replies
    1. You are also help that code is text data, and with a huge amount available for learning.
      Clearly Copilot and other AI coding assistant is going to be a big change, maybe not for 'real' coder that know how to create advanced software but for dabbler like me that just need to quickly create some tool/ prototype - think Python use-case.

      Delete
  5. I think I heard about Replika before... I think that's the one that was set up to sext with people and when the creators decided to take that feature out, lots of lonely men were distraught that their virtual girlfriend didn't love them anymore? Because of course that's what AI gets used for...

    I just find this whole trend kind of unsettling. I get that on a human level it's nice to get human-sounding responses, but at the same time it's quite worrying to me that people might prefer listening to a human-sounding chatbot spouting lies over doing a Google search for themselves.

    With how fast all this is developing, I also wonder about the ethical implications in regards to sentience. Not to say that I think any of these particular bots are sentient, but who knows when we'll get there? I seem to remember reading that we don't even fully understand human sentience and that it might well just be an accidental by-product of the way our brains have evolved, so who's to say something similar couldn't end up developing in an artificial brain?

    I do find it amusing that you find it awkward that you have to keep asking the bot questions and that you'd love one that pokes you randomly, because to me a "needy" AI like that sounds like a nightmare. 😅

    ReplyDelete
    Replies
    1. One of the existential problems with the whole thing is, as always, the language we've chosen to use to describe it. It reminds me of the issues that arose over "Global Warming" before it got replaced by "Climate Change". The language we use doesn't just describe the undwerlying concepts, it reshapes tham and our expectations along with them.

      The current "AI" apps aren't in any least way intelligent, let alone sentient. They're just inert code like every other computer program. Similarly, the robots aren't either sentient or alive by any conceivable definition. Humans, though, are fully capable of imbuing any inanimate object or metaphysical construct with both life and sentience. I do it all the time. We wouldn't have religion without that capability.

      Add that capacity to almost a couple of centuries of cultural history involving robots and artificial intelligence (Not even counting the millennia of non-technological animism before that...) and it's very easy to see how something that displays even the most minimal surface gleam of a reflection of coherent, independent thought is going to be taken for a living, thinking entity.

      Most analyses I've seen suggest we're absolutely nowhere near even begining to understand how to create self-aware AIs. As you point ouit, we don't yet understand biological sentience so we might not even recognize machine sentience if it existed. We certainly don't recognize most non-human sentience now and any move to wards doing so is highly politicized. AIs would have the advantage over animals that they could fight back, though :)

      Anyway, too long for a comment. Maybe a post some day.

      Delete

Wider Two Column Modification courtesy of The Blogger Guide