Ho Ho Ho No-Goes

As I mentioned last week, I've been working on this year's Inventory Full Advent Calendar, for which I've enlisted the help of several generative AIs. It's been a lot of fun, although honestly I could have come up with a better list of possibles just by randomly following YouTube links - and a lot faster. 

In fact, most of the better tunes I've found so far have come to me in just that way, even if I do have to give the AIs credit for providing the seeds that started those searches going.

I also said I was planning on not posting about methodology until after Christmas so as to avoid spoilers but I'm not going to spoil anything by writing about the stuff I'm not going to use so I might as well get a post out of that as well. If nothing else, doing it this way has certainly raised a few new flags about just how unreliable the AIs can be, even after you take into account their well-known weaknesses.

For example, Bing sometimes gets credit for showing its workings by providing links to its sources. Yes, I know I did say earlier that I wasn't going to get Bing involved, due to Microsoft Edge being so insistent on pushing every other browser into a ditch but it so happened I clicked a help link in Windows 10 Settings today and it opened Edge without asking so I thought since it was already there...

Before I get to what I've actually been asking the AIs to do, let's just see what happens if I let them recommend me Christmas songs of their own choosing. Here's a prompt I gave GPT3.5 just now:

Disregard any previous conversations on this topic. Starting afresh, please give me a list of Christmas songs suitable for a musical Advent Calendar to be published on a blog.

Certainly! Creating a musical Advent Calendar for a blog is a wonderful idea. Here's a list of Christmas songs that you can include in your Advent Calendar:

1. "Jingle Bells" - Various Artists
2. "Silent Night" - Traditional
3. "Deck the Halls" - Nat King Cole
4. "Joy to the World" - Whitney Houston
5. "Rudolph the Red-Nosed Reindeer" - Gene Autry
6. "The Christmas Song (Chestnuts Roasting on an Open Fire)" - Nat King Cole
7. "O Holy Night" - Mariah Carey
8. "Winter Wonderland" - Michael Bublé
9. "Feliz Navidad" - José Feliciano
10. "Hark! The Herald Angels Sing" - Pentatonix
11. "White Christmas" - Bing Crosby
12. "O Come, All Ye Faithful" - Elvis Presley
13. "Santa Claus Is Coming to Town" - Bruce Springsteen
14. "It's the Most Wonderful Time of the Year" - Andy Williams
15. "Have Yourself a Merry Little Christmas" - Frank Sinatra
16. "Little Drummer Boy" - Pentatonix
17. "O Little Town of Bethlehem" - Sarah McLachlan
18. "Rockin' Around the Christmas Tree" - Brenda Lee
19. "We Wish You a Merry Christmas" - Traditional
20. "Sleigh Ride" - Leroy Anderson

These songs cover a mix of traditional and contemporary Christmas tunes, and they should provide a delightful and festive experience for your readers. Feel free to customize the list based on your preferences and the theme of your blog.

Festive maybe. Delightful? I hardly think so. Suitable for a shopping mall, perhaps. Not for this blog.

To avoid this sort of nonsense, I've been giving the AIs some clear guidance. My prompts have varied but the one I used for Bing, which I'd previously used at OpenAI, was this:

Please give me a list of some very obscure Christmas songs in the following genres - folk-rock, Laurel Canyon singer-songwriter, shoegaze, vaporwave, freakbeat, alt-country, Americana. The songs must be very obscure. Nothing that has ever been in the Billboard Top 200 or the Top 40 of any Independent chart. They can be recent or older but no earlier than 1955.

The most recent free version of CHATGPT available through OpenAI is GPT-3.5. Bing, which is also free to use, claims to be running on GPT-4. Which is better?

 

We'll see what Bing said later but first here's how GPT3.5 responded:

Certainly! Here's a list of very obscure Christmas songs in the specified genres:

1. Folk-Rock:

   • "Snow Queen" by The Mamas & The Papas
   • "Winter Night" by The Wind in the Willows
   • "Love Surrounds Us Everywhere" by Pearls Before Swine

2. Laurel Canyon Singer-Songwriter:

   • "Christmas Cards" by Jill Sobule
   • "Snow" by Judee Sill
   • "Christmas Morning" by David Crosby

3. Shoegaze:

   • "Snowstorm" by Galaxie 500
   • "Star Roving" by Slowdive (not explicitly a Christmas song, but has a dreamy winter vibe)
   • "Winter Song" by The Dandy Warhols

4. Vaporwave:

   • "Yule Log" by 회사AUTO
   • "Christmas 通路" by waterfront dining
   • "ふわっと、さようなら" by サクラSAKURA-LEE

5. Freakbeat:

   • "Magic Potion" by The Open Mind
   • "Don't Believe in Christmas" by The Sonics
   • "Christmas Tree" by King Khan and the Shrines

6. Alt-Country:

   • "Nothing But a Child" by Steve Earle & Allison Moorer
   • "Christmastime in Washington" by The Hold Steady
   • "Christmas in Nevada" by Willard Grant Conspiracy

7. Americana:

   • "Christmas Must Be Tonight" by The Band
   • "River" by James Wesley Stemple (not to be confused with Joni Mitchell's "River")
   • "Old Toy Trains" by Nitty Gritty Dirt Band

These songs span a range of styles within each genre and offer a unique twist on the holiday season. Happy listening!

I cut and pasted every one of those into YouTube. Then I did google searches on all the ones that weren't there. It wasn't pretty.

As far as I can tell, a dozen of the twenty-one listed tracks don't exist at all. I can't even find an obvious point of confusion, like two song titles jumbled together or a real song by the wrong artist. It looks as though the AI just conjured them out of its twisted imagination.

Two are real but as far as I can see have absolutely nothing to do with Christmas or even winter. One has the right band but the name of the song is slightly off. Another is a genuine song but doesn't seem to have been recorded by the listed performer.

That leaves just five songs that are about Christmas, are called what the AI says they're called and have actually been recorded by the band or artist named. It's not a great result, is it? 

As far as the Advent Calendar goes, it's even less useful than that might suggest because one of the tracks (Don't Believe in Christmas by The Sonics.) I used last year and two of the other real ones are very much not to my taste. That leaves just two I might actually use, although I did find a few more from the YouTube suggestions that came up when I was looking for the non-existent ones.

You'll note that I didn't specify how many songs I wanted when I framed the query. I just vaguely asked for "a list". Bard and GPT3.5 both chose to interpret that as three songs in each category. I was expecting the same from Bing but I didn't get it.

Bing, as I have noticed before, tends to be quite parsimonious with its findings. Whenever I've asked it to work on the same prompt as other AIs, Bing's reply has always been the shortest. This time it outdid itself with its brevity. In answer to the exact same prompt, this is what Bing offered:

Other than the obvious fact that it's a third of the size, the first thing to say about that list is that Bing has a better hit rate when it comes to listing songs that do in fact exist. I was able to find four out of seven of those with no trouble at all. 

As far as I can tell, although the Beach Boys recorded an entire album of Christmas songs, "Christmas Time" is not on it. They did later cover Christmas Time (Is Here Again) by the Beatles though, so I guess we can give Bing a pass on that one.

I'm pretty sure Lush did not record a song called Snowfall, although it's bastard hard to be sure since all searches on the keywords are dominated by the retailer of the same name, whose odorous (Not to say odious.) products stink up the high streets of so many English towns. I can't even go into one of those stores without my eyes starting to stream so I'm not a fan. I saw Lush the band live at a festival once though and they were fine. Didn't tear up once.

Lush are also undeniably a shoegaze band; one of the best-known along with the likes of Slowdive and Ride. Bing is a bit hit-and-miss on genres in these results. I can't really see The Raveonettes as folk-rock... maybe at a stretch ... but The Monkees are categorically not freakbeat by literally any definition.

Some seem defensible. I wouldn't have lumped the Beach Boys in with the Laurel Canyon phenomenon but this is one occasion when Bing has backed up its argument with convincing evidence. That footnote, which goes to an article about the documentary movie "Echo in the Canyon", explicitly cites The Beach Boys as a key band in the movement. I should probably watch it, then I might know what I'm talking about.

While we're on the subject of Bing's citations, this is where I did actually learn something. Namely, don't trust the links they provide any more than anything else an AI tells you. You might well think that because Bing is providing actual links to its sources, they'll go to something that confirms what it said but that would be to fall into its trap.

Check them for yourself.
1. yourtango.com2. dw.com3. americana-uk.com4. vanityfair.com5. tayloronatrip.com

Firstly, six out of the seven footnotes assigned to specific songs go to the same online article: 30 Obscure Christmas Songs That Are So Underrated, compiled by Erin Watson and published by YourTango.com in December 2020. It appears all Bing did to fulfill the brief I gave it was lift almost all its results from another list. 

It's a bit like being asked to bake a cake for someone's birthday, buying one at the supermarket, putting it on a plate and then putting the plate on top of the empty box so everyone can see where it came from. And then expecting everyone to congratulate you on how clever you'd been to find such a good cake.

Except, to stretch the analogy far beyond breaking point, a close look at the ingredients would prove the cake couldn't have come out of that box at all. Bing may claim it filched six out of seven of its suggestions from that listicle but if you bother to read it all the way through, you'll find only one of them, Christmas Song by The Raveonettes, there. The rest aren't even mentioned.

I was particularly keen to find proof of the existence of Last Christmas by Saint Pepsi, a supposed cover of the Wham classic that GPT3.5 had already tried to convince me was real. It's not. None of the AIs seem able to cope with vaporwave at all, which seems somehow both appropriate and ironic at the same time.

I also searched in vain for any mention of the equally imaginary Snowfall by Lush, which obviously wasn't going to happen but even Low's quite genuine Just Like Christmas or the equally real Monkees and John Prine tracks were nowhere to be found. In some kind of doubling down on deceit, Bing seems almost self-destructively willing to refer doubters directly to the most damning evidence of it own lack of veracity. 

Even the footnotes along the bottom are weird. Only a couple refer directly to the listed tracks. The other two appear to exist purely to demonstrate that, yes, Bing does know more about the Laurel Canyon Sound than I do. It's not a competition, Bing!

Fortunately, I find all of this highly entertaining. More so than playing games right now, which is why you're reading about this instead of Tarisland. More positively, where AI is concerned, I've been having much better luck getting the image generators to come up with pictures to match the songs. That at least is something AIs are getting quite good at doing.

As for the songs, I have forty-three possible tracks listed so far to fill the twenty-four days of Advent plus Christmas Day. I might try to make it a nice, round fifty and post two a day or I might just winnow out the chaff and go with the best of the bunch. 

Honestly, that would probably be best. There's some strange stuff in there. 

However I decide to do it, It's going to be fun! No, really! It is! I promise.🤞

Notes on the AI used in this post.

Apart from the obvious. 

Header image: Dreamshaper Alpha XL2 (50/50) from the prompt "Inventory Full Advent Calendar . Rock & Roll images. Cartoon Art". I did a bunch of these and all the others were variations on an actual advent calendar but with no rock&roll whatsoever. This is all rock, no calendar. I thought that was preferable, even though every rocker seems to be some kind of mutant. I think I can see Keef and Roy Wood in there too...

Second image: Same model but at 50/0. Prompt. "The Beach Boys as Laurel Canyon folkies. 1960s underground comix poster art. No photos." Just looks like the regular Beach Boys to me. No sign of underground comix influence. Not a poster. Otherwise, exactly what I asked for...

Third image: Same model, also 50/0. Prompt: "Last Christmas by Saint Pepsi . Vaporwave" The AI clearly has no idea who or what Saint Pepsi is or might be but who can blame it for that? It does seem to have an idea what a vaporwave asthetic looks like, though. Nice image, anyway.