AI singer-songwriter āAnna Indianaā debuted her first single āBetrayed by this Townā on X, formerly Twitterāand listeners were not too impressed.
AI singer-songwriter āAnna Indianaā debuted her first single āBetrayed by this Townā on X, formerly Twitterāand listeners were not too impressed.
The human imagination also involves the phenomenal experience. You do not just record the data coming at you and regurgitate it, you experience it and then your experience further changes the data itself. We call this āsubjectivityā and itās where creativity comes from.
I am not saying that machine creativity is impossible. What Iām saying is these LLMs are not creative because they donāt even know what theyāre doing and they donāt even know ātheyā are doing it. Thereās no āthereā there. No more creative than rolling dice.
and experience is ongoing learning, so if an LLM were training on things after the pretraining period then thatād allow it to be creative in your definition?
but in that case, whatās the difference between doing that all at once, and doing it over a period of time?
experience is just tweaking your neurons to make new/different connections
This. Humans are just meat calculators when you zoom out.
Experience is ongoing learning through the subjective self. When you experience the color red you do not just record it with your photoreceptors, and your experience of the color red is different from mine because we donāt just record wavelengths of light. We donāt just continue to learn from continual exposure to new data, we also continue to learn from generating our own data. In this way our subjective experience is qualitative, not simply quantitative. I donāt just see the specific light wavelengths, I experience the ārednessā of red.
When LLM is trained on that kind of data it just starts to hallucinate. This is promising! I think the hallucination phenomenon is actually a precursor to creativity and gives us great insights into the nature of subjective experience. In a sense, my phenomenal experience of the color red is actually much like a hallucination where I am also able to experience the colorās āwarmthā and āboldnessā. Subjectivity.
itās only qualitative because we donāt understand it
when an LLM āexperiencesā new data via training, thatās subjective too: it works its way through the network in a manner thatās different depending on what came before itā¦ if different training data came before it, the network would look differently and the data would change the network as a whole in a different way
When an LLM feeds on its own outputs, though, it quickly starts to hallucinate. I think this is actually closer to creativity, but it betrays the fundamental flaw behind the technology - it does not think about its own thoughts and requires a curator to help it create.
Iāll believe something is an AI when it can be its own curator and not drive itself insane.
thatās a lack of understanding of concepts though, rather than a lack of creativityā¦ curation requires that you understand the concept that youāre trying to curate: this looks more like a dog than this; this is a more attractive sunset than this
current LLMs and ML donāt understand concepts, which is their main issue
id argue that it kind of does āthink about its own thoughtsā to some degree: modern ML is layered, and each layer of the net feeds into the nextā¦ one layer of the net āthinks aboutā the āthoughtsā of the previous layer. now, it doesnāt do this as a whole but neither do we: memories and neural connections are lossy; heck even creating a creative work isnāt going to turn out exactly like you thought it in your head (your muscle memory and skill level will effect the translation from brain to paper/canvas/screen)
but even we hallucinate in the same way. donāt look at a bike, and then try and draw a bikeā¦ youāll get general things like pedals, wheels, seat, handlebars, but itāll be all connected wrong. this is a common example people use to show how our brains arenāt as precise and we might like to thinkā¦ drawing a bike requires a lot of very specific things to be in very specific places and thatās not how our brain remembers the concept of ābikeā
This is a relevant issue to the question!
If I take a dose of LSD and paint the colors I hallucinate, is that creative? Iād argue itās not.
Only when I, the subjective self, curate my own thoughts and sensations can I engage in a creative process. I can think about my own thoughts without going insane (how do the colors make me feel, what do the colors mean?) and thatās a fundamental part of creativity and intelligence. Conceptualization is key to subjectivity.
I donāt think this is far off. I just donāt think weāre there, either, and we should be skeptical of marketing hype.
i donāt agree with that definition of creativeā¦ thereās lots of engineering work thatās creative: writing code and designing systems can be a very creative process, but doesnāt involve feelingā¦ itās problem solving, and thats a creative process. youāre narrowly defining creativity as artistic expression of emotion, however thereās lots of ways to be creative
now, i think thats a bit of a strawman (so iāll elaborate on the broader point), but i think its important to define terms
i agree we should be skeptical of marketing hype for sure: the type of creativity that i believe ML is currently capable of is directionless. it doesnāt understand what itās creatingā¦ but the truth lies somewhere in the middle
ML is definitively creating something new that didnāt exist before (in fact iād say that its trouble with hallucinations of language are a good example of that: it certainly didnāt copy those characters/words from anywhere!)ā¦ this fits the easiest definition of creative: marked by the ability or power to create
the far more difficult definition is: having the quality of something created rather than imitated
the key here being ārather than imitatedā which is a really hard thing to prove, even for humans! which is why our copyright laws basically say that if you have evidence that you created something first, you pretty much win: we donāt really try to decide whether something was created or imitated
with things like transformative works or things that are similar, itās a bit more of a grey areaā¦ but the argument isnāt about whether something is an imitation; rather itās argued about how different the work is from the original
The same could be said of a lot of creatives. You speak of greater creativity, that which evokes depth and gravity. There is still more shallow creativity. Learning creativity. That which you do before you learn to do better. Kind of what these are doing.
Iām not saying itās good or bad, though the people who hold the reigns definitely donāt have the best intentions for their use, but underestimating it is the first step to allowing them to run rampant.
āNever attribute to malice that which you can attribute to stupidityā is the slogan of those who do nothing but look down on othersā¦ who underestimate the horrible things the āstupidā can do. Donāt assume stupidity just because you donāt like something. It makes it that much easier for it to bite you on the ass in the future.
I donāt think Iād actually call that shallow thought ācreativityā.
Think of a word association game. I donāt think the first word that pops up in my head is creative at all, itās just a thoughtless reaction.
Thatās what LLMs are doing. Without that reflection and depth itās just a direct input->output