[HN Gopher] Turn ideas into music with MusicLM
___________________________________________________________________
Turn ideas into music with MusicLM
Author : Swizec
Score : 49 points
Date : 2023-05-10 21:32 UTC (2 days ago)
(HTM) web link (blog.google)
(TXT) w3m dump (blog.google)
| genghisjahn wrote:
| Make a typical techno song. I got this...
| https://youtu.be/JwZwkk7q25I
| ilikehurdles wrote:
| Looks promising, but I keep getting "could not fetch waitlist
| status" after signing in.
| aimor wrote:
| On the waitlist, but curious to know if it can generate non-
| instrumental sound effects.
| neom wrote:
| Make a song for the computer nerds at hackernews, the song should
| be geeky, with computer tones, modem sounds, very weird but also
| smooth:
|
| https://share.getcloudapp.com/7KuzO6QO
|
| Let's make some tropical house, use the lyrics "call my private
| number" as the hook, the vibes should be disco, light, fun:
|
| https://share.getcloudapp.com/2Nub4m6B
|
| Let's take some classical music and turn it into techno:
|
| https://share.getcloudapp.com/NQuW6lBP
|
| tropical house, 120bpm, bongos, wind chime, flutes:
|
| https://share.getcloudapp.com/2Nub4mDq
|
| Last one is pretty cool ^^
| smolder wrote:
| I'm curious whether they put any guard rails around what kinds
| of sounds you can invoke. Can you get it to synthesize
| offensive speech that's legible? Does it do gun shots, fart
| noises, orgasm sounds, crying, or screams of pain?
| neom wrote:
| sexy music with orgasmic moan - Oops, can't generate music
| for that
|
| sexy sax - Oops, can't generate music for that
|
| drill rap with gunshot - Oops, can't generate music for that
|
| song with farts - worked but it contained no farts sadly,
| just some bad techo
|
| sad song with crying - worked but contained no crying, just
| screechy violin folk music
| milchek wrote:
| Excellent, my gig as a sexy sax player is safe!
|
| Jokes aside, it's still very early days for this kind of
| generative AI. I see a real use case for virtual band mates
| that play along with whatever type of music you play when
| jamming at home, for example.
| LegitShady wrote:
| Maybe I'm a snob but they're all awful and I wouldn't listen to
| most of them once (I'd skip through) let alone twice.
| robinsonb5 wrote:
| OK they're not exactly captivating to listen to - but I don't
| think they're supposed to be. The use-case, I think, is more
| along the lines of generating adequate background music for a
| my-first-self-published-videogame project with next to zero
| effort.
| LegitShady wrote:
| I feel like that becomes sort of shovelware. There are some
| games with some truly amazing scores and the above stuff
| doesn't approach. Is it better than no music, or stock/open
| license music? I'm not sure. What happens if you've built
| the next hollow knight but rather than the amazing score
| from Christopher Larkin that elevates the game you get the
| garbled terrible music in the above post? It actually makes
| it worse.
|
| I fear the path into "low/no effort" versions of music
| scores, art, etc is a dilution of great works that actually
| makes a worse product than if you went to the trouble of
| finding someone to do it properly. It leads to shovelware
| and I don't think we need more shovelware. We need more
| high quality high intention music and games and images, and
| personally I think thats going to come from gifted
| individuals who put in the effort to learn it, not a neural
| network that can't experience what you want the music to
| make you feel.
|
| If I logged into a video game and I got the music in the
| above post it would detract from the game. You'd be better
| off finding open license or cheap music that a human has
| made. its way better, and would reflect better on your game
| and design process.
| hammyhavoc wrote:
| If someone has the vision and taste to create the next x,
| they likely have the appreciation of what decent music
| sounds like and the work that goes into that.
| cubefox wrote:
| I think you are not supposed to give it instructions, just
| descriptions like in your last prompt.
| amelius wrote:
| Nice, but somehow it feels like I've heard these songs before.
| Could that be possible?
| youssefabdelm wrote:
| Is there a way to get off the waitlist?
|
| They all suck. Nowhere near Jukebox. Depressing that seemingly
| zero progress has been made since that.
| ShamelessC wrote:
| Nowhere near Jukebox?! This is one hundred percent an
| improvement over jukebox.
| renewiltord wrote:
| Why would you want to get off the waitlist if they suck?
| youssefabdelm wrote:
| I was joking
| SeanAnderson wrote:
| ? I don't think these suck. I enjoyed the first and last ones
| genuinely. Shared them with friends, head bobbed a bit to the
| beat.
| salmonfamine wrote:
| They are just very bland and a little musically incoherent.
| They pass the basic sniff test of, "this sounds like music"
| but the more you listen the more they fall apart. It's a
| good muzak generator, but that's about it.
| SeanAnderson wrote:
| Yeah, I agree. It's not going to replace modern music
| theory today or tomorrow, but I'm excited to play with
| this sort of thing in dynamic video game content
| generation. I'm envisioning a user giving a bit of
| information about their mood and then getting auto-
| generated music while they use an app briefly. What I
| heard here seems good enough to impress some people with
| that use case.
| aatd86 wrote:
| The most impressive was actually the classical one.
|
| From a music producer standpoint if that's the quality
| for classical music, could definitely create some
| samples.
| TheOtherHobbes wrote:
| Uh - no. That was classic (not classical) formless AI
| noodling on an orchestral bed.
|
| It has multiple structural layers of a sort, which is
| progress over a few years ago. But they're still a long
| way short of the huge but intricate structures in real
| classical music.
| Paul-Craft wrote:
| I agree, but it's also really weird that such is the
| case. It's almost like "classical music" is some kind of
| code word prompting it to maintain a little musical
| coherence. The others break down rapidly after you listen
| to them for 5 seconds or so, but the classical-based one
| holds up a lot better.
| flaviuspopan wrote:
| Is it able to synthesize individual elements of a track? (kick,
| snare, hats, bass note in C4, etc). This might be killer for
| pairing with a sampler. Back in the early days of hip-hop,
| memory & disk limits encouraged artists to load samples that
| were pitched up 2x then slow back down on the device. This gave
| the samples a grainy quality which helped the aesthetics. With
| a human level of polish, I'm pretty sure the combo would be
| amazing.
| m3kw9 wrote:
| They sound "glitchy"
| nunja wrote:
| That's awesome! I immediately wanna build a quasi automated
| generative dj set and try to get some gigs around once I get
| access!
| kccqzy wrote:
| This is very cool! I can totally see this being used in low-
| budget indie video games (much like Midjourney already being
| used in video games).
| rcarr wrote:
| The last tropical house one is better than the first one in
| that it gets the house 4/4 drums right whereas the first one
| sounds more like a rap track. It's still got elevator music
| vibes though.
| nr2x wrote:
| Catching "will smith eating spaghetti" vibes.
| dS0rrow wrote:
| Thanks for sharing! In the 1st example ["computer nerds",
| "hackernews", "geeky", "computer tones"] seem to make the
| result sound more like an old-school RPG soundtrack rather than
| a song.
| SeanAnderson wrote:
| I dunno, I got Glitch Mob vibes from it. Not exactly Top 40,
| but still musical. Something like this?
| https://www.youtube.com/watch?v=DHGxEVHZXlU&t=40s
| moffkalast wrote:
| Feels a bit... noisy? Like a way overcompressed mp3.
| TheOtherHobbes wrote:
| It's basically overcompressed MP3 cliche soup.
|
| The information density of music is _much_ higher than that
| of text or still images. So something like this is still tech
| overreach.
| Applejinx wrote:
| That said, it'll be able to match modern overcompressed
| human cliche soup pretty easily. There's a lot of
| production out there which is really low hanging fruit for
| AI.
|
| It's not unlike how the visual AI can do 'Greg Rutkowski',
| but has a hell of a time being an actual concept artist in
| a functional way. If the cliche soup is well defined,
| you're pretty much all set, particularly if it's not a
| genre that requires a lot of character.
| Torkel wrote:
| Last one is much better than I expected this thing could do!
| mellosouls wrote:
| This is not intended as negative because this looks cool but I
| noped out when reading the small print about data privacy for
| Test Kitchen generally.
|
| Perhaps I was over-cautious or unfair in this case (I'm all in on
| MS/OpenAI stuff) just a gentle suggestion to give it some
| consideration if you don't normally pay it much mind.
| canadianwriter wrote:
| Non-clickbait link:
| https://aitestkitchen.withgoogle.com/experiments/music-lm
|
| Its part of AI test kitchen, if you were already part of that for
| playing with BARD and stuff you just need to log in and give it a
| go.
|
| If not, you'll have to get on the waitlist. When I asked I got
| access within a day, but might be much slower now with so much
| interest.
|
| Here are a bunch of examples of the music: https://google-
| research.github.io/seanet/musiclm/examples/
|
| In my opinion it isn't exactly great. It doesn't do much
| creativity. Eg. I asked it to make a dance song from a club but
| have classical instruments as well. It was unable to do that.
| It's mono and can often be out of tune.
|
| We will see how it improves, but this certainly isn't taking away
| jobs for musicians in it's current state.
| zwieback wrote:
| Those examples are really hard to listen to, except for maybe
| the techno/dancebeat ones.
|
| Part of the problem is the awful sound, would be much more
| interesting to have scores generated by AI and then have
| musicians record it. As it is you can't really tell why it
| sounds so awful.
| nr2x wrote:
| Yes, I actually found that painful to listen to. Interesting
| to see where it goes, but for this erstwhile musician it's
| like fingers in a chalk board.
| sfcarrot wrote:
| I know this is about how to convert text-to-music. But, is it
| just me to find most of the sample "music" to be melody-less. The
| beats and flow make it sounds like "music" but it is truly not.
| I'd call it text-to-noise for now lol
| waboremo wrote:
| Played around with this a bit and was impressed. It's absolutely
| not ready to mix you a background playlist for evening dinner yet
| that doesn't leave your guests with a migraine, but ok for
| prototyping different feels of whatever music you are trying to
| create if that's a part of your process.
|
| I believe the (tech using) public has had enough exposure to AI
| generated voices in music that this experiment will likely seem
| disappointing.
| zoogeny wrote:
| I'm sure that there is a good reason why, but I dislike that
| Google Workspace accounts are often excluded from these kinds of
| things. I mean, I pay Google every month real money and in
| exchange I get _less_?
|
| Add in the fact that I was denied access to the Bard trial due to
| it not being available in my country and I'm pretty uninterested
| in Google's releases in general these days. I had the same
| experience when the new Pixel Fold was released and instead of
| getting to see the product page I was redirected to the default
| Google store home page for my country. I couldn't even learn
| about the product.
| sp332 wrote:
| Workspace accounts get more support, right? So they only add
| products and features they're willing to do support for.
| blowski wrote:
| They could say "support is limited to the following
| services".
| shadowgovt wrote:
| sp332 got downvoted but is exactly right.
|
| Google has tiers of support for products (i.e. "How
| comfortable is SRE that they will be able to keep this thing
| in the air if every SWE that built it gets isekai'd into a
| magical universe that isn't ours tomorrow?").
|
| Anything they put out in front of Workspaces customers is
| max-tier support because their data shows that customers
| won't stop paying because they haven't gotten a new feature
| yet, but _will_ stop paying if they get a new feature and it
| 's broken.
| Paul-Craft wrote:
| > isekai'd
|
| I did not know this word, so...
|
| https://en.wikipedia.org/wiki/Isekai
| criley2 wrote:
| All Google One subscribers get support on any Google product.
| dragonwriter wrote:
| > Workspace accounts get more support, right?
|
| IF you want "Google's free consumer offering, but with more
| support and storage", that's Google One.
|
| Workspace is a business-focused offering that favors
| stability, enterprise-oriented administration, etc.; it gets
| business-focused features that aren't in the consumer
| offering, and it often doesn't get new consumer features
| until they have stabilized and additional work has gone into
| enterprise administration for them.
|
| Before Google One, buying Workspace (or whatever its name was
| then, its been through many name changes) as a way to get as
| close as possible to "Google consumer services (Docs, Sheets,
| Gmail, etc.) but with paid support", but that was never what
| the offering was about, and there now is a clear and explicit
| offering that is exactly that, so continuing to complain that
| Workspace _isn't_ what it is explicitly not intended to be
| and what Google One explicitly is somewhat pointless.
| dragonwriter wrote:
| > I'm sure that there is a good reason why, but I dislike that
| Google Workspace accounts are often excluded from these kinds
| of things. I mean, I pay Google every month real money and in
| exchange I get less?
|
| No, you pay Google for a business-focused offering that favors
| stability and proper integration of management tools for
| enterprise, so you get stability and products that have had
| proper integration of management tools for enterprise.
|
| You can pay (for more storage, support, and other benefits) for
| the Google consumer offering in the form of Google One, in
| which case you get the consumer offerings even when they are
| unstable and haven't been integrated with enterprise
| management, but don't get the business-focused components that
| come with Google Workspace business offerings.
| ElevenLathe wrote:
| Workspaces mainly focuses on enterprise customers whose
| administrators (with good reason) do not like to be surprised
| when new services are added to their users' accounts without
| some kind of vetting process by corporate security, legal, etc.
| I know many "powerusers" like to sign up because it lets them
| use "gmail" but with their own domain, but I suspect this is a
| niche use case compared to those who use it in
| corporate/enterprise environments.
| cloudking wrote:
| As of May 5th you can enable Bard for Workspace in admin panel:
| https://bard.google.com/updates
| smallerfish wrote:
| Where? I looked for it yesterday and still don't see it.
| localhost wrote:
| It's under experimental features or something like that. I
| just enabled Bard access for myself yesterday.
| shisisms wrote:
| Under enable early access apps, but rolling out over next
| 15 days apparently
| mark_l_watson wrote:
| I hear you. Twice I have signed up for Google Workplace with my
| custom domain name and was very happy with the service, except
| having to deal with two Google accounts. I subscribe to several
| Google service using my free gmail account.
|
| EDIT: I like MusicLM, I was trying it earlier today.
| VWWHFSfQ wrote:
| I'm pretty sure Google workplace accounts have a different
| privacy policy and ToS than regular accounts. It makes sense
| that they're separate if the service requires agreement to one
| ToS but not the other.
| unix_fan wrote:
| The AI test kitchen app is not accessible with voiceover
| user- wrote:
| No where in the article does it mention that you have to use a
| google account and sign up for the waiting list for their A.I
| labs
| luckydata wrote:
| is it really THAT important of a detail?
| serf wrote:
| a waitlist implies exclusivity, which is generally considered
| contrary to '..made public'.
| luckydata wrote:
| I'm going to save this as an example of "using pedantry to
| make a bad point that only the author really cares about".
| thank you for your contribution.
| jandrese wrote:
| Is it supposed to be shocking that you need a Google account to
| use a Google service?
| ccozan wrote:
| Well, search , for example. No account needed. The DNS maybe
| as well.
| WillPostForFood wrote:
| You need a google account, but not a google workspace
| account, to sign up for a waiting list to get access. That's
| pretty far from "Google makes its text-to-music AI public".
| kccqzy wrote:
| You need to log in to use ChatGPT too--you can't open an
| Incognito tab and use ChatGPT. By that definition, ChatGPT
| is not available to the public.
| d23 wrote:
| Needing to be approved from a waitlist is the difference.
| quaintdev wrote:
| I honestly need an AI to flag clickbait articles
| chaps wrote:
| You also can't signup with a workspace account.
| ilikehurdles wrote:
| Ah that explains it. I love being punished by Google for
| being a paying customer of theirs. It happens so ridiculously
| often.
| pb7 wrote:
| As a paying customer, your data is not used to improved the
| product per the terms of service which is the goal here.
| deltree7 wrote:
| [dead]
| MattGaiser wrote:
| I got off the waitlist in about a half hour when I signed up.
| zb3 wrote:
| I signed up yesterday and I'm still on waitlist. Maybe I was
| too honest..
| tonetheman wrote:
| I am a musician and this is interesting to me at least. Some of
| it is laughably bad but there is some goodness in it too.
|
| I could see this being used as a tool for song writers as it is
| currently it could be useful. Tell the AI a prompt listen to it
| and get some ideas. I do not think it is coherent enough to write
| a full song but I downloaded a few prompts and thought there is
| something good in this snippet.
|
| This is good work whoever did it.
| zyang wrote:
| Maybe someone more knowledgable with music theory can chime in,
| but the generated tunes sounds off to me. Bit like a render in
| the uncanny valley. Something is wrong but I can't put my finger
| on it.
| fullshark wrote:
| Agreed, reminds me of the MIDI era
| peterkos wrote:
| These models aren't actually doing any musical comparison --
| they are trained on audio, and from audio, piece apart with a
| "note" and a "melody" and an "instrument" are from the labelled
| training data. No intentional theory is being done!
|
| Algorithmic music composition has usually been split into two:
|
| 1. Generate notes (re: theory, genre)
|
| 2. Generate sound
|
| (i.e., EMI[0], Kulitta[1], MusicNet[2])
|
| Now we are doing both at the same time, and backwards. The
| model isn't (necessarily) going "write melody, then generate
| the sound", but rather, "here are 500 songs that are described
| with X, 500 with Y, and you want XY, so we'll combine these
| two" :)
|
| (This is my best understanding, so feel free to correct)
|
| [0]: http://artsites.ucsc.edu/faculty/cope/experiments.htm
|
| [1]: https://hackage.haskell.org/package/Kulitta
|
| [2]: https://zenodo.org/record/5120004
| TheOtherHobbes wrote:
| That is it, pretty much.
|
| The problem is that coherent musical structures are much more
| constrained. You can't just XYZ... into a space and get
| something that makes sense.
|
| That will kind of work for low-density music, which includes
| a lot of landfill dance + subgenres. But these statistical
| models are blind to larger and more complex structures, and
| completely unaware of cultural context and semantics.
|
| It's actually a harder problem than language modelling
| because the spaces and the grammars are _much_ larger,
| especially once you start including sound quality and
| production values as well as arrangement and core
| composition.
| kweingar wrote:
| Yeah, it's basically stable diffusion but for music
| drummojg wrote:
| I used "rollicking" in one description and it was exactly
| what it sounds like to your unconscious when you are far too
| drunk at a country bar
| og_kalu wrote:
| Not Stable Diffusion. MusicLM is a neural codec language
| model. Basically if GPT was predicting the next audio token
| while conditioned on text.
| ShamelessC wrote:
| So like DALLE-1?
|
| FWIW, the various diffusion models (tend to) use cross
| attention to an attention-based approach.
| montag wrote:
| My hot take is that music is vastly underdetermined by text
| prompts, leaving too much control up to the model, so it's kind
| of doomed to produce Muzak.
| celtoid wrote:
| Musician and composer here. This might make for some interesting
| sounds but it's not music in any human sense.
| ShamelessC wrote:
| Human here. I think you mean it's not music in an academic
| sense. Or have you forgotten the subjective nature of this? I'm
| certainly not happy about it, but I mean - people liked Soulja
| Boy's "Yah bitch yah" and Cee Lo Green's "Transformer". Surely
| they could find something like these tracks pleasant as well.
| Keyframe wrote:
| waiting list.. bard not available in country.. promises, talks..
| and I've been a paying customer of google on several of their
| fronts. Meanwhile I've been a paying customer of OpenAI for
| several months already. Google needs to replace CEO asap if they
| want to stay. It's evident how incompetent management there is.
| jefftk wrote:
| You can join the waitlist at
| https://aitestkitchen.withgoogle.com/experiments/music-lm
| ShamelessC wrote:
| Wait so they aren't releasing weights?
|
| (+deg#deg)+( +-+
| belter wrote:
| [flagged]
| milchek wrote:
| As a metal lover and hack guitarist, I would love an AI bandmate
| drummer that jams along with me.
|
| Back when I used to play in bands, I'd often jam with a mate who
| was a drummer, he'd wait for me to start with a riff and then
| join in.
|
| Sure, I can record some riffs at home and add realistic drums
| myself in a DAW, or use a basic drum machine, but the magic was
| in the unique unpredictability of how he'd interpret my riff and
| the beats he'd come up with on the fly.
|
| As I got older, I stopped playing in bands, lost touch with this
| particular mate (who also stopped drumming) and generally don't
| play guitar as much as I used to.
|
| Give me a generative AI bandmate that listens to me while I play
| for a second and joins in and you have my money.
| spaceman_2020 wrote:
| Anyone who has tried it - can you extract only specific
| instrument sounds from it instead of complete songs? Like just a
| kick drum sound?
| nico wrote:
| [flagged]
| pb7 wrote:
| While S&P500 trended downwards, Google's stock price is up 9%
| since PaLM 2 first leaked this week so the market does not
| agree with your assessment of the tech.
| nico wrote:
| Yes, clearly
|
| However I would say they don't agree with my assessment of
| the price
|
| Wall Street couldn't care less about Google's technology as
| long as google can make money of of it
| pb7 wrote:
| Google can only make money off of it if people use it and
| people will only use it if it's good so the immediate
| sentiment is that it's pretty good.
|
| Feel free to short Google, you can make a lot of money if
| you're right.
| klabb3 wrote:
| Competition is a critical driver of innovation and openness
| under capitalist incentives. Google had virtually no
| competition for a long time and they predictably stagnated.
| That's why it's so critical to curb monopolies-there's no way
| to "speak with your wallet" if the only choices are say Comcast
| and Spectrum. It's also way easier to regulate effectively and
| fairly than reform economic incentives in other ways. I'd
| prefer a free market economy with a vibrant distribution of
| small-medium entities over a highly regulated marketplace with
| only a few behemoths. Ironically, corporate behemoths operate
| in similar ways to bloated government bureaucracies on the
| complete opposite end of the political spectrum.
| andreyk wrote:
| FYI, this is MusicLM, which was just recently published and
| appears to be way ahead of anything out there so far. You can
| check out examples here https://google-
| research.github.io/seanet/musiclm/examples/
|
| So this is not just empty PR by Google.
| dang wrote:
| (This comment was in response to
| https://techcrunch.com/2023/05/10/google-makes-its-text-
| to-m.... We merged
| https://news.ycombinator.com/item?id=35919370 into this
| thread.)
| wilg wrote:
| I'm still sad Google bought and killed Ditty, which was such a
| fun app for making little songs out of text.
| stainablesteel wrote:
| a pretty big missed opportunity for a distributed record label
|
| if i had this i would let people make music with it for free, put
| it under a de-facto license that allows google & the creator to
| distribute it, host it on a semi-social platform, and take a cut
| of the money that rolls in
|
| but that's crazy, its not like they already have a massive video
| platform to host music videos
| endisneigh wrote:
| I'm happy that OpenAI has lit a fire under Google's butt.
| zone411 wrote:
| Maybe I'll be proven wrong but I believe this approach won't lead
| to unique, popular new songs. There isn't enough high quality
| training data. I think that for some time, the best approach will
| involve using AI as an assistant for different components of a
| song. That's what I've been doing with the melodies:
| https://www.melodies.ai/.
___________________________________________________________________
(page generated 2023-05-12 23:00 UTC)