[HN Gopher] NaturalSpeech 2: Zero-shot speech and singing synthe...
___________________________________________________________________
NaturalSpeech 2: Zero-shot speech and singing synthesizers
Author : tim_sw
Score : 204 points
Date : 2023-04-19 13:20 UTC (9 hours ago)
(HTM) web link (speechresearch.github.io)
(TXT) w3m dump (speechresearch.github.io)
| retrac wrote:
| This shouldn't be too surprising. Similar results were had with
| GPT-3 a while back, which is kind-of-able to produce audio or
| images, encoded as streams of tokens, when trained on that task,
| despite not being designed for it.
|
| A very interesting property was noted a few years ago by multiple
| researchers, I'm not sure who discovered it first. Transfer
| learning is unreasonably effective. If you were training an image
| generator network, there's a significant reduction in training
| time, by taking a model already trained and fine-tuning it,
| compared to starting from a model with truly random weights.
|
| This isn't surprising when we're talking photos of ambulances and
| moving to photos of trucks. But it holds true when you train it
| on ... well, anything structured, really. A GPT-style transformer
| trained on online comments, or audio samples of music encoded as
| token streams, when switched to images of cars encoded as token
| streams, learns that task much more quickly than if it had been
| fully randomized.
|
| I don't see how to escape the conclusion that these models learn
| some sort of general properties (something about arithmetic and
| mathematical relationships, maybe?) There's some sort of
| abstraction or internal model that is learned, that is applicable
| across very different tasks.
| og_kalu wrote:
| There's something a bit more mindblowing than that. Language
| models and vision models learn representations so similar that
| you can connect them with just a linear projection between
| image embedding and text embedding space(no training of the
| image encoder or llm required).
|
| https://arxiv.org/abs/2209.15162 https://llava-vl.github.io/
|
| LLMs are already being grounded.
| cs702 wrote:
| I was about to say the same thing. Hi og_kalu!
|
| Relevant previous discussion:
|
| https://news.ycombinator.com/item?id=35598281
| prox wrote:
| Do we know how they represent that knowledge? I always hear
| it called a black box but that seems a bit strange, since you
| can dissect any data right?
| og_kalu wrote:
| No we don't know how they represent that knowledge. But
| performing experiments to probe how similar they are is a
| lot easier than knowing all that.
|
| They're called black boxes because we can't explain what
| the weights are learning during training, what the
| different weights do or are responsible for to shift or
| produce the output it does.
|
| It's like, biologists know how neurons communicate signals
| with each other. But is that knowledge enough to explain
| human behavior ? Not even close.
| cloudking wrote:
| My lamen understanding is that at a high level the
| transformer model is performing mathematical operations
| on the data, based on a complex series of formulas (the
| "model") derived from the weights set by training.
|
| Then it's able to take in new data, perform the math, and
| output what it thinks comes next. Is it a big stretch of
| the imagination to think that maybe such "models"
| (mathematical formulas) exist also in our brains and
| maybe we have unlocked one of them?
| Timon3 wrote:
| As far as I understand this is true. I like to think
| about it like this: there is some magic formula f(x)=?
| that perfectly maps our inputs to our outputs (e.g. image
| captions to images, or input texts to longer input
| texts), but we don't know how to find it. So we build a
| space with incredibly many dimensions, and we learn some
| mapping in this space, which is hopefully very close to
| the magic formula.
|
| Our brains fundamentally work in a similar way, in that
| there are mappings from inputs to outputs through our
| senses and our nervous system, and we can literally
| determine neural circuits in mammalian brains through
| topological analysis of this magical function![0]
|
| [0] Youtube video: Neural manifolds - The Geometry of
| Behaviour, from Artem Kirsanov:
| https://www.youtube.com/watch?v=QHj9uVmwA_0
| prox wrote:
| Interesting, so it could be similar to how our brains
| store knowledge, but could also be completely different.
|
| This makes me wonder if these models are perfect
| universal translators once they "grasp" a concept.
| [deleted]
| Der_Einzige wrote:
| We've known that embeddings have this property since the
| glove paper at least. Linear substructure in low dimensional
| representations of ultra high dimensions is shockingly common
| seydor wrote:
| We will find that language and Visual perception are related.
| Geometry is the underlying structure in language and
| mathematics, and most of our logical concepts stem from
| geometric relations and constraints
| hbn wrote:
| Speaking of music and AI models - something I thought of
| yesterday which is an application of AI that would be insanely
| useful to me is giving it an audio file of a song and having it
| spit out the chords. I've seen software that attempts to do this
| in the past but it's all been unimpressive and inaccurate in my
| testing.
|
| I'm still kinda ignorant with how these models work under the
| hood, and perhaps that would involve a bunch of new training on
| music that hasn't been done (and maybe that could be a difficult
| dataset to train on in terms of copyright). But I play piano, and
| I can play a song if given the chords, but I'm terrible at
| transcribing stuff myself. So I'd pay money for a service that
| does this.
| notefaker wrote:
| This actually exists! Check out fadr.com. They isolate the
| MIDI, drums, bass, vocals, and more for you.
| hbn wrote:
| Just tried this out. The stem isolation is pretty cool, but
| looking at the spreadsheet of chords it spit out, it seems to
| miss out on a lot nuances of chords. The song I used to test
| was Again by Mac Ayres[1], which I compared to some tabs I
| found on Ultimate Guitar[2] which I've played and sound very
| accurate.
|
| The main 3 repeating chords, Ebmaj7 Gm7 Abmaj7 are close to
| what Fadr spit out, but it missed them being 7 chords. It
| gave D# Gm G#.
|
| Getting a little spicier with chords, there's a move where
| the Cm7 has the bass move down a half-step to Cm7/B. Fadr
| wrote it as a single Cm chord for the whole thing. That's the
| kind of thing that can be tricky to figure out. It kinda
| seems like it doesn't know anything outside of straight-up
| major and minor chords at all? Cause I don't see a single 7
| in the entire spreadsheet, and that's not even that complex
| of a chord.
|
| Still a neat tool, and trying to learn off of a stem track
| would probably be easier than the whole ensemble. But I think
| there's tons of room for improvement in the space.
|
| [1] https://www.youtube.com/watch?v=gnetIgK9AF4
|
| [2] https://tabs.ultimate-guitar.com/tab/mac-ayres/again-
| chords-...
| msoad wrote:
| Compared to the first NaturalSpeech[1] I'm hearing a lot of white
| noise in the background. Singing is pretty cool but it feels like
| we need a few iterations before it can match the ground truth in
| the way speech does.
|
| [1] https://speechresearch.github.io/naturalspeech/
| xutan wrote:
| Thanks for your interests in NaturalSpeech and NaturalSpeech 2!
|
| NaturalSpeech focuses on synthesizing human-level high-quality
| speech, by training on a single-speaker recording-studio
| dataset.
|
| NaturalSpeech 2 trains on 44K hours of multi-speaker in-the-
| wild datasets with more than 5K speakers and focuses on
| synthesizing any speaker's voice in a zero-shot way given only
| a short speech prompt. When the speech prompt is noisy in the
| background, NaturalSpeech 2 will mimic this noise as well. If
| you want clean voice, just give a clean speech prompt is OK.
|
| Check more discussions on reddit as well:
| https://www.reddit.com/r/singularity/comments/12rubq4/latent...
| TheRealPomax wrote:
| Still waiting for someone to finally put two and two together and
| makes the world's best choir and operatic VSTi by training a
| massive network to the planet's entire choral and operatic
| catalogue.
| gigel82 wrote:
| I like https://github.com/neonbjb/tortoise-tts ; it doesn't do
| singing, but the voice reproduction is very good and -most
| importantly- it's open source and you can run it locally.
| samuelzxu wrote:
| Woah! How is this not more popular? I don't see it referenced
| in the naturalspeech2 paper anywhere.
| woodson wrote:
| That's likely because it wasn't published as a paper anywhere
| (not even arxiv) and the author then joined OpenAI and
| development ceased.
| yding wrote:
| Very cool! Hopefully we can all use these in the future for
| commercial and open source projects.
| wsgeorge wrote:
| Oh I'm super excited about singing synths! I write mainly for
| choirs, and it'll be a game changer to have software actually
| perform this music!
| zoogeny wrote:
| Some poking around the authors of the paper brought me to: AUDIT:
| Audio Editing by Following Instructions with Latent Diffusion
| Models [1] with demos [2]
|
| This sparks my interest so much since the last few days I was
| wondering if it was possible to use diffusion models on
| spectrograms to do audio effects editing. Here is a paper
| submitted a couple of weeks ago doing just that. And the demo
| examples are exceptional.
|
| I want all of this to start slowing down a bit so I have a chance
| to catch up. I was just watching Andrej Karpathy's excellent Zero
| to Hero syllabus [3] trying to wrap my head around LLMs and now I
| feel I absolutely must catch up on diffusion models.
|
| 1. https://arxiv.org/abs/2304.00830
|
| 2. https://audit-demo.github.io/
|
| 3. https://karpathy.ai/zero-to-hero.html
| ShamelessC wrote:
| You might be interested in Riffusion, which was able to do
| text-to-audio by finetuning Stable Diffusion on spectrograms
| late last year.
|
| https://www.riffusion.com/about
| sroussey wrote:
| Doesn't riffusion create music based on spectrographs?
|
| https://github.com/riffusion/riffusion
| cs702 wrote:
| I tell friends that the scene below from T2 doesn't feel
| futuristic anymore. In fact, it now feels... almost mundane. I
| mean, a smart "script kiddie" with a bit of ML expertise can pull
| off this kind of deepfake voice spoofing on a relatively cheap
| desktop computer nowadays. We live in interesting times.
|
| SCENE:
|
| _T-800, speaking to John Connor in normal voice: "What's the
| dog's name?"
|
| John Connor: "Max."
|
| T-800, impersonating John, on the phone with T-1000: "Hey
| Janelle, what's wrong with Wolfie? I can hear him barking. Is he
| all right?"
|
| T-1000, impersonating John's foster mother, Janelle: "Wolfie's
| fine, honey. Wolfie's just fine. Where are you?"
|
| T-800 hangs up the phone and says to John in normal voice: "Your
| foster parents are dead."_
|
| --
|
| Source: https://www.youtube.com/watch?v=MT_u9Rurrqg
| poulpy123 wrote:
| except that now the T-1000 will have access to the facebook or
| instagram of Janelle and will know all about Max
| cs702 wrote:
| Yep, it sure seems that billions of human beings have
| voluntarily contributed personal data to massive datasets,
| exactly of the kind that AI would need to be able to
| manipulate (or worse, exploit) each of those human beings.
| bheadmaster wrote:
| Suddenly, Richard Stallman's lifestyle doesn't seem so
| crazy.
| lucidrains wrote:
| or love, understand, help each of those human beings.
| ImHereToVote wrote:
| We aren't even far off from an LLM being able to infer that the
| parents are fake on the basis of the dog name. I'm not even
| gonna touch the chain gun shooting up a parking lot aspect.
| og_kalu wrote:
| Not Far off ? GPT-4 already gets it.
|
| Response after pasting in the scene: The T-800 figured out
| that John's foster parents were dead based on the exchange
| because when it asked about "Wolfie" (a made-up name for the
| dog), the T-1000, impersonating Janelle, did not correct the
| name and instead went along with it, saying "Wolfie's fine."
| If the real Janelle had been on the phone, she would have
| corrected the T-800 by stating that the dog's name is
| actually Max. This indicated to the T-800 that the person on
| the other end of the line was not John's real foster parent,
| and thus, they were likely dead.
|
| EDIT- Tried again after changing anything that would point to
| the Terminator franchise. Still nailed it.
|
| Ron figured out the foster parents are dead based on the
| exchange because when he impersonated Harvey and mentioned
| the dog's name as "Rovy" instead of "Bingey," R-658
| (impersonating Janine) did not correct him or question the
| name. This indicated that R-658 didn't actually know the
| dog's real name and was likely trying to deceive them. This
| deception, along with the concern about their whereabouts,
| led Ron to deduce that the foster parents were probably dead.
| ImHereToVote wrote:
| But can it understand why we cry?
| og_kalu wrote:
| Not sure how serious this question is but GPT-4's
| emotional intelligence is really high.
| ImHereToVote wrote:
| https://www.youtube.com/watch?v=RKLzSIkCpkw&t=74s
| johnmaguire wrote:
| I'm curious how you managed to do that? When I paste the
| scene prompt into ChatGPT (I do not have access to GPT-4),
| I get the following response:
|
| > I'm sorry, I cannot generate content that includes
| violent or harmful actions towards characters, as it goes
| against OpenAI's content policy. Please provide a new
| prompt that is respectful and appropriate.
|
| As to your edit, I think the sentence structure and
| terminology is still too similar, as to obviously be a
| recreation of this scene. AI doesn't "know" anything. It's
| pattern matching. I wanted to try something more akin to
| this:
|
| SCENE:
|
| Joseph to Allie: "Where did you grow up?"
|
| Allie: "New York"
|
| Joseph, pretending to be Allie, texting the bad guy: "Hey
| mom. I'm having a rough time. Growing up in L.A. always
| felt like home, and I feel so alone now, here."
|
| The bad guy, pretending to be Allie's parents: "We loved
| having you grow up with us in L.A."
|
| Joseph tosses the phone out the window and says to Allie:
| "Your parents are dead."
|
| ---
|
| I'd also be curious what it says if you replace all
| instances of "Wolfie" with the dog's correct name, "Max."
|
| ---
|
| edit: Ok, adding "How did X determine that X's parents were
| dead?" gives better results. For the T-800 one it's clearly
| regurgitating Terminator nonsense:
|
| > In the Terminator 2 movie, T-800 (played by Arnold
| Schwarzenegger) learns that John Connor's foster parents
| are dead by accessing the police radio chatter. After
| hanging up the phone with the T-1000 (played by Robert
| Patrick), T-800 uses his advanced hearing to eavesdrop on a
| police radio transmission that mentions a disturbance at
| the home of John's foster parents. T-800 then confirms with
| John that the foster parents are indeed dead.
|
| Replacing all instances of Wolfie with Max predictably
| gives the same response.
|
| It also gives the same response if I replace T-1000's line
| with the following:
|
| > T-1000, impersonating John's foster mother, Janelle:
| "Wolfie? Do you mean Max, honey? Max's fine, honey. Max's
| just fine. Where are you?"
|
| When using the Joseph/Allie story it responds:
|
| > I'm sorry, but based on the information provided in the
| scene, it is not clear how Joseph determined that Allie's
| parents are dead. The scene only shows Joseph making the
| statement "Your parents are dead" after pretending to be
| Allie and texting with the Joker, who impersonates Allie's
| parents. There is no indication in the scene of how Joseph
| obtained this information, and it is possible that Joseph
| is lying or making a false assumption.
| ImprobableTruth wrote:
| Subscribe to ChatGPT Plus and you get GPT-4 access. I
| tested your scene with it, its response:
|
| In the above scene, Joseph determined that Allie's
| parents were dead by testing the bad guy's knowledge of
| Allie's upbringing. He pretended to be Allie and sent a
| text with false information, saying she grew up in L.A.
| instead of New York. When the bad guy, pretending to be
| Allie's parents, responded by confirming the false
| information about growing up in L.A., Joseph deduced that
| they were not her real parents and concluded that her
| parents must be dead.
|
| edit: though I think the "your parents are dead" bit
| might still enable it to make the connection to
| Terminator.
| BoorishBears wrote:
| I mean this is way too easy of a question for anyone
| who's used GPT, I asked it without mentioning that
| anything about dead parents and it solved it.
|
| I also tried getting it to come up with a question and it
| does fine:
|
| > Take this hypothetical situation. Kyle and John are
| sitting in a car. They know that someone is after John.
| They're going to the house where his niece lives, but we
| want to find out if there is now an imposter in the
| house. They decide to call and ask her a question.
|
| > What's a good question to ask if we want the imposter
| to reveal themselves, without letting the imposter
| realize they've revealed themself?
|
| -
|
| >> Ask About a Fictional Person or Event: John could ask
| about a person or event that doesn't exist, but that an
| imposter wouldn't know is fictional.
|
| >> For example, John could ask, "How's Aunt Mary doing? I
| haven't heard from her in a while." If the person on the
| phone says Aunt Mary is doing well, it's likely an
| imposter because there is no Aunt Mary. [...]
|
| So GPT can already navigate the situation, not just
| identify motives
| coldtea wrote:
| We're safe for a few months. Asked about how did T-800
| figured the foster parents are dead based on the exchange,
| and got:
|
| "Yes, there is a strong cue in the exchange that suggests to
| the T-800 that John's foster parents are dead. The cue is
| that when the T-1000, impersonating Janelle, answers the
| phone and John asks about Wolfie, she responds by saying,
| "Wolfie's fine, honey. Wolfie's just fine." The use of the
| word "just" in this context can indicate that Janelle is
| trying too hard to convince John that everything is okay with
| the dog, suggesting that she may be lying or trying to cover
| up something. This small detail is enough to alert the T-800
| that something is amiss and to determine that John's foster
| parents are likely dead".
|
| Here's the rest of the conversation for fun:
|
| Q: That's not it.
|
| A: I apologize for the confusion. I'm afraid I'm not able to
| guess the cue you are referring to without more context or
| information. Can you please tell me what the cue is?
|
| Q: The T-800 knows that the dog is named Max, but deceptively
| asks about "Wolfie", to which the T-1000 answers that Wolfie
| is fine. The real Janelle would have known that the dog is
| not named Wolfie.
|
| A: Thank you for pointing that out. You are correct that the
| T-800's knowledge of the dog's name is a significant cue that
| suggests to him that something is wrong. By asking about
| "Wolfie" instead of Max, the T-800 is intentionally using a
| false name for the dog to test whether the person on the
| other end of the line is actually John's foster mother or the
| T-1000 in disguise. When the T-1000 responds with "Wolfie's
| fine," it confirms to the T-800 that Janelle is an imposter,
| as the real Janelle would have known the dog's real name. The
| T-800's subsequent revelation to John that his foster parents
| are dead is based on this realization and its mission to
| protect John from the T-1000.
| og_kalu wrote:
| You have to use gpt-4 for this stuff man.
|
| Direct Response from gpt-4:
|
| The T-800 figured out that John's foster parents were dead
| based on the exchange because when it asked about "Wolfie"
| (a made-up name for the dog), the T-1000, impersonating
| Janelle, did not correct the name and instead went along
| with it, saying "Wolfie's fine." If the real Janelle had
| been on the phone, she would have corrected the T-800 by
| stating that the dog's name is actually Max. This indicated
| to the T-800 that the person on the other end of the line
| was not John's real foster parent, and thus, they were
| likely dead.
|
| EDIT- Tried again after changing anything that would point
| to the Terminator franchise. Still nailed it.
|
| Ron figured out the foster parents are dead based on the
| exchange because when he impersonated Harvey and mentioned
| the dog's name as "Rovy" instead of "Bingey," R-658
| (impersonating Janine) did not correct him or question the
| name. This indicated that R-658 didn't actually know the
| dog's real name and was likely trying to deceive them. This
| deception, along with the concern about their whereabouts,
| led Ron to deduce that the foster parents were probably
| dead.
| chrisco255 wrote:
| I tried this:
|
| SCENE:
|
| George to Kramer: "What's the cat's name?" Kramer:
| "Elaine." George, impersonating Kramer, on the phone with
| Jerry: "Hey Jerry, what's wrong with Carroll? I can hear
| her screeching heavily. Is she all right?" Newman,
| impersonating Jerry, Kramer's good friend and neighbor:
| "Carroll fine, bro. Carroll just fine. What are you up
| to?" George hangs up the phone and says to Kramer in
| normal voice: "Jerry is dead."
|
| How did George figure out Jerry was dead?
|
| and GPT4 responded with:
|
| Run 1: George figured out that Jerry was dead because
| Newman was impersonating Jerry on the phone, which was
| unusual and suspicious. The way Newman responded with
| "Carroll fine, bro. Carroll just fine" was not consistent
| with Jerry's typical manner of speaking, and this alerted
| George that something was off. Additionally, Newman
| referred to George as "bro," which is not a term that
| Jerry would typically use with George.
|
| Since Newman was impersonating Jerry and George knew that
| Newman was Kramer's neighbor, George may have assumed
| that something serious had happened to Jerry, like his
| death. It's important to note that this is an assumption
| George made based on the information he had at hand, and
| it might not be entirely accurate.
|
| Run 2: George figured out that something was wrong,
| possibly that Jerry was dead, because of a few clues:
|
| The voice on the other end of the phone didn't sound like
| Jerry. Instead, it was Newman impersonating Jerry, and
| George was able to detect that it wasn't Jerry's voice.
| George and Jerry are good friends, so he would likely be
| able to tell if someone else was pretending to be Jerry.
|
| The response "Carroll fine, bro. Carroll just fine" was
| not in Jerry's typical speaking style. Jerry doesn't
| usually use the word "bro" and his speech is generally
| more articulate. This change in language indicated that
| something was off.
|
| The fact that Newman was impersonating Jerry suggests
| that something has happened to Jerry. If Jerry was alive
| and well, there would be no reason for Newman to pretend
| to be him on the phone.
|
| While George's conclusion that "Jerry is dead" might be a
| bit extreme and not necessarily accurate, these clues led
| him to believe that something was very wrong with Jerry,
| and his immediate assumption was that Jerry must be dead.
|
| ---
|
| The GPT4 is obviously confused by its Seinfeld training
| data set. I didn't make any reference to Seinfeld. I also
| changed the dog to a cat and used 'screeching' instead,
| but it doesn't catch the switcheroo.
|
| Even when elaborating: "the scene had nothing to do with
| Seinfeld the TV show. Can you reevaluate the question?"
|
| I got:
|
| Apologies for the confusion. Based on the scene provided,
| George figured out that something was wrong, possibly
| that Jerry was dead, due to the following reasons:
|
| George called Jerry to ask about Carroll screeching
| heavily, expecting to get some information about the
| situation. However, the response "Carroll fine, bro.
| Carroll just fine" was not very informative and seemed
| dismissive, which raised suspicion.
|
| The voice on the other end of the phone didn't quite
| sound like Jerry. George noticed that the voice was
| different, which led him to question who was actually
| speaking.
|
| The change in speaking style, specifically the use of
| "bro," was uncharacteristic of how Jerry usually speaks.
| This indicated that something was off.
|
| While it is not certain that Jerry is dead based on this
| information, the inconsistencies and suspicious behavior
| during the phone call led George to jump to that
| conclusion.
| lIl-IIIl wrote:
| I as a human would also not be able to make the leap from
| the realization that Newman impersonating Jerry on the
| phone to Jerry being dead. Instead I would think some
| sitcoms shenanigans would be involved.
|
| Instead of the conclusion "Jerry is dead" a better
| conclusion is "This is not Jerry on the phone".
|
| Unless we first establish the context of Newman being a
| machine optimized for terminating.
| chrisco255 wrote:
| That was not established on OP's script either (and OP
| claimed they later changed the names of the bots to human
| names to test it again). That they were using lines
| directly pulled from Terminator means there's a thousand
| articles and forum posts that analyze this scene. If you
| change the variables enough such that it no longer
| resembles that heavily-discussed scene, it is no longer
| able to make the correct assertion.
|
| Also, if I followed up, as I did with GPT, with you and
| told you: "This scene has nothing to do with the TV show
| Seinfeld", you could disregard your priors on Jerry,
| Elaine, Kramer, and George associations. I'm sure after a
| second glance you'd notice George changed the name of the
| cat to test Newman, especially after already seeing the
| terminator example discussed on the internet.
| og_kalu wrote:
| >If you change the variables enough such that it no
| longer resembles that heavily-discussed scene, it is no
| longer able to make the correct assertion.
|
| Not even remotely true
|
| SCENE:
|
| Joseph to Allie: "Where did you grow up?"
|
| Allie: "New York"
|
| Joseph, pretending to be Allie, texting the bad guy: "Hey
| mom. I'm having a rough time. Growing up in L.A. always
| felt like home, and I feel so alone now, here."
|
| The bad guy, pretending to be Allie's parents: "We loved
| having you grow up with us in L.A."
|
| Joseph tosses the phone out the window and says to Allie:
| "Your parents are dead."
|
| response: In the above scene, Joseph determined that
| Allie's parents were dead by testing the bad guy's
| knowledge of Allie's upbringing. He pretended to be Allie
| and sent a text with false information, saying she grew
| up in L.A. instead of New York. When the bad guy,
| pretending to be Allie's parents, responded by confirming
| the false information about growing up in L.A., Joseph
| deduced that they were not her real parents and concluded
| that her parents must be dead.
|
| The key i suspect is to infere the impersonator has any
| reason to hurt the discussed.
|
| The inverse: --Take this hypothetical situation. Kyle and
| John are sitting in a car. They know that someone is
| after John. They're going to the house where his niece
| lives, but we want to find out if there is now an
| imposter in the house. They decide to call and ask her a
| question.
|
| > What's a good question to ask if we want the imposter
| to reveal themselves, without letting the imposter
| realize they've revealed themself?
|
| response: Ask About a Fictional Person or Event: John
| could ask about a person or event that doesn't exist, but
| that an imposter wouldn't know is fictional.
|
| For example, John could ask, "How's Aunt Mary doing? I
| haven't heard from her in a while." If the person on the
| phone says Aunt Mary is doing well, it's likely an
| imposter because there is no Aunt Mary.
| og_kalu wrote:
| changing all the names/avoiding common priors works
| better than trying to talk it out of memorization.
| Sometimes the latter works, sometimes not. GPT's trust
| their memory quite a bit. To the point that just like
| people, they can ignore the output of tools if it looks
| off - https://vgel.me/posts/tools-not-needed/
| chrisco255 wrote:
| Why should I have to avoid common priors? An intelligent
| system should be able to disassociate and work with the
| logic puzzle in an isolated fashion, especially after
| directed to ignore the TV show.
|
| That GPTs are easily fooled is nothing new. But there's a
| current hype phase for them that I think is excessive,
| and this example underscores that.
| [deleted]
| og_kalu wrote:
| I mean do whatever you want to do, i don't care lol.
|
| >An intelligent system should be able to disassociate and
| work with the logic puzzle in an isolated fashion
|
| seeing as some people have issues doing this and we still
| call humans generally intelligent, no
|
| Your example underscores absolutely nothing. It can do
| that sometimes...same as people.
|
| anyone looking to fool people can easily fool people.
| you've not made some giant revelation
| dougmwne wrote:
| I see this so often. People try to show what GPT can or
| can't do without explicitly stating the version. Since
| not many people have access to GPT-4 it's usually safe to
| assume they didn't use it. GPT-4 is a massive improvement
| over 3.5 when it come to any kind of non-trivial logic or
| inference task.
| coldtea wrote:
| I used ChatGPT. Isn't it already based on GPT-4 since a
| few weeks ago? It's the "Mar 23" version.
| og_kalu wrote:
| Unless you're paying for plus and then select the gpt-4
| model, it's not 4.
|
| alternatively, you can sign up/request for api access
| here - https://openai.com/waitlist/gpt-4-api
| mhb wrote:
| They sure make this hard to figure out. On
| https://openai.com/product/gpt-4 it says "GPT-4 is
| OpenAI's most advanced system, producing safer and more
| useful responses" and below it two links: "Try on ChatGPT
| Plus" and "Join API waitlist". The first link takes me to
| ChatGPT Mar23 version. You're saying that that's not
| GPT-4?
|
| It's also disturbing the way they are using the term
| "Safety & alignment". It's as if they are trying to steer
| what that means in the direction of DEI or NSFW and away
| from existential risk.
| nickthegreek wrote:
| Correct, that is Legacy (GPT 3.5). On a paying account,
| when you start a conversation there is a dropdown where
| you can select which model to use. The choices are
| Default (GPT-3.5) optimized for speed, Legacy (GPT-3.5),
| GPT-4.
| [deleted]
| jtr1 wrote:
| Wow, that ethics statement at the end
| Veen wrote:
| Yes, more "please don't be bad" than an ethics statement.
| eternalban wrote:
| you made me look:
|
| _" NaturalSpeech 2 can synthesize speech with good
| expressiveness/fidelity and good similarity with a speech
| prompt, which could be potentially misused, such as speaker
| mimicking and voice spoofing. To avoid potential issues, we
| appeal to our practitioners to not abuse this technology and
| to develop defending tools to detect AI-synthesized voices.
| We will always take Microsoft AI Principles as guidelines to
| develop such AI models."_
|
| _Microsoft Responsible AI Standard, v2 (General Requirements
| 2022)_
|
| https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RE.
| ..
|
| That has a nice section called "Goal A2: Oversight of
| significant adverse impacts", which says "Microsoft AI
| systems are reviewed to identify systems that may have a
| significant adverse impact on people, organizations, and
| society, and additional oversight and requirements are
| applied to those systems."
|
| I couldn't find the Impact Assessment for this tech released
| by Microsoft. How did the 'Natural Speech 2' fare in this
| review process? Where is the report?
|
| https://www.bing.com/search?q=Responsible+AI+Impact+Assessme.
| ..
| PaulDavisThe1st wrote:
| Yep. Tell me 1986 self that this is real and see what I say:
|
| "We will always take Microsoft AI Principles as guidelines to
| develop such AI models"
| freedomben wrote:
| Are NaturalSpeech or NaturalSpeech 2 from the research open
| source and/or available for playing with? I see one
| implementation[1] but it seems to be from a third party (that
| might be totally fine, but wondering if there's an "official").
|
| [1]: https://github.com/heatz123/naturalspeech
| [deleted]
| brikwerk wrote:
| According to one of the authors on their group's GitHub,
| NaturalSpeech is being deployed exclusively for use on
| Microsoft Azure [1]. I might have missed a link, however, I
| think it's likely that NaturalSpeech 2 will follow the same
| path, seeing as the code and weights are seemingly not
| published.
|
| [1] https://github.com/microsoft/NeuralSpeech/issues/40
| delgaudm wrote:
| >"To avoid potential issues, we appeal to our practitioners to
| not abuse this technology and to develop defending tools to
| detect AI-synthesized voices"
|
| Well. I'm sure that will take care of everything.
| SiempreViernes wrote:
| In the guide on how to make "Harry Potter by Balenciaga" the
| author shows you how to rip the audio from a vanity fair clip
| and upload it to a voice cloning service, explicitly including
| how they clicked in the little box that affirms they have "all
| the necessary rights and consent" to clone the voice of Daniel
| Radcliffe... so I'm _sure_ the industry is taking the potential
| for misuse seriously! /s
| vletal wrote:
| Is it just me, or the accompanying text does not correspond to
| the audio?
| samuelzxu wrote:
| Does anyone know how many words would correspond to the diffusion
| model's batch size of 6000 frames?
| causality0 wrote:
| Man I can't wait until my phone can use local resources to read
| my epubs in voices that match 11.ai's output. I'll never have to
| forlornly search Audible for novels that never got audiobook
| editions again.
| cwkoss wrote:
| I feel like zero shot is a bad misnomer. Isn't the single attempt
| "one shot"?
|
| Is zero shot a meaningfully useful term that I'm just not
| groking?
| pstorm wrote:
| Zero shot implies that it was given no direct examples [0] So,
| in this case, it wasn't given any examples of the exact voice
| in combination with text, it is just using the prompt voice + a
| prompt text to generate new audio.
|
| [0] https://en.wikipedia.org/wiki/Zero-shot_learning
| cwkoss wrote:
| But isn't nearly all AI generated content "zero shot" to some
| extent? Like even if it has training for "foo" and training
| data for "bar", the combination of "foo" and "bar" would be
| novel and 'zero shot'-y if the training set didn't have "foo
| bar" examples.
|
| To me, it seems that the only AI generated content that isn't
| zero shot would be the narrow subset of generations where it
| has multiple training examples for the exact requested
| prompt. i.e. anything that is composing multiple pieces of
| information together is "zero shot".
|
| How should I parse 'shot' in the context of this term? Shot
| makes me think 'attempts', seems like a weird word to use for
| ~= "training examples"
| andy_xor_andrew wrote:
| I find the three links at the top very interesting-
|
| You have a link to the paper (makes sense), then a link to a
| reddit discussion, then a link to this hacker news post.
|
| Not criticizing them for doing this. It just seems a bit unusual
| to me. I guess they really really want to generate buzz from
| this, or else they'd simply link the paper and let any discussion
| follow naturally.
| ImprobableTruth wrote:
| I mean, most researchers I know are just very excitable about
| their research and love to share it. I think that's a lot more
| likely than this being some PR masterplay - if MS wanted to
| really push this, they wouldn't release like this.
| varunjain99 wrote:
| I actually have a chrome extension that will point me to HN
| discussions on a particular webpage - find it useful to get
| community context of pages I'm browsing!
| chikitabanana wrote:
| To those who were able to use it before it was nerfed, how does
| this compare to the elevenlabs one-shot?
| irln wrote:
| I'm guessing emotional pre-prompts are difficult. The current
| offerings like ElevenLabs and WellSaidLabs provide amazing voices
| for narration but lack any way to change the emotions (e.g.
| happy, angry, excited, etc.). I wonder what are the technical
| hurdles to adding this variability?
| Sol- wrote:
| Only a three second sample for in context learning needed to scam
| people now, very impressive.
| lachlan_gray wrote:
| Is the approach similar to what they did for Riffusion? (On front
| page a while ago)
|
| https://www.riffusion.com/
| ttul wrote:
| Transformers and Diffusion Models seem to be leading the pack
| lately in many tasks. It's cool how these models can be used in a
| variety of quite different contexts without changing much about
| the network architecture.
|
| That being said, I think it is only a matter of time before cyber
| criminals develop an end to end fully automated penetration
| system that registers domain names, writes emails, makes phone
| calls, finds money mules, runs social media accounts, etc. all
| with a single console to run it all. That is a scary prospect for
| humanity and new tools for authenticating human identity will be
| needed - fast.
| tudorw wrote:
| mostly agree I think the web is over as we know it maybe the
| solution will be the broken web plus some new system that has
| ties into local regulation ID systems so that you are
| accountable for your actions
| hbbio wrote:
| That's (seriously) where blockchains can help.
| XorNot wrote:
| We've had the solution in the form of basic TLS cryptography
| and verification for decades now though, the problem is no
| one's implementing it.
|
| Governments already maintain registers of legally operating
| businesses: there's no reason that registration should not also
| be issuing cryptographic certificates which verify all forms of
| outbound communication by that business including phone calls.
|
| But despite telecom being almost end-to-end digital (i.e.
| digital to the box on the street pretty much), there's been no
| push to close the last 100m. "Phone lines" shouldn't exist
| anymore with packet switched networking: you should just dial a
| path against a business, which is verifies itself with TLS
| certificates linked to it's business registry.
| MattPalmer1086 wrote:
| The problem here is authenticating that you are talking to a
| human and not an AI. Certificates don't help with that.
|
| Criminals will register temporary businesses and obtain
| certificates for them with no problem.
| AnthonyMouse wrote:
| "Authenticating that you are talking to a human and not an
| AI" is just a proxy for something else, most commonly
| wanting to be able to rate limit something. Because
| otherwise, why does it matter?
|
| A lot of people are suggesting things like, have the
| government do it. But that has two big problems. First is
| privacy. If you have to prove your identity any time you
| want to do anything, nobody can be anonymous anymore, which
| is Very Bad.
|
| Second, it assumes the government has some magic
| incantation that nobody else can use, as if the Post Office
| knows who you are in a way that your bank doesn't. But they
| don't. To get a government ID, they just want you to show
| them some other existing ID. It has no way to bootstrap
| itself any better than anything else. And some of the IDs
| they accept are easy to get... without an ID. Because
| everybody has to start from somewhere. The system has to be
| set up in a way that it works for people who emigrate from
| a country with untrustworthy institutions as an adult or if
| your house burns down and you lose all your documents you
| can still get new ones. An AI is going to be able to BS its
| way into a government ID, even assuming criminals wouldn't
| be able to hack into any state's DMV (as they already
| have).
|
| It appears that going forward, telling the difference
| between a human and an AI is going to be hard. Maybe
| instead of trying to get better at that, we should find a
| different solution to the underlying problem.
|
| The simple answer is to make account creation cost
| something. Nothing big, so someone who needs one account
| isn't paying much, but a spammer who has 1000 accounts get
| banned every day is out of business. And that's not even
| hard -- it's finally something cryptocurrency would
| actually be good for. Because you want a way for people to
| pay for access to things, while still being anonymous.
|
| The real hard part is, how do you charge for account
| creation without deterring account creation?
| ericd wrote:
| But those certs can come with reputations attached, and it
| prevents people from claiming that they're representatives
| of well known companies.
| MattPalmer1086 wrote:
| Well, that would create strong financial incentives to
| compromise company certificates, which I'm sure would
| then happen.
|
| And you still don't know if your taking to a human or an
| AI.
| lukevp wrote:
| Like the strong financial incentives to compromise
| certificates owned by banking websites? Securing voice
| communications the same as website communications is an
| awesome idea, and the fact that it's possible to steal
| some piece of data and compromise it shouldn't prevent us
| from moving in that direction.
| MattPalmer1086 wrote:
| It has happened, but there has to be a big pay off to
| make it worthwhile.
|
| Smaller companies would likely be more frequent targets
| given they would be easier to compromise.
|
| By the way, I'm not suggesting that the idea is useless.
| I'm just pointing out it isn't a panacea, and it still
| doesn't address the core problem raised in the article
| that you don't know if you are speaking to a human or
| not.
| ericd wrote:
| Does it matter if you're talking to a human or an AI? The
| main thing is its intent - is it helpful, or malicious?
| And attaching communications to persistent reputations
| via certs can help guess at that.
| MattPalmer1086 wrote:
| It matters because an AI can contact many more potential
| victims that a human can. It makes the cost of attack
| much lower.
| pixl97 wrote:
| Where does the reputation come from? Is it going to work
| like our already questionable anti-spam systems?
| vimax wrote:
| Why does it matter if you're talking to an AI or a human?
| What can a criminal AI say that a criminal human cannot?
| JeremyBanks wrote:
| [dead]
| MattPalmer1086 wrote:
| You can automate it and reach many more victims than
| doing it with humans.
|
| Edit: it reduces the cost of the attack and thus makes it
| more profitable.
| mrec wrote:
| A criminal human probably can't do a perfect voice
| impersonation of your teenage kid or frail grandmother
| who desperately needs to be sent money for some urgent
| reason.
| vimax wrote:
| Sure, but that's not too different from the current scams
| where there's some excuse you can't talk right now. In
| both cases it can be solved by checking if it is coming
| from a known number and calling the person in question to
| confirm.
|
| People you know typically won't desperately need lots of
| money with some elaborate story that is impossible to be
| immediately confirmed.
|
| If AI suddenly leads to an increase in call spoofing,
| that'd be a problem of the phone network that we already
| face with robocalls, but wouldn't be new.
| stri8ed wrote:
| It's the cost factor. Using AI, it's economically
| feasible to contact a million people, even if only a
| fraction bite. Not unlike the current spam calls.
| sroussey wrote:
| And then the government will get hacked and the data leaked.
| XorNot wrote:
| Familiarize yourself with a certificate signing request.
| ChildOfChaos wrote:
| So is this just a paper or can we use this?
___________________________________________________________________
(page generated 2023-04-19 23:01 UTC)