Post 9gdcaPf0HWFPPPdiYS by aparrish@mastodon.social
 (DIR) More posts by aparrish@mastodon.social
 (DIR) Post #9gdbtCxTdED3Yug8uW by aparrish@mastodon.social
       2019-01-26T21:52:34Z
       
       0 likes, 0 repeats
       
       and a little interface for it. this is trying to spell the words using phonetic information (using a sequence-to-sequence neural network), the temperature parameter basically controls how the probabilities are distributed (at low temperatures, only the most likely characters are generated according to the information in the model; at higher temperatures, any character might be generated)I need to stop playing with this, I have other stuff to do geez
       
 (DIR) Post #9gdbtDCMjs7SJ5e27M by aparrish@mastodon.social
       2019-02-02T19:17:32Z
       
       0 likes, 0 repeats
       
       still at work on this english nonsense word vae. here are some  nonsense words sampled from the latent space of the latest trained model...twidletuppilledentedrultremdpobechominsbowgripkandirquineusdudenowedrostorekigannedermottasastorslielandizessermasricknestchatedthese are generated by feeding the decoder with normally-distributed random numbers. pretty happy with how they all seem like jabberwockian-yet-plausible english words
       
 (DIR) Post #9gdbtDLECvCykZn6vo by aparrish@mastodon.social
       2019-02-02T19:20:49Z
       
       0 likes, 0 repeats
       
       by contrast, results of feeding normally-distributed random numbers into the decoder on the RNN without the VAE:flfingengaughumsalohondismh'sh'sautabovagakeleghearh'salliltallesbarngnongh'smookshewstlatscrethhuthurecheltharth'snot as good! which is encouraging, since it shows that the VAE model does actually have a "smoother" space than the non-VAE model.
       
 (DIR) Post #9gdbtDVVahQpGSbJxI by aparrish@mastodon.social
       2019-02-02T19:25:40Z
       
       0 likes, 0 repeats
       
       (I have to admit that when I started this project I was like, "why do you even need a variational autoencoder, if just plugging random vectors into the decoder was good enough for jesus it's good enough for me," but there really is something magical and satisfying about being able to get more-or-less plausible generated results for basically any randomly sampled point in the distribution)
       
 (DIR) Post #9gdbtDhYrt4ZrqEwk4 by aparrish@mastodon.social
       2019-02-05T19:21:54Z
       
       0 likes, 0 repeats
       
       progress: at 50 epochs, even w/KL annealing, 32dims is not enough for the VAE latent vector to represent much of anything. leads to reconstructions that are probably just the orthography model doing its best with next-to-noise, but sometimes amusing, e.g.cart → puachliotta → pinterajanintellectually → achingcapella → pellakaphotometer → aughsympathizer → disteghwaybutrick → jorserichbotha's → szineclayman → tsantierschesparkles → trenlewcalamity → mulissthermoplastic → tphare
       
 (DIR) Post #9gdbtDrqFfIQNj39lY by aparrish@mastodon.social
       2019-02-05T19:24:08Z
       
       0 likes, 0 repeats
       
       (posted this mainly because "butrick → jorserich" seems like something mastodon people would like, e.g. "my name is Butrick Jorserich, follow me at jeans.butrick.horse/@rich")
       
 (DIR) Post #9gdbtE1lelEgsVh5Em by aparrish@mastodon.social
       2019-02-12T23:33:35Z
       
       0 likes, 0 repeats
       
       apparently the trick to training a VAE w/annealing is to *never* let the KL loss go below the reconstruction loss. otherwise you get beautifully distributed, wonderfully plausible reconstructions that have almost nothing to do with your training data, i.e., "allison" becomesuszecuinauruselin-timerellefleighcarmistachubaralsaahoughtrodhanasceareddingearpughihtioz
       
 (DIR) Post #9gdbtEDSxGarSnAQTI by aparrish@mastodon.social
       2019-02-19T14:51:07Z
       
       0 likes, 0 repeats
       
       exploring the latent phonetic nonsense space around "typewriter"—using the best model I've managed to train yet (100 epochs on HPC, managed to keep the reconstruction loss fairly low while also getting some semblance of a low KL loss)
       
 (DIR) Post #9gdbtEKuVaY3pseN4i by aparrish@mastodon.social
       2019-03-10T19:35:23Z
       
       0 likes, 1 repeats
       
       using the phonetic VAE to interpolate between US state names in a grid
       
 (DIR) Post #9gdbtEUpugUKKfIIXw by kragen@nerdculture.de
       2019-03-10T19:38:59Z
       
       0 likes, 0 repeats
       
       @aparrish some of these rows and columns make good nonsense poetry
       
 (DIR) Post #9gdcaPf0HWFPPPdiYS by aparrish@mastodon.social
       2019-03-10T19:46:50Z
       
       0 likes, 0 repeats
       
       @kragen that's the idea!