> In September 2019 this must have been a GAN face.
Well, yes,¹ but actually no. StyleGAN1's public release was February 2019, and it's capable of far more than just faces.
> Three years ago there was nothing remotely as impressive as modern GPT style
I predicted that! Albeit, not publicly, so I can't proveit.
My predictions claimed it would have certain limitations, which GPT-3 (and, later, GPT-4) exhibit. I can show that those still exist, but few people on Hacker News seem to understand when I try to communicate that.²
> or text to image models.
Artbreeder (then called Ganbreeder) existed in early 2020, and it didn't take me by surprise when it came out. It parameterises the output of the model by mapping sliders to regions of the latent space; quite an obvious thing to do, if you want to try getting fine-grained control over the output. (A 2015 paper built on this technique: https://arxiv.org/abs/1508.06576)
I was using spaCy back in 2017–2018-time. It represents sentences as vectors, that you can do stuff like cosine similarity on.
If I'd been more interested in the field back then, I could have put two and two together, and realised you could train a net on labelled images (with supervised learning) to map a spaCy model's space to StyleGAN's, which would be a text-to-image model. It was very much imaginable back before April of 2020; a wealthy non-researcher hobbyist could've made one, using off-the-shelf tools.
If I were better at literature searches, I could probably find you an example of someone who'd done that, or something like it!
---
¹: That file was, because that's what I was playing with in September. I do have some earlier ones, of landscapes; they're in a different folder.
²: See e.g. here: they tell me GPT-4 can translate more than just explicitly-specified meaning, and the "evidence" doesn't even manage that. https://news.ycombinator.com/item?id=35530316 (They also think translating the title of a game is the same as translating the game, for some reason; that confusion was probably my fault.)
Well, yes,¹ but actually no. StyleGAN1's public release was February 2019, and it's capable of far more than just faces.
> Three years ago there was nothing remotely as impressive as modern GPT style
I predicted that! Albeit, not publicly, so I can't proveit.
My predictions claimed it would have certain limitations, which GPT-3 (and, later, GPT-4) exhibit. I can show that those still exist, but few people on Hacker News seem to understand when I try to communicate that.²
> or text to image models.
Artbreeder (then called Ganbreeder) existed in early 2020, and it didn't take me by surprise when it came out. It parameterises the output of the model by mapping sliders to regions of the latent space; quite an obvious thing to do, if you want to try getting fine-grained control over the output. (A 2015 paper built on this technique: https://arxiv.org/abs/1508.06576)
I was using spaCy back in 2017–2018-time. It represents sentences as vectors, that you can do stuff like cosine similarity on.
If I'd been more interested in the field back then, I could have put two and two together, and realised you could train a net on labelled images (with supervised learning) to map a spaCy model's space to StyleGAN's, which would be a text-to-image model. It was very much imaginable back before April of 2020; a wealthy non-researcher hobbyist could've made one, using off-the-shelf tools.
If I were better at literature searches, I could probably find you an example of someone who'd done that, or something like it!
---
¹: That file was, because that's what I was playing with in September. I do have some earlier ones, of landscapes; they're in a different folder.
²: See e.g. here: they tell me GPT-4 can translate more than just explicitly-specified meaning, and the "evidence" doesn't even manage that. https://news.ycombinator.com/item?id=35530316 (They also think translating the title of a game is the same as translating the game, for some reason; that confusion was probably my fault.)