Yeah, I did a few trial runs on the cluster gpu instances maybe 4 months ago. I found that, while the gpus themselves were really quite fast, moving data in and out of gpu was not. Maybe amazon will focus on increasing bandwidth to the gpu for the new boxes.
To be fair, there should not be a lot of data transfers to and from the GPU. Moving larger chunks of data (instead of many smaller ones) when you are running out of memory, using asynchronous data / compute streams would increase the performance.
