Java needs an environment that can generate, parse and rearrange Java code in a way that vimscript just isn't suited for. Even in Emacs it's a Herculean effort to make the editor understand Java at a deep enough level. Generally, if the language you're using co-evolved with a particular editor, use that editor, not vim. (That's not limited to Java.)
On the other hand, C as a language grew up alongside vi, as well as the Unix environment it leverages so well. Languages with a nice REPL, like Python and Ruby, also are concise and flexible enough that any text editor will do; the languages are meant to work well enough with minimal editing capabilities. In those cases, vim is still great, because it's optimized for rearranging text of any kind.
I know hard-core Java people that use Vim. It integrates with Eclipse (by talking to a headless instance over a socket) for the refactoring stuff. Vim for the editing, Eclipse for the Java munging. Seems to work very well.
Never use an "IDE" unless there is absolutely no other way.
On the other hand, C as a language grew up alongside vi, as well as the Unix environment it leverages so well. Languages with a nice REPL, like Python and Ruby, also are concise and flexible enough that any text editor will do; the languages are meant to work well enough with minimal editing capabilities. In those cases, vim is still great, because it's optimized for rearranging text of any kind.