IMO Xeon will be tough to unseat for performance. But a lot of server consumers only want to be able to scale ever-more guest OS/containers on a single platform, so these challengers do pose a real threat to Intel's server market.
Open source software has been a powerful economic force: the fact that so much software exists that's designed to target many similar platforms and is frequently recompiled for those targets means that the impact to the end-consumer of switching among x86/ppc/arm is much lower than it was in the past.
I don't know if they have to beat Intel on performance per core. There are markets that require the best performance per core, but there is also a big market that wants more reasonably powerful cores for a given price point.
For instance, in the VPS hosting market, you see most providers offer "vCPUs" (virtual CPUs), because then they can sell their customers "four cores" (or 20 cores) on the cheap, and so on. It's a good marketing strategy.
What if they can offer real cores for the same low price? That would be pretty appealing to their customers. They can also offer "24-core dedicated servers" on the cheap, and so on.
Also, in the long run, it would be best for hosting and cloud providers if they would at least adopt a strategy of 50% Intel chips, and 50% AMD + Power + ARM. That should get them much better prices in the future (from all the chip providers), if they did that. Monopolies don't serve anyone but the monopolist.
Open source software has been a powerful economic force: the fact that so much software exists that's designed to target many similar platforms and is frequently recompiled for those targets means that the impact to the end-consumer of switching among x86/ppc/arm is much lower than it was in the past.