The CPUs are close, but the Indy is otherwise pretty different from the N64. Totally different graphics architecture, and - relevant to getting it on MiSTer - it’s a workstation rather than a video game console, necessitating quite a bit more complexity. I’d be really surprised if it could be squeezed on.
(Though, full disclosure, I said the same thing about the N64 before the core for it came out - the folks working on MiSTer are incredible.)
To my knowledge - and I'm not an expert here - the N64 hardware is pretty unique and doesn't really resemble any of SGI's other chipsets. Not in precise capabilities - the XZ, for instance, didn't even support hardware texture mapping - and not in overall technical design.
It does seem a little bit like an ultra-simplified, integrated version of the RealityEngine [0]. The RealityEngine had "6, 8, or 12 Geometry Engines" split out across three to six boards, each powered by an Intel i860XP, that then passed their work along to Fragment Generators. This roughly corresponds to the RSP, which was just another MIPS core (with vector/matrix extensions), passing its work along to the RDP on the N64. I'm not sure how programmable the RealityEngine's pipeline was compared to the surprisingly flexible RSP.
Remember, the constraints for a graphics workstation are really different than for a game console - especially on the low-end, totally different corners are going to be cut. An Indy still needed to be able to generate a high resolution display and allow modelling complex scenes for film and TV; but while some degree of real-time 3D was important, it was expected that artists could be modelling using wireframe or simplified displays. A game console was displaying low-resolution and relatively low-detail scenes, but they still wanted them to look aesthetically "complete" - shading, textures, fog, lighting, particles - while running at real-time speeds. SGI used their expertise and built something custom-fit for the job at hand, rather than just reusing an existing solution.
Nay, the N64 is pretty unique hardware-wise. Conceptually it's vaguely similar to the O2, the RCP is an R4000 fixed function CPU with some extra graphics instructions IIRC.
I'd love an Onyx/RE on an FPGA someday. Next to my FPGA cray.