There isn’t an off-the shelf method. You could potentially put something together using the core::scoring::rms_at_corresponding_atoms_no_super() function, but that’d be a little involved. You could also attempt to do something like create a params file for just the core atoms, and then have Rosetta use that instead of the normal ligand params file. Rosetta would discard the extra atoms and if you did your rmsd comparision would only be over the atoms in your reduced params file. This is a little kludgy, though.
It would probably be easier to use pymol to do it. Pymol is scriptable, so even if you have a large number of structures it shouldn’t be too painful.
.
