All the RMSD calculating functions in Rosetta that I’m aware of will do the alignment and RMSD calculation against the same set of residues.
If you want to align to one subset of residues and then calculate the RMSD for a different subset, then you’ll need to separate the two step. First align (transform) the poses appropriately between the one subset of residues, and then calculate an RMSD without superposition between the other reside subsets.
As Steven says, this will likely be easiest in PyRosetta.