I don’t see why it would be a major problem.
Rosetta (and Phenix) do need to know about the chemical structure of the ligand in order to actually model it, so you’d basically be modeling the apo structure into the density of the holo structure.
So as long as you’re okay with having unexplained density in the binding pocket, it should be fine. This is actually the approach that people often use with ligand-bound structures, at least in the early stages. You model just the protein, then figure out where the excess density is with a difference map, place the ligands into those locations, and then refine further.
Certainly there’d be inaccuracies in modeling, given that there’s *something* present and you’re modeling it as a void/water, but if the rest of the density is clear enough the effect of missing a small ligand should be minor. (If you can add the ligand back in for subsequent rounds, that’d be ideal, but not strictly necessary from a technical perspective – reviewers might disagree about the scientific necessity …)
.
