Rosetta3 will ignore your heme group (or just crash) when you try to read a PDB with a heme. To get it to read the heme group, you need to make a “parameters file” for the ligand. Look in the documentation for ligand-related stuff; you should start here:
src/python/apps/molfile_to_params.py
This program will write out a parameters file for your ligand. Look up the flag extra_res_fa as well (it’s used to include the new parameter file in the set of residuetypes considered when your PDB is read in).
If all you want is the ligand present, then having the parameter file is probably sufficient. If you want ligand movement then try poking around the ligand docking documentation.