score and relax

What is your ligand/cofactor? First thing you will want to do is look at the file rosetta_database/chemical/residue_type_sets/fa_standard/residue_types.txt
Not all of these params are on by default to keep memory low. If you luckily see yours, you can uncomment (delete the #) the line, and it will load the param the next time Rosetta runs.

If not you will want to generate new params. Take a look at this page for some help in this: http://www.pyrosetta.org/obtaining-and-preparing-ligand-pdb-files (The script they are talking about here is in (rosetta_source/src/python/apps/public)
To generate the mol2 file I used http://davapc1.bioch.dundee.ac.uk/prodrg/ Download the all H’s mol2 (SYBYL2) file.

For running molfile_to_params.py, you will want to look at the options in the file (python molfile_to_params.py -help). You will probably want to use the options -n name_of_ligand, and –keep-names to make sure your atom names match in the resultant PDB file.

You will want to do this for both your ligand and cofactor if they are not already in the Rosetta params database.

To actually run the relax, you will want to edit the energy function weights file to switch the statistical electrostatic potential to the coulombic potential (known as hack_elec for now). Doing that is covered elsewhere on the forums: https://www.rosettacommons.org/search/node/hack_elec

Depending on your purposes, you may also want to think about adding ligand/cofactor rotamers. See the rosetta demo ligand_dock/ligand_setup for instructions on how to set this up.

Additionally, you can pass the -ignore_unrecognized_res to flatout ignore the ligand + cofactor. However, Rosetta will pretend it is not there, which could mess up the relax results significantly.