Rosetta Dock with MPI / PBS

[Anonymous]

Hello, again.

I have another (hopefully simple) question regarding rosetta, specifically RosettaDock.

I have compiled rosetta3.4 with MPI support (thanks smlewis!), and am trying to run a test docking job to ensure it is working. This cluster uses PBS as the scheduler.

My PBS script is as follows:

---

#!/bin/sh
#PBS -l select=2:mpiprocs=4:ncpus=4
#PBS -q nopreempt
#PBS -l walltime=1:00:00
#PBS -j oe

module load mpich2-intel/1.3

cd $PBS_O_WORKDIR

rbin=/home/eep3m/rosetta_source/bin
rdb=/home/eep3m/rosetta_database
pdb=1brs.pdb
outfile=scores_mpi
flags=”-s $pdb -dock_pert 3 8 -spin -ex1 -ex2aro -nstruct 25 -mute core.util.prof -out:file:scorefile $outfile -out:file:fullatom -mute core.io.database”

mpiexec -np 8 $rbin/docking_protocol.mpi.linuxiccrelease $flags –database $rdb/ > rosetta_output

---

When I submit this to the queue, I get the following error in return:

---

Job is running on node(s):

---

lc5-compute-3-43 lc5-compute-4-45

---

ERROR: Option matching -p4wd not found in command line top-level context
mpiexec: Warning: task 0 exited before completing MPI startup.
mpiexec: Warning: task 1 was never spawned due to earlier errors.

---

I have looked for the past few hours, and have not seen any instance of the flag -p4wd anywhere. I also tried running:

mpiexec -np 8 $rbin/docking_protocol.mpi.linuxiccrelease -help

and get the same error.

I am forgetting / overlooking something simple?

Thanks again for all of your help!
–Eddie

[Anonymous]

Apparently PBS (or mpiexec) is injecting -p4wd into Rosetta’s options list. The ERROR line is the standard message for non-existent options.

Very brief googling suggests that PBS/mpiexec/MPI are not playing well together (in other words, it’s probably not something I can fix).

Here’s one thing to try: try running docking_protocol.mpi.linuxiccrelease with no options and no mpiexec. You should get some sort of error about an uninitialized MPI (I think).

Also try this: try mpirun instead of mpiexec. I don’t know the difference, but I do know that I always use the former and not the latter. (I don’t know if this is even possible or sensible to suggest).

[Anonymous]

Thanks for the quick reply (again).

Your statement about mpiexec injecting the -p4wd flag into the options list made me double check that the MPI libraries were correct. They weren’t. I had to add one additional flag and now everything is playing nice, and running smoothly.

In the event that this may help someone in the future (although I think it is pretty specific to this cluster) here is the mpiexec command that worked:

mpiexec -np 8 -comm mpich2-pmi $rbin/docking_protocol.mpi.linuxiccrelease $flags –database $rdb/ > rosetta_output

Thanks again for all of your help. As someone relativly new to Rosetta, this is a fantastic resource.

[Anonymous]

Just for comparison, my job submissions look something like this:

bsub -q day -n 10 -J $cwd -o log.%J -B -N -a mvapich mpirun /path/to/rosetta/bin/mpi_msd.mpi.linuxgccrelease @options

So, yeah, different formats for different clusters.

[Author]

Posts