Member Site › Forums › Rosetta 3 › Rosetta 3 – Build/Install › loopmodel mpi build fail
- This topic has 4 replies, 2 voices, and was last updated 9 years, 7 months ago by
Anonymous.
-
AuthorPosts
-
-
May 4, 2015 at 2:48 pm #2191
Anonymous
Dear Rosetta Developers
I’m quite new to Rosetta. Recently, I want to perform a loop modeling using Rosetta. So I want to compile Rosetta-3.5 on our cluster.
[root@c02125 ~]# cat /proc/version
Linux version 2.6.32-220.el6.x86_64 (mockbuild@c6b18n3.bsys.dev.centos.org) (gcc version 4.4.6 20110731 (Red Hat 4.4.6-3) (GCC) ) #1 SMP Tue Dec 6 19:48:22 GMT 2011If I didn’t use ‘extras=mpi’ flag and compile Rosetta using gcc. It worked just fine.
But if I compile Rosetta with flag ‘extras=mpi’, things are not pretty. Error are like this[?1034hscons: Reading SConscript files …
svn_version.py reports this is a non-version-control release version of Rosetta
scons: done reading SConscript files.
scons: Building targets …
mpiCC -o build/src/release/linux/2.6/64/x86/gcc/4.4/mpi/apps/public/AbinitioRelax.o -c -std=c++98 -isystem external/boost_1_46_1/ -pipe -ffor-scope -Wall -Wextra -pedantic -Wno-long-long -O3 -ffast-math -funroll-loops -finline-functions -s -Wno-unused-variable -finline-limit=487 -DNDEBUG -DUSEMPI -Isrc -Iexternal/include -Isrc/platform/linux/64/gcc/4.4 -Isrc/platform/linux/64/gcc -Isrc/platform/linux/64 -Isrc/platform/linux -I/opt/intel/Compiler/11.1/072/mkl/include -Iexternal/boost_1_46_1 -Iexternal/dbio -I/usr/include -I/usr/local/include src/apps/public/AbinitioRelax.cc
icc: command line warning #10159: invalid argument for option ‘-std’
icc: command line warning #10006: ignoring unknown option ‘-Wextra’
icc: command line remark #10148: option ‘-pedantic’ not supported
icc: command line warning #10006: ignoring unknown option ‘-Wno-long-long’
icc: command line warning #10120: overriding ‘-O0’ with ‘-O3’
icc: command line warning #10006: ignoring unknown option ‘-ffast-math’
****
src/protocols/enzdes/EnzdesJobInputter.cc(48): remark #383: value copied to temporary, reference to temporary used
static basic::Tracer TR(“protocols.enzdes.EnzdesJobInputter”);
^src/protocols/enzdes/EnzdesJobInputter.cc(60): remark #981: operands are evaluated in unspecified order
if( !enz_loops_file_->read_loops_file( basic::options::option[ basic::options::OptionKeys::enzdes::enz_loops_file ] ) ){
^src/protocols/enzdes/EnzdesFixBBProtocol.cc(120): remark #981: operands are evaluated in unspecified order
enzdes_pack( pose, design_pack_task, scorefxn_, design_min_cycles, basic::options::option[basic::options::OptionKeys::enzdes::cst_min], false, true );
^/usr/include/c++/4.4.6/cmath(500): error: identifier “__builtin_fpclassify” is undefined
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL,
^
detected during:
instantiation of “__gnu_cxx::__enable_if
instantiation of “size_t={unsigned long} boost::hash_detail::float_hash_value(T) [with T=float]” at line 329 of “external/boost_1_46_1/boost/functional/hash/hash.hpp”/usr/include/c++/4.4.6/cmath(500): error: identifier “__builtin_fpclassify” is undefined
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL,
^
detected during:
instantiation of “__gnu_cxx::__enable_if
instantiation of “size_t={unsigned long} boost::hash_detail::float_hash_value(T) [with T=long double]” at line 339 of “external/boost_1_46_1/boost/functional/hash/hash.hpp”/usr/include/c++/4.4.6/cmath(500): error: identifier “__builtin_fpclassify” is undefined
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL,
^
detected during:
instantiation of “__gnu_cxx::__enable_if
instantiation of “size_t={unsigned long} boost::hash_detail::float_hash_value(T) [with T=float]” at line 329 of “external/boost_1_46_1/boost/functional/hash/hash.hpp”/usr/include/c++/4.4.6/cmath(500): error: identifier “__builtin_fpclassify” is undefined
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL,
^
detected during:
instantiation of “__gnu_cxx::__enable_if
instantiation of “size_t={unsigned long} boost::hash_detail::float_hash_value(T) [with T=long double]” at line 339 of “external/boost_1_46_1/boost/functional/hash/hash.hpp”compilation aborted for src/protocols/enzdes/BackboneSampler.cc (code 2)
scons: *** [build/src/release/linux/2.6/64/x86/gcc/4.4/mpi/protocols/enzdes/BackboneSampler.os] Error 2
src/protocols/enzdes/EnzdesJobOutputter.cc(64): remark #981: operands are evaluated in unspecified order
silent_job_outputter_->set_silent_file_name( utility::file::FileName( basic::options::option[ basic::options::OptionKeys::out::file::o ]) );
^src/protocols/enzdes/EnzdesJobOutputter.cc(81): remark #981: operands are evaluated in unspecified order
silent_job_outputter_->final_pose(job, pose);
^
src/protocols/enzdes/EnzdesFlexBBProtocol.cc(139): remark #383: value copied to temporary, reference to temporary used
if( scorefxn_->has_zero_weight( core::scoring::mm_bend ) ){ scorefxn_->set_weight( core::scoring::mm_bend, 1.0 ); }
^src/protocols/enzdes/EnzdesFlexBBProtocol.cc(139): remark #383: value copied to temporary, reference to temporary used
if( scorefxn_->has_zero_weight( core::scoring::mm_bend ) ){ scorefxn_->set_weight( core::scoring::mm_bend, 1.0 ); }
^src/protocols/enzdes/EnzdesFlexBBProtocol.cc(139): remark #981: operands are evaluated in unspecified order
if( scorefxn_->has_zero_weight( core::scoring::mm_bend ) ){ scorefxn_->set_weight( core::scoring::mm_bend, 1.0 ); }
^/usr/include/c++/4.4.6/cmath(500): error: identifier “__builtin_fpclassify” is undefined
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL,
^
detected during:
instantiation of “__gnu_cxx::__enable_if
instantiation of “size_t={unsigned long} boost::hash_detail::float_hash_value(T) [with T=float]” at line 329 of “external/boost_1_46_1/boost/functional/hash/hash.hpp”/usr/include/c++/4.4.6/cmath(500): error: identifier “__builtin_fpclassify” is undefined
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL,
^
detected during:
instantiation of “__gnu_cxx::__enable_if
instantiation of “size_t={unsigned long} boost::hash_detail::float_hash_value(T) [with T=long double]” at line 339 of “external/boost_1_46_1/boost/functional/hash/hash.hpp”compilation aborted for src/protocols/enzdes/EnzdesJobOutputter.cc (code 2)
scons: *** [build/src/release/linux/2.6/64/x86/gcc/4.4/mpi/protocols/enzdes/EnzdesJobOutputter.os] Error 2
/usr/include/c++/4.4.6/cmath(500): error: identifier “__builtin_fpclassify” is undefined
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL,
^
detected during:
instantiation of “__gnu_cxx::__enable_if
instantiation of “size_t={unsigned long} boost::hash_detail::float_hash_value(T) [with T=float]” at line 329 of “external/boost_1_46_1/boost/functional/hash/hash.hpp”src/protocols/enzdes/EnzdesBaseProtocol.cc(87): remark #383: value copied to temporary, reference to temporary used
static basic::Tracer tr(“protocols.enzdes.EnzdesBaseProtocol”);
^src/protocols/enzdes/EnzdesBaseProtocol.cc(98): remark #383: value copied to temporary, reference to temporary used
Mover::type( “EnzdesFixBBProtocol” );
^src/protocols/enzdes/EnzdesBaseProtocol.cc(99): remark #981: operands are evaluated in unspecified order
restype_set_ = core::chemical::ChemicalManager::get_instance()->residue_type_set( core::chemical::FA_STANDARD ) ;
^src/protocols/enzdes/EnzdesBaseProtocol.cc(113): remark #981: operands are evaluated in unspecified order
reduced_sfxn_ = ScoreFunctionFactory::create_score_function( “enzdes_polyA_min”, option[ OptionKeys::score::patch ]() );
^src/protocols/enzdes/EnzdesBaseProtocol.cc(154): remark #981: operands are evaluated in unspecified order
scorefxn_ = ScoreFunctionFactory::create_score_function( “enzdes”, option[ OptionKeys::score::patch ]() );
^src/protocols/enzdes/EnzdesBaseProtocol.cc(161): remark #383: value copied to temporary, reference to temporary used
if (scorefxn_->has_zero_weight( core::scoring::coordinate_constraint ) ){
^src/protocols/enzdes/EnzdesBaseProtocol.cc(161): remark #981: operands are evaluated in unspecified order
if (scorefxn_->has_zero_weight( core::scoring::coordinate_constraint ) ){
^/usr/include/c++/4.4.6/cmath(500): error: identifier “__builtin_fpclassify” is undefined
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL,
^
detected during:
instantiation of “__gnu_cxx::__enable_if
instantiation of “size_t={unsigned long} boost::hash_detail::float_hash_value(T) [with T=long double]” at line 339 of “external/boost_1_46_1/boost/functional/hash/hash.hpp”src/protocols/enzdes/EnzdesBaseProtocol.cc(167): remark #383: value copied to temporary, reference to temporary used
if (scorefxn_->has_zero_weight( core::scoring::atom_pair_constraint ) ){
^src/protocols/enzdes/EnzdesBaseProtocol.cc(167): remark #981: operands are evaluated in unspecified order
if (scorefxn_->has_zero_weight( core::scoring::atom_pair_constraint ) ){
^compilation aborted for src/protocols/enzdes/EnzdesBaseProtocol.cc (code 2)
scons: *** [build/src/release/linux/2.6/64/x86/gcc/4.4/mpi/protocols/enzdes/EnzdesBaseProtocol.os] Error 2
compilation aborted for src/protocols/enzdes/enzdes_util.cc (code 2)
scons: *** [build/src/release/linux/2.6/64/x86/gcc/4.4/mpi/protocols/enzdes/enzdes_util.os] Error 2
compilation aborted for src/protocols/enzdes/EnzdesFlexBBProtocol.cc (code 2)
scons: *** [build/src/release/linux/2.6/64/x86/gcc/4.4/mpi/protocols/enzdes/EnzdesFlexBBProtocol.os] Error 2
scons: building terminated because of errors.
It seems that something is wrong with ‘icc’, when I tried to compile Rosetta using ‘cxx=icc’ and no ‘extras=mpi’ , it still not workingErrors are like this:
[?1034hscons: Reading SConscript files …
svn_version.py reports this is a non-version-control release version of Rosetta
scons: done reading SConscript files.
scons: Building targets …
icpc -o build/src/release/linux/2.6/64/x86/icc/11.1/default/protocols/init/init.os -c -Wp64 -wd279,2259,1682 -ip -O3 -Qoption,c,-ip_ninl_max_stats=500 -Qoption,c,-ip_ninl_max_total_stats=5000 -fPIC -DNDEBUG -Isrc -Iexternal/include -Isrc/platform/linux/64/icc/11.1 -Isrc/platform/linux/64/icc -Isrc/platform/linux/64 -Isrc/platform/linux -I/opt/intel/Compiler/11.1/072/mkl/include -Iexternal/boost_1_46_1 -Iexternal/dbio -I/usr/include -I/usr/local/include src/protocols/init/init.cc
/usr/include/c++/4.4.6/cmath(500): error: identifier “__builtin_fpclassify” is undefined
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL,
^
detected during:
instantiation of “__gnu_cxx::__enable_if
instantiation of “size_t={unsigned long} boost::hash_detail::float_hash_value(T) [with T=float]” at line 329 of “external/boost_1_46_1/boost/functional/hash/hash.hpp”/usr/include/c++/4.4.6/cmath(500): error: identifier “__builtin_fpclassify” is undefined
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL,
^
detected during:
instantiation of “__gnu_cxx::__enable_if
instantiation of “size_t={unsigned long} boost::hash_detail::float_hash_value(T) [with T=double]” at line 334 of “external/boost_1_46_1/boost/functional/hash/hash.hpp”/usr/include/c++/4.4.6/cmath(500): error: identifier “__builtin_fpclassify” is undefined
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL,
^
detected during:
instantiation of “__gnu_cxx::__enable_if
instantiation of “size_t={unsigned long} boost::hash_detail::float_hash_value(T) [with T=long double]” at line 339 of “external/boost_1_46_1/boost/functional/hash/hash.hpp”compilation aborted for src/protocols/init/init.cc (code 2)
scons: *** [build/src/release/linux/2.6/64/x86/icc/11.1/default/protocols/init/init.os] Error 2
scons: building terminated because of errors.
I don’t know what went wrong here. Maybe something is wrong with configuration or shouldn’t I use the Inter Compiler ? But I tried to use ‘cxx=gcc’ and ‘extras=mpi’ it still not working. Can anyone give me a hand or tip ?Thank you very much
Yours sincerely
Nicky
-
May 4, 2015 at 3:27 pm #10998
Using the Intel compiler should be fine — but you’ll need to tell the scons build system that’s what you’ll be using. “cxx=gcc” says to use the GCC compiler, rather than the Intel one. (GCC is also set as the default on most systems, so it will normally be what you get if you don’t specify anything.) Try using “cxx=icc” instead with your MPI compiles – don’t use “cxx=gcc” with your MPI compile, as your MPI compiler is Intel and not GCC.
The problems you’re running into with the plain icc compiles look to be related to the build system not being able to find and use the ICC-associated libraries. __builtin_fpclassify is a GCC specific function. It’s being called from your system standard libraries, which are set up for GCC. ICC should have its own standard libraries. These might not be in your default path. Often on clusters there is a module system you need to invoke in order to use “non-standard” compilers and packages (something like “module load icc” – as your cluster admin for details.) … If you’re the cluster admin, check your ICC installation to see if you need to add anything to the path to get it to work appropriately, and it might even be worth double checking the ICC installation to make sure if it works on non-Rosetta compiles.
On the Rosetta side, you may need to explicitly tell the scons build system that you’re using the environment-set directories to build. In tools/build/ there’s a “site.settings.killdevil” file which is a good general-purpose setting file. Just copy it over the existing site.settings file to enable. Sometimes that doesn’t work, and you have to get more complicated, for example site.settings.csbmpi which is for a specific MPI system. It all depends on what it takes to get Intel and/or the MPI compiler to compile files.
-
May 5, 2015 at 2:25 am #11000
Hi,rmoretti, thanks for your quick reply. I tried as you told me to check the icc setup on my cluster. I think it works well, check below
[root@c02125 rosetta_source]# icc -V
Intel(R) C Intel(R) 64 Compiler Professional for applications running on Intel(R) 64, Version 11.1 Build 20100414 Package ID: l_cproc_p_11.1.072
Copyright (C) 1985-2010 Intel Corporation. All rights reserved.
[root@c02125 rosetta_source]# echo $PATH
/usr/lib64/qt3.3/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/sbin:/
usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/g09:/usr/local/g09/bsd:/usr/local/g09/local:/usr/local/g09/extras:/
usr/local/g09:/usr/local/bin:/usr/local/amber12/bin:/usr/local/bin:/usr/local/cuda/bin:/root/bin
[root@c02125 rosetta_source]# echo $LD_LIBRARY_PATH
/usr/local/g09/bsd:/usr/local/g09/local:/usr/local/g09/extras:/usr/local/g09:/opt/intel/Compiler/11.1/072/mkl/lib/em64t:
/opt/intel/Compiler/11.1/072/lib/intel64:/opt/intel/Compiler/11.1/072/lib/intel64:/usr/local/g09:/usr/local/gv/lib:/usr/local/lib:
/usr/local/cuda/lib64:/usr/local/cuda/lib
[root@c02125 rosetta_source]# echo $LIBRARY_PATH
/opt/intel/Compiler/11.1/072/mkl/lib/em64t:/opt/intel/Compiler/11.1/072/lib/intel64:/opt/intel/Compiler/11.1/072/lib/intel64:/usr/local/lib
[root@c02125 rosetta_source]# echo $INCLUDE
/usr/include/c++/4.4.6/:/usr/include/:/opt/intel/Compiler/11.1/072/include:/opt/intel/Compiler/11.1/072/mkl/include (I added /usr/include/c++/4.4.6/:/usr/include/:/opt/intel/Compiler/11.1/072/include: to this, before there was only /opt/intel/Compiler/11.1/072/mkl/include )I guess the MPI program works fine, because we got other software like Amber on our cluster and it worked fine. When I tried to use
[root@c02125 rosetta_source]# ./scons.py -j8 mode=release bin cxx=icc extras=mpi > mpi_icc__try 2>&1 &
to compile Rosetta, it still gives me the same error.
/usr/include/c++/4.4.6/cmath(500): error: identifier “__builtin_fpclassify” is undefined
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL,
^
detected during:
instantiation of “__gnu_cxx::__enable_if
instantiation of “size_t={unsigned long} boost::hash_detail::float_hash_value(T) [with T=float]” at line 282 of “/usr/include/boost/functional/hash/hash.hpp”
compilation aborted for src/protocols/enzdes/EnzdesMovers.cc (code 2)
scons: *** [build/src/release/linux/2.6/64/x86/icc/11.1/mpi/protocols/enzdes/EnzdesMovers.os] Error 2
compilation aborted for src/protocols/enzdes/EnzdesFlexBBProtocol.cc (code 2)
scons: *** [build/src/release/linux/2.6/64/x86/icc/11.1/mpi/protocols/enzdes/EnzdesFlexBBProtocol.os] Error 2
scons: building terminated because of errorsI guess scons system still not detect the intel Compiler. So I modified the setting files. I modified several files, include basic.settings/site.setting.csb/site.settings.csbmpi/site.settings.killdevil/user.settings to uncomment #import os and #”program_path” : os.environ[“PATH”].split(“:”) to let scons detect icc. I also manually add the icc library and include path to the basic.settings file. like below
import os
settings = {
“base” : {
“overrides” : {
# SCons starts CXXFLAGS with a reference to CCFLAGS
# We want the two to be separate.
“CCFLAGS” : “”,
“CXXFLAGS” : “”,
# SCons starts with a (short) default PATH.
# We want only what is explicitly specified here.
#”ENV” : {
# “PATH” : [ ]
#},
“program_path” : [
“/bin”,
“/usr/bin”,
“/usr/local/bin”,
],
“include_path” : [
“#external/boost_1_46_1”,
“#external/dbio”,
“/usr/include”,
“/usr/local/include”,
“/opt/intel/Compiler/11.1/072/include”
],
“library_path” : [
“/usr/lib”,
“/usr/local/lib”,
“/opt/intel/Compiler/11.1/072/mkl/lib”,
“/opt/intel/Compiler/11.1/072/lib”
],
},
},But still, I got the same error and compilation failed. I think something is wrong with my scons build setup, because when compiling mpi, I found the compilation can find the path containing the icc head files. Maybe something is wrong when switch the compiler from default gcc to icc. I don’t quite understand the files in tools/build directory. Can you tell me something more about the files and which ones should I modify to get the compilation done ?
Thank you very much
Nicky
-
May 5, 2015 at 9:24 pm #11001
The files in tools/build/ are set up in a hierarchical system. The settings start out with the basic.settings file, which lists the defaults for various platforms/scenarios. These should be generally applicable to all systems. Normally you don’t have to modify these for local system paths.
Next the site.settings file is read. Note that only the site.settings file itself is read. All the site.settings.XXXXXX files are simply examples and are ignored, unless you rename them to be just “site.settings”. The site.settings file has modifications to the basic.settings file, and is intended to customize the build system for a particular computer/cluster.
After that the user.settings file gets a chance to modify the settings. This is intended for individual user modifications. (E.g. there’s a general setting for the cluster you’re using, but you personally want a slightly different build.)
So the results of the build system settings is what happens after all three of those files get read.
###
After a bit of searching, it looks like this particular issue is a know issue with the Intel 11.1 compiler and the 4.4 series of standard libraries for GCC. https://software.intel.com/en-us/forums/topic/291787 Apparently it’s fixed with an updated version of the Intel compiler (version 11.1 Build 20100806 Package ID: l_cproc_p_11.1.073)
Is there any way you can install an updated version of the Intel compiler? If not, I don’t think there’s anything that we’d be able to fix on the Rosetta end.
(BTW I notice your command prompts say “root” – you don’t need to be root to compile Rosetta, and in fact I’d recommend against it. Instead I’d recommend building Rosetta as a regular user, and changing the file permissions later, if necessary. Rosetta keeps itself to its own directories, so doesn’t need access to the system directories to “install”. Any installation to system directories would have to be done manually by yourself later.)
-
May 7, 2015 at 1:28 am #11006
Hi,rmoretti, thank you for your nice reply. I’ll try to update the icc version to see if I can get my problem fixed. I’ll let you know if I get it done. Thanks again !
-
-
AuthorPosts
- You must be logged in to reply to this topic.