Member Site Forums Rosetta 3 Rosetta 3 – Build/Install mpi building error

  • This topic has 12 replies, 4 voices, and was last updated 11 years ago by Anonymous.
Viewing 3 reply threads
  • Author
    Posts
    • #1537
      Anonymous

        Hey everyone, I’m trying to compile in mpi mode on a cluster. I used:
        ./scons.py bin mode=release cxx=icc extras=mpi

        First, everything looked good but almost after 3 hours latter i had an error;


        src/protocols/features/ReportToDB.cc(107): warning #780: using-declaration ignored -- it refers to the current namespace
        using protocols::features::ProteinRMSDFeatures;
        ^
        src/protocols/features/ReportToDB.cc(108): warning #780: using-declaration ignored -- it refers to the current namespace
        using protocols::features::ProtocolFeatures;
        ^
        src/protocols/features/ReportToDB.cc(109): warning #780: using-declaration ignored -- it refers to the current namespace
        using protocols::features::StructureFeatures;
        ^
        src/protocols/features/ReportToDB.cc(110): warning #780: using-declaration ignored -- it refers to the current namespace
        using protocols::features::FeaturesReporterFactory;
        ^
        (0): internal error: backend signals
        compilation aborted for src/protocols/features/ReportToDB.cc (code 4)
        scons: *** [build/src/release/linux/2.6/64/x86/gcc/4.1/mpi/protocols/features/ReportToDB.os] Error 4

        Can you help me a bout this error?

      • #8514
        Anonymous

          A) Most of those are warnings (warning #780: using-declaration ignored) – those are not causing the problem (at least, they almost certainly aren’t).

          B) This is *really* weird: ” scons: *** [build/src/release/linux/2.6/64/x86/gcc/4.1/mpi/protocols/features/ReportToDB.os] Error 4″ – notice that’s a gcc path, not an icc path.

          C) “internal error: backend signals: appears to be an internal ICC problem…it doesn’t tell us what code ICC has a problem with, so it will be challenging to help you fix it.

          Can you verify that scons is submitting icc build commands, not gcc? (the compile commands should start with something icc-like instead of gcc-like)? Can you confirm that your MPI libraries are icc MPI libraries, not gcc MPI libraries? I’ve seen problems with the latter, when icc is installed but the system points to gcc mpi libraries in mpi mode.

          Does this duplicate in the non-MPI compile? Does it duplicate in the debug compile? Google suggests it is optimization related (release-mode related).

        • #8648
          Anonymous

            Hi, I am also having an MPI building error, but of a different sort. Honestly, no idea what it means. The last bit of output is posted below. Everything seemed to be going okay up until this point. The non-mpi build is working fine, though. Any ideas on how to fix this would be appreciated. I’m running it on OS X 10.7.


            src/utility/io/ozstream.cc:42:111: required from here
            src/utility/io/mpistream.hh:166:5: error: 'init' was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
            src/utility/io/mpistream.hh:166:5: note: declarations in dependent base 'std::basic_ios' are not found by unqualified lookup
            src/utility/io/mpistream.hh:166:5: note: use 'this->init' instead
            In file included from src/utility/io/mpistream.hh:338:0,
            from src/utility/io/ozstream.hh:35,
            from src/utility/io/ozstream.cc:19:
            src/utility/io/mpistream.ipp: In instantiation of 'utility::io::mpi_stream::basic_mpi_streambuf::basic_mpi_streambuf(std::string, size_t, int, bool) [with Elem = char; Tr = std::char_traits
            ; ElemA = std::allocator; ByteT = unsigned char; ByteAT = std::allocator; std::string = std::basic_string; size_t = long unsigned int]':
            src/utility/io/mpistream.hh:164:56: required from 'utility::io::mpi_stream::basic_mpi_ostreambase::basic_mpi_ostreambase(std::string, size_t, int, bool) [with Elem = char; Tr = std::char_traits
            ; ElemA = std::allocator; ByteT = unsigned char; ByteAT = std::allocator; std::string = std::basic_string; size_t = long unsigned int]'
            src/utility/io/mpistream.hh:258:35: required from 'utility::io::mpi_stream::basic_mpi_ostream::basic_mpi_ostream(std::string, int, std::stringstream&, bool, size_t) [with Elem = char; Tr = std::char_traits
            ; ElemA = std::allocator; ByteT = unsigned char; ByteAT = std::allocator; std::string = std::basic_string; std::stringstream = std::basic_stringstream; size_t = long unsigned int]'
            src/utility/io/ozstream.cc:42:111: required from here
            src/utility/io/mpistream.ipp:41:3: error: 'setp' was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
            src/utility/io/mpistream.ipp:41:3: note: declarations in dependent base 'std::basic_streambuf
            ' are not found by unqualified lookup
            src/utility/io/mpistream.ipp:41:3: note: use 'this->setp' instead
            mpiCC -o build/src/release/macos/11.4/64/x86/gcc/mpi/utility/sql_database/DatabaseSessionManager.os -c -isystem external/boost_1_46_1/boost/ -isystem external/boost_1_46_1/boost/ -m64 -march=nocona -mtune=generic -O3 -ffast-math -funroll-loops -finline-functions -finline-limit=20000 -s -Wno-unused-variable -fPIC -DNDEBUG -DUSEMPI -Isrc -Iexternal/include -Isrc/platform/macos/64/gcc -Isrc/platform/macos/64 -Isrc/platform/macos -Iexternal/boost_1_46_1 -Iexternal/dbio -I/usr/local/include -I/usr/include src/utility/sql_database/DatabaseSessionManager.cc
            scons: *** [build/src/release/macos/11.4/64/x86/gcc/mpi/utility/io/ozstream.os] Error 1
            scons: building terminated because of errors.

          • #8685
            Anonymous

              Thanks for your help on this. I will try using 3.4 and building that way. In the meantime, I realized that at least for running Rosetta on my personal computer I could (and probably should) be using OpenMP, which has a secret hidden “extras” build command that is called with

              extras=omp

              This worked fine for me. I'll repost if 3.4 doesn't fix my problems with MPI.

            • #8651
              Anonymous

                I don’t have any ideas either. What MPI package did you install? What libz package did you install? I notice it is failing in the izstream/ozstream code, which is where Rosetta depends externally on a gzip library (on Ubuntu, it’s zlib1g-dev, but I have no idea what it is for a mac.)

                You are attempting to use gcc not icc, right?

              • #8654
                Anonymous

                  I am using open MPI and have not modified my libz at all. OS X comes with zlib preinstalled. I’m using the newest GNU gcc (4.7.1). Maybe there’s something wrong with the zlib library and I should reinstall. Have you encountered people using the MPI Rosetta on OS X previously?

                • #8656
                  Anonymous

                    Sure, there are several developers that use it (whose attention I have tried to draw)…I’m sure it’s a question of some minor incompatibility. I’m not sure any developers are using 4.7 yet.

                  • #8657
                    Anonymous

                      Thanks. I also tried to build with an earlier gcc and that also failed (can’t remember if it was the same error, though).

                    • #8658
                      Anonymous

                        Hi, can you post the exact scons command you are running? In the meantime you can try building with clang. To do this for an MPI build first set the environment variable ‘OMPI_CXX’ to be set to ‘clang++’, then run scons with:

                        ./scons.py -j*num_proc* bin extras=mpi cxx=clang

                        If you try the clang approach and it doesn’t work, please post the error message (clang tends to have much better error messages than gcc).

                      • #8659
                        Anonymous

                          Thanks for the suggestion. See the following result:

                          ####################:rosetta_source Bronco$ export OMPI_CXX=clang++
                          ####################:rosetta_source Bronco$ echo $OMPI_CXX
                          clang++
                          ####################:rosetta_source Bronco$ scons bin mode=release extras=mpi cxx=clang -j4
                          scons: Reading SConscript files ...
                          Traceback (most recent call last):
                          File "/Users/Bronco/Downloads/rosetta3.3_bundles/rosetta3.3_bundles/rosetta3.3_source/rosetta_source/SConstruct", line 139, in main
                          build = SConscript("tools/build/setup.py")
                          File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/SCons/Script/SConscript.py", line 614, in __call__
                          return method(*args, **kw)
                          File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/SCons/Script/SConscript.py", line 551, in SConscript
                          return _SConscript(self.fs, *files, **subst_kw)
                          File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/SCons/Script/SConscript.py", line 260, in _SConscript
                          exec _file_ in call_stack[-1].globals
                          File "/Users/Bronco/Downloads/rosetta3.3_bundles/rosetta3.3_bundles/rosetta3.3_source/rosetta_source/tools/build/setup.py", line 411, in
                          build = setup()
                          File "/Users/Bronco/Downloads/rosetta3.3_bundles/rosetta3.3_bundles/rosetta3.3_source/rosetta_source/tools/build/setup.py", line 402, in setup
                          build.options_requested, build.options = setup_build_options()
                          File "/Users/Bronco/Downloads/rosetta3.3_bundles/rosetta3.3_bundles/rosetta3.3_source/rosetta_source/tools/build/setup.py", line 108, in setup_build_options
                          supported, actual.cxx, requested.cxx_ver
                          File "/Users/Bronco/Downloads/rosetta3.3_bundles/rosetta3.3_bundles/rosetta3.3_source/rosetta_source/tools/build/setup_platforms.py", line 83, in select_compiler_version
                          (actual, compiler)
                          KeyError: "Unknown version number 4.1 for compiler 'clang'"
                          scons: done reading SConscript files.
                          scons: Building targets ...
                          scons: `bin' is up to date.
                          scons: done building targets.
                          ####################:rosetta_source Bronco$ clang --version
                          Apple clang version 4.1 (tags/Apple/clang-421.11.66) (based on LLVM 3.1svn)
                          Target: x86_64-apple-darwin11.4.2
                          Thread model: posix

                        • #8660
                          Anonymous

                            Apple clang version 4.1 (tags/Apple/clang-421.11.66) (based on LLVM 3.1svn)

                            Clang is at version 3.1, but apple has incremented it to 4.1 for some reason. We can trick SCons into treating faux 4.1 as 3.1. In tools/build/options.settings, add 4.1 as a choice. (I know there is a * there but it doesn’t work).

                          • #8661
                            Anonymous

                              Wow, that’s an amazingly weird choice made by apple to change the version number on us. Thanks, your trick solved the clang problem. Here’s the new error, though:


                              In file included from src/core/io/silent/SilentFileData.cc:50:
                              src/numeric/random/reservoir_sample.hh:43:19: error: reference to non-static member function must be called; did you mean to call it with no
                              arguments?
                              if ( n_vals() < n_wanted ) {
                              ^~~~~~~~
                              ()
                              1 error generated.
                              clang++ -o build/src/release/macos/11.4/64/x86/clang/mpi/core/io/silent/BinaryRNASilentStruct.os -c -O3 -fPIC -DNDEBUG -Isrc -Iexternal/include -Isrc/platform/macos/64/clang -Isrc/platform/macos/64 -Isrc/platform/macos -Iexternal/boost_1_46_1 -Iexternal/dbio -I/usr/local/include -I/usr/include src/core/io/silent/BinaryRNASilentStruct.cc
                              scons: *** [build/src/release/macos/11.4/64/x86/clang/mpi/core/io/silent/SilentFileData.os] Error 1
                              In file included from src/core/io/silent/BinaryRNASilentStruct.cc:49:
                              src/core/io/silent/Binary_Util.hh:59:14: warning: equality comparison with extraneous parentheses [-Wparentheses-equality]
                              if( ( _8bit == '+') ) return 62;
                              ~~~~~~^~~~~~
                              src/core/io/silent/Binary_Util.hh:59:14: note: remove extraneous parentheses around the comparison to silence this warning
                              if( ( _8bit == '+') ) return 62;
                              ~ ^ ~
                              src/core/io/silent/Binary_Util.hh:59:14: note: use '=' to turn this equality comparison into an assignment
                              if( ( _8bit == '+') ) return 62;
                              ^~
                              =
                              In file included from src/core/io/silent/BinaryProteinSilentStruct.cc:44:
                              src/core/io/silent/Binary_Util.hh:59:14: warning: equality comparison with extraneous parentheses [-Wparentheses-equality]
                              if( ( _8bit == '+') ) return 62;
                              ~~~~~~^~~~~~
                              src/core/io/silent/Binary_Util.hh:59:14: note: remove extraneous parentheses around the comparison to silence this warning
                              if( ( _8bit == '+') ) return 62;
                              ~ ^ ~
                              src/core/io/silent/Binary_Util.hh:59:14: note: use '=' to turn this equality comparison into an assignment
                              if( ( _8bit == '+') ) return 62;
                              ^~
                              =
                              1 warning generated.
                              1 warning generated.
                              scons: building terminated because of errors.

                            • #8666
                              Anonymous

                                I get no errors compiling Rosetta 3.3 via clang 3.0; I don’t have 3.1 handy to test.

                                I’m not sure MPI and clang work together with 3.3 – actually I’m pretty sure they don’t; you don’t have a USEMPI in your build command, nor an MPI-labeled compiler, so I think we aren’t getting the MPI compile anyway. I’m looking into how to fix that.

                                Is there a particular reason you are using Rosetta 3.3 over 3.4?

                            Viewing 3 reply threads
                            • You must be logged in to reply to this topic.