99% success rate in unit test, Will it work?

That’s concerning — the pass rate should be 100% on all public releases, and in our hands on our testing server, it is.  It suggests a system-specific problem that we need to fix.  Which compiler were you using, what is your operating system, and what version of Rosetta are you running?   Also could you run one of the tests that failed and post the output here?  (For example, ./test/run.py –mode=debug conformation_stored_restypes_tests).

I am having a similiar problem. 

I am on a cluster which runs CentOS Linux release 8.2.2004. I am trying to compile Rosetta 3.9 (in order to link with RifDock), mode=release. I thought it might have been a new gcc (9.3) issue so I tried deleting and compiling again with older gcc versions (4.8, 5.5, 8.3.1) but 21-25 unit tests always fail. 

Here is the most recent: Rosetta 3.9 and tests were compiled/run with mode=release  and gcc 8.3.1 

This is the current unit test results: 

---

Unit test summary

---

Total number of tests: 3813

  number tests passed: 3792

  number tests failed: 21

  failed tests:

    protocols.test: LoopsFileIOTest:test_LoopsFileIO_read_JSON_loops_file

    protocols.test: StructureDataTests:test_enzdes_remarks

    protocols.test: StructureDataTests:test_non_peptidic_bonds

    protocols.test: GenericSimulatedAnnealerTests:test_boltzmann

    protocols.test: StructureDataTests:test_move_segments

    protocols.test: AlignResiduesMoverTests:test_align_theozyme

    protocols.test: GenericSimulatedAnnealerTests:test_acceptance

    protocols.test: StructureDataTests:test_delete_segment

    protocols.test: AlignResiduesMoverTests:test_tomponent_cstfile

    protocols.test: ScoreTest:test_Score12

    protocols.test: GenericSimulatedAnnealerTests:test_checkpointing

    protocols.test: StructureDataTests:test_slice

    core.test: conformation_stored_restypes_tests:test_load_pdb_with_pose_copy

    core.test: conformation_stored_restypes_tests:test_load_pdb

    core.test: conformation_stored_restypes_tests:test_load_restype_conf

    core.test: conformation_stored_restypes_tests:test_score_restype_noset

    core.test: conformation_stored_restypes_tests:test_min_restype_noset

    core.test: conformation_stored_restypes_tests:test_serialization

    core.test: conformation_stored_restypes_tests:test_pack_restype_noset

    core.test: conformation_stored_restypes_tests:test_cartmin_restype_noset

    numeric.test: GeneralizedEigenSolverTests:test_4x4_complex_problem

Success rate: 99.4492525570417%

---

End of Unit test summary

Done!

 

Here is the output from the failed tests: 

Running one suite: conformation_stored_restypes_tests

Test suite: conformation_stored_restypes_tests (test/core/conformation/conformation_stored_restypes.cxxtest.hh)

basic.io.database: Database file opened: scoring/score_functions/hbonds/ref2015_params/HBPoly1D.csv

basic.io.database: Database file opened: scoring/score_functions/hbonds/ref2015_params/HBFadeIntervals.csv

basic.io.database: Database file opened: scoring/score_functions/hbonds/ref2015_params/HBEval.csv

basic.io.database: Database file opened: scoring/score_functions/hbonds/ref2015_params/DonStrength.csv

basic.io.database: Database file opened: scoring/score_functions/hbonds/ref2015_params/AccStrength.csv

basic.io.database: Database file opened: scoring/score_functions/rama/fd/all.ramaProb

basic.io.database: Database file opened: scoring/score_functions/rama/fd/prepro.ramaProb

basic.io.database: Database file opened: scoring/score_functions/omega/omega_ppdep.all.txt

basic.io.database: Database file opened: scoring/score_functions/omega/omega_ppdep.gly.txt

basic.io.database: Database file opened: scoring/score_functions/omega/omega_ppdep.pro.txt

basic.io.database: Database file opened: scoring/score_functions/omega/omega_ppdep.valile.txt

basic.io.database: Database file opened: scoring/score_functions/P_AA_pp/P_AA

basic.io.database: Database file opened: scoring/score_functions/P_AA_pp/P_AA_n

basic.io.database: Database file opened: scoring/score_functions/P_AA_pp/shapovalov/10deg/kappa131/a20.prop

basic.io.database: Database file opened: scoring/score_functions/bondlength_bondangle/default-lengths.txt

basic.io.database: Database file opened: scoring/score_functions/bondlength_bondangle/default-angles.txt

basic.io.database: Database file opened: scoring/score_functions/bondlength_bondangle/default-torsions.txt

basic.io.database: Database file opened: scoring/score_functions/bondlength_bondangle/default-improper.txt

basic.io.database: [ WARNING ] Unable to locate database file chemical/residue_type_sets/fa_standard/residue_types/devel/denovo_design/D2I.params

ERROR: Cannot find file ‘devel/denovo_design/D2I.params or /grid/furukawa/home_norepl/dawood/rosetta_src_3_9/main/database/chemical/residue_type_sets/fa_standard/residue_types/devel/denovo_design/D2I.params’

ERROR:: Exit from: src/core/chemical/residue_io.cc line: 212

BACKTRACE:

/grid/furukawa/home_norepl/dawood/rosetta_src_3_9/main/source/build/src/release/linux/4.18/64/x86/gcc/8/default/libutility.so(utility::exit(char const*, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)+0x2cd) [0x15554647240d]

/grid/furukawa/home_norepl/dawood/rosetta_src_3_9/main/source/build/src/release/linux/4.18/64/x86/gcc/8/default/libcore.2.so(core::chemical::read_topology_file(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::weak_ptr<core::chemical::AtomTypeSet const>, std::weak_ptr<core::chemical::ElementSet const>, std::weak_ptr<core::chemical::MMAtomTypeSet const>, std::weak_ptr<core::chemical::orbitals::OrbitalTypeSet const>)+0x440) [0x155547a57090]

/grid/furukawa/home_norepl/dawood/rosetta_src_3_9/main/source/build/src/release/linux/4.18/64/x86/gcc/8/default/libcore.2.so(core::chemical::ResidueTypeSet::add_base_residue_type(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x130) [0x155547af2e40]

./core.test() [0xcd2e61]

./core.test() [0xf0eda2]

./core.test() [0x56438a]

/lib64/libc.so.6(__libc_start_main+0xf3) [0x1555444716a3]

./core.test() [0x56441e]

ERROR: Unhandled Exception caught by cxxtest:

File: src/core/chemical/residue_io.cc:212

[ ERROR ] UtilityExitException

ERROR: Cannot find file ‘devel/denovo_design/D2I.params or /grid/furukawa/home_norepl/dawood/rosetta_src_3_9/main/database/chemical/residue_type_sets/fa_standard/residue_types/devel/denovo_design/D2I.params’

Command line:  ./core.test conformation_stored_restypes_tests –database=/grid/furukawa/home_norepl/dawood/rosetta_src_3_9/main/database –mute core

 

These two also have the same error: 

• Test suite: StructureDataTests (test/protocols/denovo_design/components/StructureDataTests.cxxtest.hh)
• Test suite: GenericSimulatedAnnealerTests (test/protocols/simple_moves/GenericSimulatedAnnealer.cxxtest.hh)

 

And looking in the database folder there is no ‘devel’ folder on this path: main/database/chemical/residue_type_sets/fa_standard/residue_types/

Here are the other 4 failed unit tests: 

Running one suite: GeneralizedEigenSolverTests

Test suite: GeneralizedEigenSolverTests (test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh)

In GeneralizedEigenSolverTests::test_4x4_complex_problem:

./test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh:382: Error: Expected (actual_1.real() == expected_1_real) up to epsilon (0.0000), found (0.4153 != 0.4153)

./test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh:383: Error: Expected (actual_1.imag() == expected_1_imag) up to epsilon (0.0000), found (-0.2293 != -0.2293)

./test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh:384: Error: Expected (actual_2.real() == expected_2_real) up to epsilon (0.0000), found (-0.2579 != -0.2579)

./test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh:385: Error: Expected (actual_2.imag() == expected_2_imag) up to epsilon (0.0000), found (-0.1653 != -0.1653)

./test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh:386: Error: Expected (actual_3.real() == expected_3_real) up to epsilon (0.0000), found (0.5137 != 0.5137)

./test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh:387: Error: Expected (actual_3.imag() == expected_3_imag) up to epsilon (0.0000), found (-0.1274 != -0.1274)

./test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh:388: Error: Expected (actual_4.real() == expected_4_real) up to epsilon (0.0000), found (-0.6303 != -0.6303)

./test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh:389: Error: Expected (actual_4.imag() == expected_4_imag) up to epsilon (0.0000), found (-0.0592 != -0.0592)

./test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh:382: Error: Expected (actual_1.real() == expected_1_real) up to epsilon (0.0000), found (0.4153 != 0.4153)

./test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh:383: Error: Expected (actual_1.imag() == expected_1_imag) up to epsilon (0.0000), found (0.2293 != 0.2293)

./test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh:384: Error: Expected (actual_2.real() == expected_2_real) up to epsilon (0.0000), found (-0.2579 != -0.2579)

./test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh:385: Error: Expected (actual_2.imag() == expected_2_imag) up to epsilon (0.0000), found (0.1653 != 0.1653)

./test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh:386: Error: Expected (actual_3.real() == expected_3_real) up to epsilon (0.0000), found (0.5137 != 0.5137)

./test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh:387: Error: Expected (actual_3.imag() == expected_3_imag) up to epsilon (0.0000), found (0.1274 != 0.1274)

./test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh:388: Error: Expected (actual_4.real() == expected_4_real) up to epsilon (0.0000), found (-0.6303 != -0.6303)

./test/numeric/linear_algebra/GeneralizedEigenSolverTests.cxxtest.hh:389: Error: Expected (actual_4.imag() == expected_4_imag) up to epsilon (0.0000), found (0.0592 != 0.0592)

CXXTEST_ERROR: test_4x4_complex_problem Failed!

Failed 1 of 169 tests

Success rate: 99%

Running one suite: LoopsFileIOTest

Test suite: LoopsFileIOTest (test/protocols/loops/LoopsFileIO.cxxtest.hh)

protocols.loops.LoopsFileIO: [ ERROR ] Error parsing bogus ( line 1 ): [WARNING] DEPRECATED old style extended marker X is used

In LoopsFileIOTest::test_LoopsFileIO_read_JSON_loops_file:

./test/protocols/loops/LoopsFileIO.cxxtest.hh:380: Error: Assertion failed: lfd[ 1 ].skip_rate() == 0.5

./test/protocols/loops/LoopsFileIO.cxxtest.hh:391: Error: Assertion failed: sloops[ 1 ].skip_rate == 0.5

CXXTEST_ERROR: test_LoopsFileIO_read_JSON_loops_file Failed!

Failed 1 of 1673 tests

Success rate: 99%

 

 

Running one suite: AlignResiduesMoverTests

Test suite: AlignResiduesMoverTests (test/protocols/denovo_design/movers/AlignResiduesMoverTests.cxxtest.hh)

ERROR: Cannot open file devel/denovo_design/cat_residues.pdb

ERROR:: Exit from: src/core/io/pdb/build_pose_as_is.cc line: 176

BACKTRACE:

/grid/furukawa/home_norepl/dawood/rosetta_src_3_9/main/source/build/src/release/linux/4.18/64/x86/gcc/8/default/libutility.so(utility::exit(char const*, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)+0x2cd) [0x15554627040d]

/grid/furukawa/home_norepl/dawood/rosetta_src_3_9/main/source/build/src/release/linux/4.18/64/x86/gcc/8/default/libcore.3.so(core::io::pdb::build_pose_from_pdb_as_is(core::pose::Pose&, core::chemical::ResidueTypeSet const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, core::io::StructFileReaderOptions const&)+0x249) [0x1555491ba029]

/grid/furukawa/home_norepl/dawood/rosetta_src_3_9/main/source/build/src/release/linux/4.18/64/x86/gcc/8/default/libcore.3.so(core::io::pdb::build_pose_from_pdb_as_is(core::pose::Pose&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, core::io::StructFileReaderOptions const&)+0x40) [0x1555491ba390]

/grid/furukawa/home_norepl/dawood/rosetta_src_3_9/main/source/build/src/release/linux/4.18/64/x86/gcc/8/default/libcore.3.so(core::io::pdb::build_pose_from_pdb_as_is(core::pose::Pose&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x2a) [0x1555491ba60a]

./protocols.test() [0x59bdc4]

./protocols.test() [0xe6a992]

./protocols.test() [0x58fd5a]

/lib64/libc.so.6(__libc_start_main+0xf3) [0x15554426f6a3]

./protocols.test() [0x58fdee]

ERROR: Unhandled Exception caught by cxxtest:

File: src/core/io/pdb/build_pose_as_is.cc:176

[ ERROR ] UtilityExitException

ERROR: Cannot open file devel/denovo_design/cat_residues.pdb

Command line:  ./protocols.test AlignResiduesMoverTests –database=/grid/furukawa/home_norepl/dawood/rosetta_src_3_9/main/database –mute core

Running one suite: ScoreTest

Test suite: ScoreTest (test/protocols/moves/ScoreTest.cxxtest.hh)

Start All Scoring tests

basic.io.database: Database file opened: scoring/score_functions/PairEPotential/pdb_pair_stats_fine

basic.io.database: Database file opened: scoring/score_functions/hbonds/score12_params/HBPoly1D.csv

basic.io.database: Database file opened: scoring/score_functions/hbonds/score12_params/HBFadeIntervals.csv

basic.io.database: Database file opened: scoring/score_functions/hbonds/score12_params/HBEval.csv

basic.io.database: Database file opened: scoring/score_functions/P_AA_pp/P_AA

basic.io.database: Database file opened: scoring/score_functions/P_AA_pp/P_AA_n

basic.io.database: Database file opened: scoring/score_functions/P_AA_pp/P_AA_pp

basic.io.database: Database file opened: scoring/score_functions/rama/Rama_smooth_dyn.dat_ss_6.4

File: ./test/protocols/moves/ScoreTest.cxxtest.hh Line:82

MoversTest::ScoreMover Files protocols/moves/score12.pdb and protocols/moves/score12.pdb._tmp_ are not equal!

CXXTEST_ERROR: test_AllMovers:ScoreMover Failed!

In ScoreTest::test_Score12:

./test/protocols/moves/ScoreTest.cxxtest.hh:82: Error: Test failed: test_AllMovers

UTracer(protocols/moves/score12.u) line 1 not equal:

 protocols.simple_moves.ScoreMover: E               fa_atr        fa_rep        fa_sol  fa_intra_rep     pro_close       fa_pair   hbond_sr_bb   hbond_lr_bb   hbond_bb_sc      hbond_sc   dslf_ss_dst   dslf_cs_ang   dslf_ss_dih   dslf_ca_dih          rama         omega        fa_dun      p_aa_pp           ref  !=  protocols.simple_moves.ScoreMover: E               fa_atr        fa_rep        fa_sol  fa_intra_rep     pro_close       fa_pair   hbond_sr_bb   hbond_lr_bb   hbond_bb_sc      hbond_sc   dslf_ss_dst   dslf_cs_ang   dslf_ss_dih   dslf_ca_dih          rama         omega        fa_dun       p_aa_pp           ref

old: protocols.simple_moves.ScoreMover: E               fa_atr        fa_rep        fa_sol  fa_intra_rep     pro_close       fa_pair   hbond_sr_bb   hbond_lr_bb   hbond_bb_sc      hbond_sc   dslf_ss_dst   dslf_cs_ang   dslf_ss_dih   dslf_ca_dih          rama         omega        fa_dun       p_aa_pp           ref

new: protocols.simple_moves.ScoreMover: E               fa_atr        fa_rep        fa_sol  fa_intra_rep     pro_close       fa_pair   hbond_sr_bb   hbond_lr_bb   hbond_bb_sc      hbond_sc   dslf_ss_dst   dslf_cs_ang   dslf_ss_dih   dslf_ca_dih          rama         omega        fa_dun       p_aa_pp           ref./test/UTracer.hh:100: Error: Test failed: ” x1B[0mprotocols.simple_moves.ScoreMover: x1B[0mE               fa_atr        fa_rep        fa_sol  fa_intra_rep     pro_close       fa_pair   hbond_sr_bb   hbond_lr_bb   hbond_bb_sc      hbond_sc   dslf_ss_dst   dslf_cs_ang   dslf_ss_dih   dslf_ca_dih          rama         omega        fa_dun       p_aa_pp           ref  !=  protocols.simple_moves.ScoreMover: E               fa_atr        fa_rep        fa_sol  fa_intra_rep     pro_close       fa_pair   hbond_sr_bb   hbond_lr_bb   hbond_bb_sc      hbond_sc   dslf_ss_dst   dslf_cs_ang   dslf_ss_dih   dslf_ca_dih          rama         omega        fa_dun       p_aa_pp          ref “

CXXTEST_ERROR: test_Score12 Failed!

Failed 1 of 1673 tests

Success rate: 99%

 

Would really appreciate your help! Thanks in advance!