Make error during installation of updated ncbi-blast-2.9.0+-src plus error with nr-database

Member Site › Forums › Rosetta 3 › Rosetta 3 – Applications › Make error during installation of updated ncbi-blast-2.9.0+-src plus error with nr-database

Viewing 2 reply threads
  • Author
    Posts
    • October 1, 2019 at 9:04 am #3278
      Anonymous

        Dear fellows,

        my recent situation with setting up the fragment generation for structure prediction ab initio consists in follow:

        – It seems as if I solved my problems with version compatibility of gcc-compiler and the environment variable FRAGMENT_PICKER.

        – But a problem with installation of updated make_fragments.pl and install_dependencies.pl is still remained unsolved. I carried out next actions with the ncbi-blast-2.9.0+-src package:

        1) wget ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/ncbi-blast-2.9.0+-src.tar.gz.

        2) tar xvzf ncbi-blast-2.9.0+-src.tar.gz.

        3) ./configure at ~/ncbi-blast-2.9.0+-src/c++/ directory.

        4) /usr/bin/make all_r at ~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/build directory.

        I got such errors at the last step:

        “make[3]: Leaving directory ‘~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/build/corelib’

        make[3]: *** [~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/build/Makefile.rules:88: test_boost.o] Error 1

        FAILED: src/corelib/Makefile.test_boost.lib

        make[3]: Entering directory ‘~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/build/corelib’

        /bin/rm -f libtest_boost.a .test_boost.dep .libtest_boost.a.stamp

        /bin/rm -f ~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/lib/libtest_boost.a ~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/status/.test_boost.dep ~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/lib/libtest_boost-static.a ~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/status/.test_boost-static.dep

        make[3]: Leaving directory ‘~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/build/corelib’

        make[2]: *** [~/ncbi-blast-2.9.0+-src/c++/src/build-system/Makefile.meta_l:318: all.nonusr] Error 2

        make[2]: Leaving directory ‘~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/build/corelib’

        make[1]: *** [~/ncbi-blast-2.9.0+-src/c++/src/build-system/Makefile.meta_l:276: all_l.real] Error 2

        make[1]: Leaving directory ‘~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/build/corelib’

        make: *** [~/ncbi-blast-2.9.0+-src/c++/src/build-system/Makefile.meta_r:41: all_r.real] Error 5.”

        I attach a file with a full report about make-command running.

        On the other hand I got such output with nr-databases errors during fragment generation trial:

        “Use of uninitialized value $ENV{“FRAGMENT_PICKER_NUM_CPUS”} in int at ~/rosetta_bin_linux_2019.22.60749_bundle/tools/fragment_tools/make_fragments.pl line 34.

        Use of uninitialized value $ENV{“BLAST_NUM_CPUS”} in int at ~/rosetta_bin_linux_2019.22.60749_bundle/tools/fragment_tools/make_fragments.pl line 40.

        Use of uninitialized value $ENV{“PSIPRED_USE_weights_dat4”} in int at ~/rosetta_bin_linux_2019.22.60749_bundle/tools/fragment_tools/make_fragments.pl line 50.

        Use of uninitialized value $ENV{“SLAVE_LAUNCHER_MAX_JOBS”} in int at ~/rosetta_bin_linux_2019.22.60749_bundle/tools/fragment_tools/make_fragments.pl line 69.

        Use of uninitialized value $ENV{“SLAVE_MAX_WAIT”} in int at ~/rosetta_bin_linux_2019.22.60749_bundle/tools/fragment_tools/make_fragments.pl line 71.

        Use of uninitialized value $ENV{“SLAVE_MAX_ATTEMPTS”} in int at ~/rosetta_bin_linux_2019.22.60749_bundle/tools/fragment_tools/make_fragments.pl line 72.

        Use of uninitialized value $ENV{“PDB2VALL_IGNORE_ERRORS”} in int at ~/rosetta_bin_linux_2019.22.60749_bundle/tools/fragment_tools/make_fragments.pl line 78.

        Dependency ‘nr’ does not exist!

        Running install_dependencies.pl

        Note: the NCBI non-redundant (nr) sequence database is very large.

        Please be patient…..

        Fetching NR database from NCBI. Be very patient ……

        URL transformed to HTTPS due to an HSTS policy

        –2019-10-01 10:57:43– https://www.ncbi.nlm.nih.gov/blast/docs/update_blastdb.pl

        Resolving http://www.ncbi.nlm.nih.gov (www.ncbi.nlm.nih.gov)… 130.14.29.110, 2607:f220:41e:4290::110

        Connecting to http://www.ncbi.nlm.nih.gov (www.ncbi.nlm.nih.gov)|130.14.29.110|:443… connected.

        HTTP request sent, awaiting response… 404 Not Found

        2019-10-01 10:57:44 ERROR 404: Not Found.

        ERROR! wget http://www.ncbi.nlm.nih.gov/blast/docs/update_blastdb.pl failed.

        ERROR! ~/rosetta_bin_linux_2019.22.60749_bundle/tools/fragment_tools/databases/nr does not exist.”

        I will be really grateful for your help and tips in solving these problems with “make” and nr-databases and I am looking forward to it.

        Best regard,

        Corvin.

      • October 1, 2019 at 4:55 pm #14997
        Anonymous

          Hello Corvin

          I recently updated install dependencies to not have this error anymore.  could you try the newest available version of ‘install_dependencies.pl’? (probably located in the newest available rosetta)

          • October 2, 2019 at 10:00 am #14998
            Anonymous

              Dear danpf,

              I have just installed newer version of Rosetta (version 3.11, which was updated on 26st September of this year) and run make_fragments.pl. And I got such error:

              “2019-10-02 12:50:02 (1.81 MB/s) – ‘ncbi-blast-2.9.0+-src.tar.gz.2’ saved [25411110]

              Can’t locate JSON.pm in @INC (you may need to install the JSON module) (@INC contains: /usr/lib/perl5/site_perl/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.28.1 /usr/lib/perl5/vendor_perl/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.28.1 /usr/lib/perl5/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/5.28.1 /usr/lib/perl5/site_perl) at ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/update_blastdb.pl line 43.

              BEGIN failed–compilation aborted at ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/update_blastdb.pl line 43.

              Can’t locate JSON.pm in @INC (you may need to install the JSON module) (@INC contains: /usr/lib/perl5/site_perl/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.28.1 /usr/lib/perl5/vendor_perl/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.28.1 /usr/lib/perl5/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/5.28.1 /usr/lib/perl5/site_perl) at ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/update_blastdb.pl line 43.

              BEGIN failed–compilation aborted at ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/update_blastdb.pl line 43.

              Can’t locate JSON.pm in @INC (you may need to install the JSON module) (@INC contains: /usr/lib/perl5/site_perl/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.28.1 /usr/lib/perl5/vendor_perl/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.28.1 /usr/lib/perl5/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/5.28.1 /usr/lib/perl5/site_perl) at ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/update_blastdb.pl line 43.

              BEGIN failed–compilation aborted at ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/update_blastdb.pl line 43.

              Can’t locate JSON.pm in @INC (you may need to install the JSON module) (@INC contains: /usr/lib/perl5/site_perl/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.28.1 /usr/lib/perl5/vendor_perl/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.28.1 /usr/lib/perl5/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/5.28.1 /usr/lib/perl5/site_perl) at ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/update_blastdb.pl line 43.

              BEGIN failed–compilation aborted at ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/update_blastdb.pl line 43.

              Aborting….

              rm: cannot remove ‘~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/nr.*tar.gz’: No such file or directory

              Aborted!

              ERROR! ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/nr does not exist.”

              I should note, that JSON module were installed at the moment of make_fragments.pl script running. I suppose, that a problem consists in nr-databases. How to fix this problem?

              Thanks in advance,

              Corvin.

            • October 2, 2019 at 11:21 pm #14999
              Anonymous

                hey corvin:

                you have to install some perl stuff to get this script to work (I do so VIA conda, but if you know perl you can do so with an alternative method)

                  


                
# This should be somewhat close to what you should run (might be slightly off)
                
conda create --name make_frags --python=2
                
conda activate make_frags
                
conda config --env --add channels bioconda
                
conda config --env --add channels compbiocore
                
conda install -y perl perl-if perl-parallel-forkmanager perl-switch perl-text-balanced perl-json
                

                
# check for conda env location via conda env list (fill in manually the { } stuff
                
export PERL5LIB=${put where your conda env is}/lib/site_perl/5.26.2/
                
export ROSETTA_DATABASE={rosetta location}/database
                
export VALL={rosetta location}/../tools/fragment_tools/vall.jul19.2011
                
export FRAGMENT_PICKER={rosetta location}/source/bin/fragment_picker.{extension}
                
export FRAGMENT_PICKER_NUM_CPUS=1
                
export BLAST_NUM_CPUS={num_cores_to_use}
                

                
# now I run
                
./install_dependencies.pl standard

                if you know how to install perl modules via the system (like apt-get) then you’re welcome to do it that way, I found it to be very annoying though.

              • October 3, 2019 at 3:04 pm #15000
                Anonymous

                  Dear danpf,

                  I am sincerely thankful for the given algorithm at this response. It works well and installs nr-databases successfully.

                  But during nr-databases downloading I realized, that their overall size (136 packages, each of them contains several hundred MB) could be too big for my laptop SSD with a memory volume 256 GB, on which my Linux is placed. But my laptop also has the HDD with a memory volume 1 TB, on which the Windows10 is installed. I can make files on HDD accessible for Linux. Is Rosetta software moving to HDD, installation of nr-databases here and call the Rosetta scripts (which are placed on HDD) from Linux for their running on Linux a good solution of this problem? Or creation of separated directory with nr-databases on HDD and export their path for their availability for Linux is better idea?

                  Kind regards,

                  Corvin.

              • October 3, 2019 at 4:45 pm #15001
                Anonymous

                  Hey Corvin,

                  The install_dependencies script is unfortunately hardcoded to install the database files wherever ‘install_dependencies.pl’ is located.

                  I would go to your 1TB hdd, make a folder, and copy ‘install_dependencies.pl’ and ‘make_fragments.pl’ to that location, and run them from there.

                  I can’t remember the total size, but it’s approximately 500gb (+-100gb haha) so make sure you have the space

                   

                  ~Dan

                  • October 7, 2019 at 8:20 am #15004
                    Anonymous

                      Dear danpf,

                      my “install_dependencies.pl” script is running correctly. Now it is working on the nr-pfilt data extracting and formatting stage. But it’s looked like, that the overall size of nr-databases will be far more than 600 GB. There is some unneccessary files, which I can remove with no harm for this process? For example, can I delete the text file “nr” in ~/NR-databases/databases (size – 138 GB), while “install_dependencies” works with “nr_pfilt” data extracting and formatting?

                      Many thanks in advance.

                      Kind regards,

                      Corvin.

                • Author
                  Posts
                Viewing 2 reply threads
                • You must be logged in to reply to this topic.