Re: [Outreachy] predictprotein and other RostLab packages
Good morning Tanya,
thanks for your diligent and educated work on the topic. I'm impressed
about your performance.
On Wed, Jun 22, 2016 at 09:02:26AM +0300, merlettaia wrote:
> First part of this letter is just to inform you, but I also have several
> questions about packaging with perl for RostLab packages (questions are
> given at the end of this letter).
>
> 0) First of all, this is what I did for now:
> * Concavity,
This was just uploaded.
> Conservation-code,
I've build this and tried the two tests. The second one looks suspicious:
sh non-default-params-test
Checking alignment matrix file usage
Could not load similarity matrix: /usr/share/conservation-code/matrix/blosum63.bla. Using identity matrix...
Checking distribution file usage
[Errno 2] No such file or directory: '/usr/share/conservation-code/distributions/swissprot1.distribution' Using default (BLOSUM62) background.
I realised that there is no such matrix blosum63.bla even in the source
package (only 62). Similarliy with swissprot1.distribution - we just
have swissprot.distribution (without "1"). Do you have any explanation
for this?
> Dssp - these packages you already saw.
Was uploaded yesterday. I simply tried to go through all your commits
step by step to decrease my backlog from vaccation. :-)
> * Tm-align - I made 1 test for both programs in it (TMscore and TMalign),
> added gzipped example files (there should be no related lintian warnings).
Nice. I've just uploaded. Please notice the slight change in the
sequence of mentioning files in d/copyright. The DEP5 format specifies
that you first specify the global copyright (in Files: debian/*) and
than the exceptions (in Files: debian/something). You get a lintian
information if you use
lintian -I
I'm using the following configuration to get this:
$ cat ~/.lintianrc
color=always
display-experimental=no
display-info=yes
pedantic=no
## show-overrides=yes
> * predictnls - made 1 test,
Hmmm, it results in
$ sh /usr/share/doc/predictnls/installation-test
Can't locate Config/IniFiles.pm in @INC (you may need to install the Config::IniFiles module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.20.2 /usr/local/share/perl/5.20.2 /usr/lib/x86_64-linux-gnu/perl5/5.20 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.20 /usr/share/perl/5.20 /usr/local/lib/site_perl .) at /usr/bin/predictnls line 28.
BEGIN failed--compilation aborted at /usr/bin/predictnls line 28.
I've fixed this by adding a
Depends: libconfig-inifiles-perl
I'm quite happy about this kind of test since it uncovers a packaging
problem.
> * profbval - made 1 test,
Just uploaded.
> * pdb2pqr - added 3 tests, fixed lintian warnings.
At first I need to say that I'm very happy about your engaged work on
the packages your are touching. It is a proof of deeper understanding
of Debian packaging. Your changelog regarding non-test issues is
impressing and very appreciated. I wonder whether you are interested in
staying in the Debian Med team even after this outreachy project and to
apply as Debian Maintained. I would definitely support this intention.
> This package contains a
> big pipeline, in some cases it uses different subprograms, and some files
> were not installed when necessary - I fixed this. That's the reason why
> there are 3 separate tests. I couldn't utilize upstream tests/, because
> pdb2pqr uses complex built system (known as `scons`), and I haven't figured
> out how to do it yet. I plan to add them only to autopkgtest, because my
> tests check the same functionality as upstream tests, but they are
> simplier. Some upstream examples contains broken paths, I won't copy to
> examples them either. Except that these tests are is ok.
I consider it a sensible strategy to provide other tests than those that
are done at build time if it turns out as to complex for the moment.
Your tests are looking sensible and we can keep on working on even more
fine grained tests later if necessary.
$ sh pdb2pka-test
Run pdb2pqr...
--------------------------
PDB2PQR - a Python-based structural conversion utility
--------------------------
...
Updating SS bridges...
PATCH INFO: CYS A 2 patched with CYX
CYS A 2 - CYS A 12
PATCH INFO: CYS A 12 patched with CYX
CYS A 12 - CYS A 2
Done.
Checking if we must debump any residues...
Done.
Running PDB2PKA and applying at pH 7.00...
Traceback (most recent call last):
File "/usr/bin/pdb2pqr", line 62, in <module>
mainCommand(sys.argv)
File "/usr/share/pdb2pqr/main.py", line 740, in mainCommand
include_old_header = options.include_header)
File "/usr/share/pdb2pqr/main.py", line 341, in runPDB2PQR
myRoutines.runPDB2PKA(ph, ff, pdblist, ligand, verbose, ph_calc_options)
File "/usr/share/pdb2pqr/src/routines.py", line 1503, in runPDB2PKA
import pka
File "/usr/share/pdb2pqr/pka.py", line 24, in <module>
from pdb2pka.pka_routines import smooth
File "/usr/share/pdb2pqr/pdb2pka/pka_routines.py", line 16, in <module>
import pMC_mult
File "/usr/share/pdb2pqr/pdb2pka/pMC_mult.py", line 93, in <module>
class SwigPyIterator(_object):
File "/usr/share/pdb2pqr/pdb2pka/pMC_mult.py", line 102, in SwigPyIterator
__swig_destroy__ = _pMC_mult.delete_SwigPyIterator
AttributeError: 'module' object has no attribute 'delete_SwigPyIterator'
This smells like a missing Dependency. Could you please investigate
which package might provide this?
> * proftmb - 1 test.
Just uploaded after cosmetic changes.
> There are two serious reasons why I decided to slightly change my plan and
> for now to add tests for most part (or as much as possible) of RostLab's
> packages (and to shift in my timeline pymol/jmol, garlic and r-cran-bio3d
> and to work on them after finishing these packages):
I'm prefectly fine if you manage your personal timeline.
> 1. I planned to work on predictprotein package, but realized that it's a
> big pipeline, and it uses several other RostLab packages, and some of them
> depend on others. When I made simple test, it shown the same deprecation
> error as described here: http://www.perlmonks.org/?node=1077762
> In my case, deprecation error came from the predictprotein's dependency -
> librg-utils-perl. I noticed this closed bug #789410, but it seems that
> package still doesn't work well with perl 5.22 (I'll fix this problem after
> finishing this letter).
> (There is another problem with default database paths, but I'll fix it
> after finishing tests to packages predictprotein relies on).
Sounds verxy sensible. Thanks for your obviously educated take on this
topic.
> 2. When I worked on profbval, there was a related package from the RostLab
> named 'profphd'. After problems with predictprotein tests, I became
> curious, and it turned out that it uses old version of perl (<=5.16). I
> installed it with perlbrew, but it still didn't work well (I'll provide
> some details if necessary in next letter).
I'd recommend to ask on debian-perl list which is very responsive and
helpful in case of any problems. I personally can't be very helpful
with Perl issues since my knowledge here is exsiting but rudimentary.
> I have seen that some of these packages has bugs - I'll try to fix them one
> by one after I'll do tests for corresponding packages.
Very cool! See above my suggestion to become DM.
> ____________________
> Questions:
>
> 1. To check how `librg-utils-perl` works, I used Debian Perl Group's
> default perl packaging testsuite (
> http://pkg-perl.alioth.debian.org/autopkgtest.html).
> Is it fine if I use their recipe for perl libraries' testsuites - to add
> "Testsuite: autopkgtest-pkg-perl" to debian/control and use it instead of
> hand-written tests? At least in simple cases. For 'librg-utils-perl' I'll
> use this test for now.
>
> (In fact I haven't used Perl before and for now I think that Debian Perl
> Group's testsuite will find more problems, than mine).
Yes, please. The Debian Perl Group is to my perception one of the
highly organised group striving for perfection and using what they
use is definitely a good idea. We even moved some of our packages
into their maintenance.
> 2. What is the proper way to add tests for profphd? I should run it with
> perl <=5.16, or to make it work in current unstable versions of perl?
IMHO, we are always targeting unstable. I have no idea what
implications this might have for older Perl versions. May be
asking on the Perl Team list could give better advise.
> 3. There is also a stupid question about a package named
> 'pp-popularity-contest'. It knocks to rostlab server and sends simplest
> possible usage report. In fact, testsuite for this package can contain
> only 1 'control' file with 1 test command, with 'isolation-container'
> restriction.
> This package also starts cron task, but I'm not sure how to check that task
> correct in proper way.
> Debian Med Group Policy states that 'Packages of priority “extra” are
> excluded from some QA tests.', and this package has priority 'extra' -
> maybe I should skip it, or add just simple test with test command, or
> should I also check related cron job - what strategy would you suggest (and
> how to check cron job correctness properly)?
Feel free to ignore pp-popularity-contest. I consider the output of the
rostlab packages to install this package a bit invasive to the user.
The package was invented to get some response about the usage of the
package to the authors. I understand their motivation but default
Debian popularity-contest gives the option to opt-out (actually if I
remember you need to opt-in if you want to report - I hope every Debian
Med user says yes here). So my feelings about this package and its
usage are a bit mixed but it definitely does not need any test since its
no biological application.
Again, thanks a lot for your work. Its exactly the way I was imaging
when starting the GSoC project. Just keep on the good work.
Kind regards
Andreas.
--
http://fam-tille.de
Reply to: