Re: HTSeq test case failures
Hi Diane,
On Fri, Oct 24, 2014 at 06:30:18PM -0700, Diane Trout wrote:
>
> what's the environment the test cases pass in? Is there any randomness in the
> source of test data?
I admit I have no idea about htseq and since you seemed to care
competently about the package. Without doing a web search I remember
vaguely that you reported the failed test problem and I now tried to
verify it. You can see the problem when disabling the latest patch.
Some differences simply smell like rounding problems but there is also
some difference in an integer number which reflects the length of a
list. I left an according comment inside the patch
FIXME: Test suite at build time results in 30085 ... hmmm
I have no idea whether some randomness is involved - but I would not
call this a sensible idea to have randomness in tests and verify for
exactness later.
> I have had enough things go wrong at my job I haven't had
> time to investigate much.
For me the decision is: Would it be better to stick to version 0.5.4.p3
(which might or might not suffer from the same problem which is just not
tested??) or to upload the new version and document the differences in
the test in a README.test file or so. While I also think we should
provide an autopkgtest script I think this is to late for Jessie since
everything uploaded after tomorrow will not make it right in time for
Jessie.
I'm just asking for your opinion since you seem to have the deepest
insight into htseq in our packaging team.
Kind regards
Andreas.
> > > On Sat, Oct 25, 2014 at 12:27:48AM +0200, Andreas Tille wrote:
> > > > Hi,
> > > >
> > > > I have added a quilt patch that skips those four tests that are failing.
> > > > I'm not fully sure whether this is the correct method to deal with these
> > > > issues. Some differences are clearly locking like rounding errors but
> > > > some others do not.
> > > >
> > > > Any opinions whether to upload with skiping those tests or to stay with
> > > > the old version. It's probably to late to ask upstream about this
> > > > (which would be a sensible means to find a real solution).
> > > >
> > > > Kind regards
> > > >
> > > > Andreas.
> > > >
> > > > On Fri, Oct 10, 2014 at 10:07:03AM +0200, Andreas Tille wrote:
> > > > > Hi Diane,
> > > > >
> > > > > the freeze is approaching quickly. Can you give any update of the
> > > > > state
> > > > > of htseq?
> > > > >
> > > > > Kind regards
> > > > >
> > > > > Andreas.
> > > > >
> > > > > On Fri, Jun 13, 2014 at 10:52:34AM +0200, Andreas Tille wrote:
> > > > > > [Debian Science list in CC hoping for extra input about testing with
> > > > > >
> > > > > > differences caused by rounding]
> > > > > >
> > > > > > Hi Diane,
> > > > > >
> > > > > > On Thu, Jun 12, 2014 at 02:51:22PM -0700, Diane Trout wrote:
> > > > > > > I had time to make more progress on improving the debian HTSeq
> > > > > > > package, and
> > > > > > > have updated it to run your test code.
> > > > > >
> > > > > > Thanks. I really appreciate the effort to add tests and I try to
> > > > > > approach this for all packages where upstream provides a test suite.
> > > > > >
> > > > > > > Unfortunately I had some test case failures. I was wondering if
> > > > > > > the tests are failing for you, or if its something about my
> > > > > > > configuration. Below is the relevant error messages.
> > > > > > >
> > > > > > > One change from your code was I changed added
> > > > > > >
> > > > > > > import matplotlib
> > > > > > > matplotlib.user('agg')
> > > > > > >
> > > > > > > to the start of test/test.py and test/tss_test.py to avoid an
> > > > > > > error I was
> > > > > > > getting when trying to import _TkAgg.
> > > > > > >
> > > > > > > I'm using HTSeq 0.6.1p1 and pysam 0.7.7.
> > > > > >
> > > > > > I builded the package from Git (on amd64, rescent pbuilder unstable
> > > > > > chroot) and had quite the same result as you. I'm afraid it depends
> > > > > > from
> > > > > > architecture and used compiler whether these tests might pass or
> > > > > > fail but
> > > > > > I'm lacking qualified experience.
> > > > > >
> > > > > > Any ideas?
> > > > > >
> > > > > > Kind regards
> > > > > >
> > > > > > Andreas.
> > > > > > >
> > > > > > > Doctest of tss.rst:
> > > > > > > ******************************************************************
> > > > > > > ****
> > > > > > > File "../doc/tss.rst", line 52, in tss.rst
> > > > > > >
> > > > > > > Failed example:
> > > > > > > for feature in itertools.islice( gtffile, 100):
> > > > > > > if feature.type == "exon" and feature.attr["exon_number"]
> == "1":
> > > > > > > print feature.attr["gene_id"],
> > > > > > > feature.attr["transcript_id"],
> > > > > > >
> > > > > > > feature.iv.start_d_as_pos
> > > > > > >
> > > > > > > Expected:
> > > > > > > ENSG00000223972 ENST00000456328 1:11873/+
> > > > > > > ENSG00000223972 ENST00000450305 1:12009/+
> > > > > > > ENSG00000227232 ENST00000423562 1:29368/-
> > > > > > > ENSG00000227232 ENST00000438504 1:29368/-
> > > > > > > ENSG00000227232 ENST00000488147 1:29568/-
> > > > > > > ENSG00000227232 ENST00000430492 1:29341/-
> > > > > > > ENSG00000243485 ENST00000473358 1:29553/+
> > > > > > > ENSG00000243485 ENST00000469289 1:30266/+
> > > > > > > ENSG00000221311 ENST00000408384 1:30365/+
> > > > > > > ENSG00000237613 ENST00000417324 1:36079/-
> > > > > > > ENSG00000237613 ENST00000461467 1:36071/-
> > > > > > > ENSG00000233004 ENST00000421949 1:53048/+
> > > > > > > ENSG00000240361 ENST00000492842 1:62947/+
> > > > > > > ENSG00000177693 ENST00000326183 1:69054/+
> > > > > > >
> > > > > > > Got:
> > > > > > > ENSG00000223972 ENST00000456328 1:11873/+
> > > > > > > ENSG00000223972 ENST00000450305 1:12009/+
> > > > > > > ENSG00000227232 ENST00000423562 1:29369/-
> > > > > > > ENSG00000227232 ENST00000438504 1:29369/-
> > > > > > > ENSG00000227232 ENST00000488147 1:29569/-
> > > > > > > ENSG00000227232 ENST00000430492 1:29342/-
> > > > > > > ENSG00000243485 ENST00000473358 1:29553/+
> > > > > > > ENSG00000243485 ENST00000469289 1:30266/+
> > > > > > > ENSG00000221311 ENST00000408384 1:30365/+
> > > > > > > ENSG00000237613 ENST00000417324 1:36080/-
> > > > > > > ENSG00000237613 ENST00000461467 1:36072/-
> > > > > > > ENSG00000233004 ENST00000421949 1:53048/+
> > > > > > > ENSG00000240361 ENST00000492842 1:62947/+
> > > > > > > ENSG00000177693 ENST00000326183 1:69054/+
> > > > > > >
> > > > > > > ******************************************************************
> > > > > > > ****
> > > > > > > File "../doc/tss.rst", line 313, in tss.rst
> > > > > > >
> > > > > > > Failed example:
> > > > > > > len( list( tssarray.chrom_vectors["1"]["."].steps() ) )
> > > > > > >
> > > > > > > Expected:
> > > > > > > 30089
> > > > > > >
> > > > > > > Got:
> > > > > > > 30085
> > > > > > >
> > > > > > > ******************************************************************
> > > > > > > ****
> > > > > > > File "../doc/tss.rst", line 338, in tss.rst
> > > > > > >
> > > > > > > Failed example:
> > > > > > > for step_iv, step_set in tssarray[ almnt.iv ].steps():
> > > > > > > print "Step", step_iv, ", contained by these windows:"
> > > > > > >
> > > > > > > for p in step_set:
> > > > > > > print " Window around TSS at", p
> > > > > > >
> > > > > > > Expected:
> > > > > > > Step 1:[169677680,169677837)/. , contained by these windows:
> > > > > > > Window around TSS at 1:169679671/-
> > > > > > > Window around TSS at 1:169677779/-
> > > > > > >
> > > > > > > Step 1:[169677837,169677880)/. , contained by these windows:
> > > > > > > Window around TSS at 1:169680837/-
> > > > > > > Window around TSS at 1:169679671/-
> > > > > > > Window around TSS at 1:169677779/-
> > > > > > >
> > > > > > > Got:
> > > > > > > Step 1:[169677680,169677838)/. , contained by these windows:
> > > > > > > Window around TSS at 1:169677780/-
> > > > > > > Window around TSS at 1:169679672/-
> > > > > > >
> > > > > > > Step 1:[169677838,169677880)/. , contained by these windows:
> > > > > > > Window around TSS at 1:169680838/-
> > > > > > > Window around TSS at 1:169679672/-
> > > > > > > Window around TSS at 1:169677780/-
> > > > > > >
> > > > > > > ******************************************************************
> > > > > > > ****
> > > > > > > File "../doc/tss.rst", line 359, in tss.rst
> > > > > > >
> > > > > > > Failed example:
> > > > > > > s ##doctest:+NORMALIZE_WHITESPACE
> > > > > > >
> > > > > > > Expected:
> > > > > > > set([<GenomicPosition object '1':169680837, strand '-'>,
> > > > > > >
> > > > > > > <GenomicPosition object '1':169677779, strand '-'>,
> > > > > > > <GenomicPosition object '1':169679671, strand '-'>])
> > > > > > >
> > > > > > > Got:
> > > > > > > set([<GenomicPosition object '1':169680838, strand '-'>,
> > > > > > > <GenomicPosition
> > > > > > >
> > > > > > > object '1':169677780, strand '-'>, <GenomicPosition object
> > > > > > > '1':169679672,
> > > > > > > strand '-'>])
> > > > > > > ******************************************************************
> > > > > > > ****
>
>
--
http://fam-tille.de
Reply to: