Re: [MoM] bcalm package
Hi Shayan,
On Fri, Sep 27, 2019 at 06:15:37PM +0100, Shayan Doust wrote:
> Hello Andreas,
>
> > UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position
> 0: invalid start byte
> > test KO
> > autopkgtest [06:51:22]: test run-unit-test: -----------------------]
> > autopkgtest [06:51:22]: test run-unit-test: - - - - - - - - - -
> results - - - - - - - - - -
> > run-unit-test FAIL non-zero exit status 1
> > autopkgtest [06:51:22]: @@@@@@@@@@@@@@@@@@@@ summary
> > run-unit-test FAIL non-zero exit status 1
>
> This is strange. I don't understand why I can't seem to get the error
> you are getting (both on my buster workstation and within a sid
> environment). Maybe enforcing as string in recent commit might work but
> I am not sure why it works for me.
I admit that error is gone now but I get:
Within that,
creating buckets from superbuckets: 0.0 secs
bucket compaction (wall-clock during threads): 0.0 secs
within all bucket compaction threads,
adding nodes to subgraphs: 0.0 secs
subgraphs constructions and compactions: 0.0 secs
compacted nodes redistribution: 0.0 secs
Sum of CPU times for bucket compactions: 0.0 secs
BCALM total wallclock (excl kmer counting): 0.1 secs
Maximum number of kmers in a subgraph: 4
Performance of compaction step:
Wallclock time spent in parallel section : 0.0 secs
Best theoretical speedup in parallel section : 2.1x
Best theor. speedup in parallel section using 1 threads : 1.0x
Sum of longest bucket compaction for each sb : 0.0 secs
Sum of best scheduling for each sb : 0.0 secs
Done with all compactions 05:41:33 memory [current, maxRSS]: [ 98, 98] MB
[Iterating DSK partitions ] 100 % elapsed: 0 min 0 sec remaining: 0 min 0 sec
bglue_algo params, prefix:tiny_read.unitigs.fa k:13 threads:4
Starting bglue with 4 threads 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
number of sequences to be glued: 8 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
8 marked kmers, 2 unmarked kmers 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
created vector of hashes, size approx 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
pass 1/3, 4 unique hashes written to disk, size 0 MB 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
0 marked kmers, 2 unmarked kmers 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
created vector of hashes, size approx 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
pass 2/3, 0 unique hashes written to disk, size 0 MB 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
6 marked kmers, 2 unmarked kmers 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
created vector of hashes, size approx 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
pass 3/3, 3 unique hashes written to disk, size 0 MB 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
loaded all unique UF elements (7) into a single vector of size 0 MB 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
[Building BooPHF] 100 % elapsed: 0 min 0 sec remaining: 0 min 0 secBitarray 6400 bits (100.00 %) (array + ranks )
final hash 0 bits (0.00 %) (nb in final hash 0)
UF MPHF constructed (0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
[Building BooPHF] 100 % elapsed: 0 min 0 sec remaining: 0 min 0 sec
UF constructed 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
freed original UF (0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
loaded 32-bit UF (0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Allowed 24 MB memory for buffers 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Disk partitioning of glue 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Done disk partitioning of glue 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Top 10 glue partitions by size:
Glue partition 2 has 8 sequences
Glue partition 4 has 0 sequences
Glue partition 8 has 0 sequences
Glue partition 9 has 0 sequences
Glue partition 7 has 0 sequences
Glue partition 3 has 0 sequences
Glue partition 1 has 0 sequences
Glue partition 0 has 0 sequences
Glue partition 5 has 0 sequences
Glue partition 6 has 0 sequences
Glueing partitions 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 0 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 20 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 40 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 60 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 80 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 100 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 120 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 140 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 160 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 180 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 200 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 220 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 240 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 260 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 280 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 300 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 320 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 340 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 360 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 380 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 400 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 420 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 440 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 460 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 480 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Gluing partition 500 (size: 0 MB) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
end 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
debug: not deleting glue files
Finding links between unitigs 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
step 1 pass 0 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
step 2 (0kmers/0extremities) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
step 1 pass 1 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
step 2 (0kmers/0extremities) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
step 1 pass 2 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
step 2 (0kmers/0extremities) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
step 1 pass 3 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
step 2 (0kmers/0extremities) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
step 1 pass 4 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
step 2 (2kmers/2extremities) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
step 1 pass 5 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
step 2 (0kmers/0extremities) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
step 1 pass 6 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
step 2 (0kmers/0extremities) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
step 1 pass 7 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
step 2 (0kmers/0extremities) 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
gathering links from disk 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Done finding links between unitigs 05:41:33 memory [current, maxRSS]: [ 51, 98] MB
Traceback (most recent call last):
File "compare_fasta.py", line 20, in <module>
s1 = read_seqs(fasta1)
File "compare_fasta.py", line 17, in read_seqs
seqs.add(normalize(str(line).strip()))
File "compare_fasta.py", line 10, in normalize
rev = revcomp(seq)
File "compare_fasta.py", line 8, in <lambda>
revcomp = lambda x: ''.join([{'A':'T','C':'G','G':'C','T':'A'}[B] for B in x][::-1])
File "compare_fasta.py", line 8, in <listcomp>
revcomp = lambda x: ''.join([{'A':'T','C':'G','G':'C','T':'A'}[B] for B in x][::-1])
KeyError: 'H'
test KO
autopkgtest [05:41:33]: test run-unit-test: -----------------------]
autopkgtest [05:41:33]: test run-unit-test: - - - - - - - - - - results - - - - - - - - - -
run-unit-test FAIL non-zero exit status 1
autopkgtest [05:41:33]: @@@@@@@@@@@@@@@@@@@@ summary
run-unit-test FAIL non-zero exit status 1
> I guess I will try to find some solution or in the worst case convert
> this into a bash file to diff two files that have already been processed
> by the lambda function.
I'm running the autopkgtest inside a pbuilder hook (see
/usr/share/doc/pbuilder/examples/B20autopkgtest
).
> > for the ping. It might happen that my network connection is to weak to
> > upload before Sunday evening or Monday. Sorry for the delay - please
> > ping again if you do not see anything until Tuesday.
>
> No worries. The size can be a bit of a pain to handle :).
At least fast is in new now. ;-)
Kind regards
Andreas.
--
http://fam-tille.de
Reply to: