[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: PyTroll satpy packaging

Hi Bas,

Il 29/12/18 21:32, Sebastiaan Couwenberg ha scritto:
> On 12/29/18 8:01 PM, Antonio Valentino wrote:
>> pyspectral is ready for the upload
> There were a bunch of permission errors due to unwritable $HOME like this:
> WARNING: autodoc: failed to import module 'pyspectral.rsr_reader'; the
> following exception was raised:
> Traceback (most recent call last):
>   File "/usr/lib/python3/dist-packages/sphinx/ext/autodoc/importer.py",
> line 152, in import_module
>     __import__(modname)
>   File "/build/pyspectral-0.8.6+ds/pyspectral/rsr_reader.py", line 35,
> in <module>
>     from pyspectral.utils import WAVE_NUMBER
>   File "/build/pyspectral-0.8.6+ds/pyspectral/utils.py", line 179, in
> <module>
>     os.makedirs(LOCAL_RSR_DIR)
>   File "/usr/lib/python3.7/os.py", line 211, in makedirs
>     makedirs(head, exist_ok=exist_ok)
>   File "/usr/lib/python3.7/os.py", line 211, in makedirs
>     makedirs(head, exist_ok=exist_ok)
>   File "/usr/lib/python3.7/os.py", line 211, in makedirs
>     makedirs(head, exist_ok=exist_ok)
>   File "/usr/lib/python3.7/os.py", line 221, in makedirs
>     mkdir(name, mode)
> PermissionError: [Errno 13] Permission denied: '/nonexistent'
> This looks like a violation of Policy 4.9:
> "
>  Required targets must not attempt to write outside of the unpacked
>  source package tree. There are two exceptions. Firstly, the binary
>  targets may write the binary packages to the parent directory of the
>  unpacked source package tree. Secondly, required targets may write to
>  /tmp, /var/tmp and to the directory specified by the TMPDIR environment
>  variable, but must not depend on the contents of any of these.
>  This restriction is intended to prevent source package builds creating
>  and depending on state outside of themselves, thus affecting multiple
>  independent rebuilds. In particular, the required targets must not
>  attempt to write into HOME.
> "
> https://www.debian.org/doc/debian-policy/ch-source.html#main-building-script-debian-rules
> The defaults in the upstream sources may need to be changed, or a custom
> config file needs to be used to use $TMPDIR instead of $HOME.

this one should be fixed now (in git)

>> pyhdf
>> Should be OK but it is still not in the archive and I'm currently not
>> able to build it (after a gcc update). Not sure if we should file a bug
>> for this. Please let me know.
> I sent a message about that earlier, but it never made it to the list
> probably due to the strace attachments.
> The issue is not related to hardening flags nor parallel builds,
> disabling both did not resolve the issue. As these are common causes I
> had to check that.
> Running the `python3 setup.py build` command through strace shows that
> one of the child processes does fail, see the attached traces (in the
> tarball) and python3-build.strace.2241 specifically.
> Perhaps upstream can help troubleshoot the issue.
>> pylibtiff
>> has been adopted and updated to the latest version but it has some build
>> issue on different architectures.
>> An issue has been filed to upstream [1] but I do not expect a quick reply.
>> The issue seems to be related to the use of the bittools extension
>> module on 32bit and/or bigendian architectures.
>> Looking at the code the the bittools extension is only used in the
>> libtiff.lzw module that is deprecated in favor of the tif_lzw extension.
>> Probably it doesn't worth to spend too much time trying to fix it our
>> self. We could just disable the offending tests and put a note in the
>> README.Debian file.
> Filing an RM bug for the package on the failing architectures is also an
> option. That just leaves i386 as this is treated specially but britney
> along with amd64 (NOBREAKALL_ARCHES).
> Since the package has no reverse dependencies, having just amd64 may be
> good enough for now. How much do you care about satpy on other
> architectures?

My first option is still to disable the internal bittools totally.
I verified that the bittools extension module is not part of the public
API and it is not mentioned in the documentation.

Also, it can be replaced by a bitarray (that is packaged in debian) as
backend for the libtiff.lzw module.

Even if it is not the default I think it is perfectly fail to use the
alternative backend.
So my plan is to disable the bittool extension totally: no build and no

Do you think it could be an reasonable solution?

>> satpy
>> It is almost ready. I have already added dependencies for pyspectral and
>> pygac that are still not in the archive.
>> Working on satpy I discovered I discovered that there are 3 undocumented
>> dependencies:
>> * pydecorate (from PyTroll) [2] that seems to be used also in some test.
>> I made a temporary patch to disable tests using pydecorate.
>> * glymur [3] a Python interface for JPEG 2000 that is imported at
>> toplevel in the satpy.readers.msi_safe reader module.
>> * pyninjotiff (also form PyTroll) [4] required by the
>> satpy.writers.ninjotiff writer module.
> Are there patches or issues upstream to add these dependencies to the
> module metadata and documentation?

just filed issue #562 [1]

[1] https://github.com/pytroll/satpy/issues/562

>> IMO packaging also missing modules should not require to much effort so
>> I would like to do it.
>> Can you please create repositories for them on salsa?
> Done:
>  * https://salsa.debian.org/debian-gis-team/pydecorate
>  * https://salsa.debian.org/debian-gis-team/glymur
>  * https://salsa.debian.org/debian-gis-team/pyninjotiff

thank ypu very much

Antonio Valentino

Reply to: