Re: PyTroll satpy packaging
Hi Bas,
Il 29/12/18 21:32, Sebastiaan Couwenberg ha scritto:
> On 12/29/18 8:01 PM, Antonio Valentino wrote:
>> pyspectral is ready for the upload
>
> There were a bunch of permission errors due to unwritable $HOME like this:
>
> WARNING: autodoc: failed to import module 'pyspectral.rsr_reader'; the
> following exception was raised:
> Traceback (most recent call last):
> File "/usr/lib/python3/dist-packages/sphinx/ext/autodoc/importer.py",
> line 152, in import_module
> __import__(modname)
> File "/build/pyspectral-0.8.6+ds/pyspectral/rsr_reader.py", line 35,
> in <module>
> from pyspectral.utils import WAVE_NUMBER
> File "/build/pyspectral-0.8.6+ds/pyspectral/utils.py", line 179, in
> <module>
> os.makedirs(LOCAL_RSR_DIR)
> File "/usr/lib/python3.7/os.py", line 211, in makedirs
> makedirs(head, exist_ok=exist_ok)
> File "/usr/lib/python3.7/os.py", line 211, in makedirs
> makedirs(head, exist_ok=exist_ok)
> File "/usr/lib/python3.7/os.py", line 211, in makedirs
> makedirs(head, exist_ok=exist_ok)
> File "/usr/lib/python3.7/os.py", line 221, in makedirs
> mkdir(name, mode)
> PermissionError: [Errno 13] Permission denied: '/nonexistent'
>
> This looks like a violation of Policy 4.9:
>
> "
> Required targets must not attempt to write outside of the unpacked
> source package tree. There are two exceptions. Firstly, the binary
> targets may write the binary packages to the parent directory of the
> unpacked source package tree. Secondly, required targets may write to
> /tmp, /var/tmp and to the directory specified by the TMPDIR environment
> variable, but must not depend on the contents of any of these.
>
> This restriction is intended to prevent source package builds creating
> and depending on state outside of themselves, thus affecting multiple
> independent rebuilds. In particular, the required targets must not
> attempt to write into HOME.
> "
>
> https://www.debian.org/doc/debian-policy/ch-source.html#main-building-script-debian-rules
>
> The defaults in the upstream sources may need to be changed, or a custom
> config file needs to be used to use $TMPDIR instead of $HOME.
this one should be fixed now (in git)
>> pyhdf
>> Should be OK but it is still not in the archive and I'm currently not
>> able to build it (after a gcc update). Not sure if we should file a bug
>> for this. Please let me know.
>
> I sent a message about that earlier, but it never made it to the list
> probably due to the strace attachments.
>
> The issue is not related to hardening flags nor parallel builds,
> disabling both did not resolve the issue. As these are common causes I
> had to check that.
>
> Running the `python3 setup.py build` command through strace shows that
> one of the child processes does fail, see the attached traces (in the
> tarball) and python3-build.strace.2241 specifically.
>
> Perhaps upstream can help troubleshoot the issue.
>
>> pylibtiff
>> has been adopted and updated to the latest version but it has some build
>> issue on different architectures.
>> An issue has been filed to upstream [1] but I do not expect a quick reply.
>> The issue seems to be related to the use of the bittools extension
>> module on 32bit and/or bigendian architectures.
>> Looking at the code the the bittools extension is only used in the
>> libtiff.lzw module that is deprecated in favor of the tif_lzw extension.
>> Probably it doesn't worth to spend too much time trying to fix it our
>> self. We could just disable the offending tests and put a note in the
>> README.Debian file.
>
> Filing an RM bug for the package on the failing architectures is also an
> option. That just leaves i386 as this is treated specially but britney
> along with amd64 (NOBREAKALL_ARCHES).
>
> Since the package has no reverse dependencies, having just amd64 may be
> good enough for now. How much do you care about satpy on other
> architectures?
My first option is still to disable the internal bittools totally.
I verified that the bittools extension module is not part of the public
API and it is not mentioned in the documentation.
Also, it can be replaced by a bitarray (that is packaged in debian) as
backend for the libtiff.lzw module.
Even if it is not the default I think it is perfectly fail to use the
alternative backend.
So my plan is to disable the bittool extension totally: no build and no
test.
Do you think it could be an reasonable solution?
>> satpy
>>
>> It is almost ready. I have already added dependencies for pyspectral and
>> pygac that are still not in the archive.
>> Working on satpy I discovered I discovered that there are 3 undocumented
>> dependencies:
>>
>> * pydecorate (from PyTroll) [2] that seems to be used also in some test.
>> I made a temporary patch to disable tests using pydecorate.
>> * glymur [3] a Python interface for JPEG 2000 that is imported at
>> toplevel in the satpy.readers.msi_safe reader module.
>> * pyninjotiff (also form PyTroll) [4] required by the
>> satpy.writers.ninjotiff writer module.
>
> Are there patches or issues upstream to add these dependencies to the
> module metadata and documentation?
just filed issue #562 [1]
[1] https://github.com/pytroll/satpy/issues/562
>> IMO packaging also missing modules should not require to much effort so
>> I would like to do it.
>> Can you please create repositories for them on salsa?
>
> Done:
>
> * https://salsa.debian.org/debian-gis-team/pydecorate
> * https://salsa.debian.org/debian-gis-team/glymur
> * https://salsa.debian.org/debian-gis-team/pyninjotiff
thank ypu very much
--
Antonio Valentino
Reply to: