Thoughts about Debian's present and future Deep Learning Packages
Hello Debian-Science folks,
I'd like to put forward several thoughts which were not discussed
publically, and by the way update the status of Debian's deep learning
software packages.
Status about Deep Learning Packages
-----------------------------------
In my fuzzy memory I may have posted similar things to the list, but
I guess those points are outdated since this field is rapidly
developing, including software stacks.
Here[1] is a not complete list of the state-of-the-art open-source deep
learning frameworks to date. Many of them cannot be found in the Debian
archive, but it's fine.
As time goes by, the research community has gradually moved from the
old-genreation software stacks to new-generation software stacks. The
old generation includes (but not complete)
* Caffe (UC Berkeley) [3]
* Theano (MILA) [4]
* Torch7 (Facebook) [5]
Caffe itself might have been used in some industries, but someday it will
be replaced by new software stacks. I'm still keeping two of the old
frameworks functional because their code is still very valuable for one
to learn about deep learning. The ideas behind these implementations
are different, which may bring people different inspirations.
The new-generation of toolings includes (but not complete)
* Caffe2 (Facebook, for deployment)
* Pytorch (Facebook, for research)
* Cognitive Toolkit (Microsoft, but actually supports linux) [2]
* Matlab (Proprietary as you know)
* MXNet (Apache)
* TensorFlow/Keras (Google, Theano successor)
One of the significant differences between the old and new generation
is DYNAMIC COMPUTATION GRAPH. With dynamic computation graph supported,
new tooling enables reasearcher to deal with data series in variable length
in a much easier way. BTW, the ONNX framework also allows model
transformation between these new tools.
IMHO What we shouldn't do, and what we can do
------------------------------------------------
* We shouldn't aim to provide toolings for researchers, due to DFSG and
proprietary binary blobs. Crutial performance libraries such as
MKL[6] and cuDNN[7] cannot be absent for research.
There is really a giant performance gap between proprietary. By using
these binary blobs researchers can save up hours and days of time.
Reasons:
1. Proprietary blobs are not DFSG-compliant. Hard to find sponsor.
2. We hate binary blobs.
3. Some toxic upstream, such as Nvidia, don't reply email about
redistributability confirmation.
4. There are already upstream binary whls that linked against those
performance libraries. Users just pip3 install xxx.
* We can provide binary and source packages for learners and light-weight
deployment scenarios where the performance isn't critical.
* We can help upstream to keep those tools compiling and working on Debian.
My plan
-------
Caffe2/Pytorch have been merged into one code base. I plan to package
it since I use Pytorch everyday in my research work. However before
that being done, there are still many dependency libraries not present
in the archive.
Pytorch/Caffe2 is scheduled behind spaCy, a industrial level Natual
Language Processing library (Python).
That's all.
[1] https://developer.nvidia.com/deep-learning-frameworks
[2] I'll never touch it even if it's one of the state-of-the-art implementations.
[3] https://tracker.debian.org/pkg/caffe
https://tracker.debian.org/pkg/caffe-contrib
[4] https://tracker.debian.org/pkg/theano
[5] https://tracker.debian.org/pkg/meta-torch-core-free
[6] I've done the packaging. Need a sponsor.
[7] I've done the packaging. The toxic Nvidia doesn't reply my email.
Reply to: