Clojure Build Tools in Debian (GSoC 2021 Project - Final Report)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Clojure Build Tools in Debian
(GSoC 2021 Project - Final Report)
Intern: Leandro Doctors (allentiak) <ldoctors@gmail.com>
Mentor: Louis-Philippe Véronneau (pollo) <pollo@debian.org>
Co-mentor: Utkarsh Gupta (utkarsh{2102}) <utkarsh@debian.org>
1 Intro
My name is Leandro Doctors (“allentiak” on IRC), and I've been
the GSoC intern working with the Debian Clojure Team during 2021.
This is my final report. You can also check my original proposal [1]
and my first report [2].
[1]: https://salsa.debian.org/allentiak/gsoc-2021-proposal-feedback/-/blob/main/full.proposal--debian--2021--clojure-cli--leandro.doctors.pdf
[2]: https://lists.debian.org/debian-clojure/2021/07/msg00000.html
2 Summary
Whereas the raw data may not sound by itself very positive, my
personal conclusion is. This is, whereas I didn't fully finish
the required deliverables envisioned in my original proposal, I
do feel I am much closer to, eventually, becoming a Debian
Developer. So, by all means, I consider this project has had a
positive outcome.
3 Project
The goal of the “Clojure Build Tools in Debian” project was to
provide Clojure Debian users with some of the latest advanced
build tools and libraries the upstream Clojure developers have
been lately working on. These include tools.deps.alpha, a library
for dependency graph resolution and classpath building, and the
CLI tool clj, for REPL interaction. If time permitted, I was also
to improve the quality of both new and existing Clojure packages,
and the overall Debian Clojure packaging process. My mentor was
Louis-Louis-Philippe Véronneau, and my co-mentor was Utkarsh
Gupta.
4 Motivation
Why this project? On the one side, if you're a Clojure lover like
me, you may have noticed that the Clojure experience in Debian
is, as of mid 2021, well... still quiet limited. Additionally,
this project aligned with my own background in Free Software
community building and my research interest in Peer Production.
When I mention how limited today's Clojure experience in Debian
is, I can see two reasons for this, deeply intertwined. The first
one is that there currently aren't many Clojure-specific
packaging tools in Debian (such as a clojure-debian-helper). The
second reason for which we only currently have a suboptimal
Clojure experience in Debian, and probably the root of the
previous one, is that many core build tools and libraries for the
language have not simply been packaged yet. My project aimed to
attack that seemingly root cause.
As I said, another reason for me choosing this project is my own
experience as the Co-founder and Leader of, probably, the first
Free Software Community experience in my hometown of San Juan,
Argentina. That interest in Free Software evolved in a first PhD
attempt in what is now known as the field of Peer Production. A
subject that has lived within me as a research interest during my
day job at a University.
Being a Clojure fan, it felt only logical combining all those
interests somehow. And this project seemed like the ideal
combination.
5 The Debian Clojure Team
I've been working with a small, yet very warm team. The current
incarnation of the Debian Clojure Team exists thanks to the hard
work of three people.
Elana Hashman (aka “the Clojure necromancer”), revived the team
around three years ago. Later on, the team gained the invaluable
presence of Louis-Philippe Véronneau and Utkarsh Gupta (my mentor
and co-mentor, respectively).
Together, these “Three Musketeers” have maintained the team
alive, allowing us, Debian users, to enjoy Clojure.
6 Status
During the first part of my project, I mainly worked on learning
the basics of Debian packaging, and got my first package
uploaded. I have to thank Louis-Philippe, Utkarsh, and Elana for
their immense patience and support during that part, as it took
me quite some effort grasping the basics of Debian packaging.
During the second part of my project, I worked on my last
packages, and almost completed the originally required scope of
the project. I only have to finish working on the transition from
the currently provided set of packages (based on a
Debian-specific clojure runner) to the newly provided upstream
clojure and clj runners.
Unfortunately, I didn't have much time left to start working on
the opportunities for improvement already identified by the
Debian Clojure Team originally outlined in my proposal. Whereas I
did update one older Clojure package not built using leiningen
(tools-data-xml-clojure), I didn't write any Lintian tags to make
Clojure packaging in Debian more robust, nor worked towards the
automation of Clojure unit tests in autopkgtests via autodep8.
6.1 Deliverables: Data vs. Conclusions
If we are to talk about deliverables, we should start with the
data. According to my original proposal, I was required to
provide both new and updated Clojure packages accepted into
Debian “unstable”, and updated Clojure packaging documentation.
Additionally, if time permitted, I was to also provide new
Clojure Lintian tags merged by the Lintian maintainers, and new
Clojure autodep8 scripts merged by the autodep8 maintainers.
Whereas I partially accomplished both required tasks, I didn't
manage to start working on any of the optional deliverables.
When looked in isolation, those numbers may look somewhat
disappointing for some people. However, I can draw a much more
positive conclusion. Why?
Firstly, GSoC is supposed to be a learning experience. Moreover,
as I said in my original proposal, “I approach[ed] this project
as a great opportunity to, finally, start my journey towards
becoming a Debian Developer”. In that sense, I consider the time
invested into this project fruitful. In this way, I have learned
the basics of packaging, how to interact with the Debian Clojure
Team, and and already got my first packages accepted. Plus, I'm
looking forward to continuing to work with the Debian Clojure
Team so I can attain the original scope of the project.
Therefore, all things considered, I can consider this experience
as a moderate success.
7 Lessons Learned
Technically speaking, if I have learned one thing during these
weeks, is that packaging, although easy to be underestimated, is
by no means a trivial process. As any Debian Developer surely
knows, the onboarding process can take some time. Plus, what is
easy for some people, can be difficult for others. In my case,
this was quite evident. Whereas I can speak several languages and
learning new ones takes me little effort, grasping the basics of
packaging took me (literally!) blood, sweat and tears. Indeed,
the packaging learning curve was quite steep for me.
That being said, I did learn a thing or two about packaging. So,
if I managed to get here, I'm sure many others can. It may take
them more or less time than what it took me, but learning (at
least the basics) of packaging is an achievable goal.
Technical skill learning aside, I value very highly the
non-technical skills I have so far improved during this project.
For instance, I also learned that it can take some time to adapt
to real-time online communication. Before this project, remote
working meant either exchanging emails or getting into video or
audio calls, with a low emphasis on chat-based interaction. Early
on, I realized that the Debian Clojure Team interacts almost
exclusively via, well... chat! And those two approaches are very
different indeed. It has taken me some time to adjust, but I've
improved greatly in this aspect as well.
Finally, improving my time management skills has been also a key
part of this process. Whereas I had already been working remotely
for over a year and a half already, my day job is not so
interaction-dependent as this project (specially in the
beginning). So it took me some time to adapt to this way of
working, and to plan my workload so I could use those waiting
moments to advance in other parts of the project. Still a lot to
improve here, but improving nevertheless.
8 Acknowledgments
I first have to thank upstream. More specifically, one of the
upstream developers of the clojure-tools, Alex Miller. Everytime
I needed specific information on what do specific parts of the
Clojure CLI tools's codebase do, tools.deps.alpha do, he popped
up a reply in a matter of hours. He has shown genuine interest in
the success of is project during by carefully replying to my
emails with detailed explanations of code intent and form, both
in private and in public conversations. Thank you for all that,
Alex!
Let's move on to the Debian Clojure Team.
First, Elana. I thank Elana for her initial openness when I first
contacted her about this idea. It was *her* who initially
contacted Louis-Philippe so he would become my mentor. I wouldn't
have started to work on this project if it wasn't for her. Plus,
she provided quite a piece of advise in more that one ocassion.
So, thank you very much for all that, Elana!
I also thank Utkarsh, my co-mentor, for his overall technical
advise. And a special mention to his initial help to setup my
Matrix client for OFTC chat. At that moment, it was *him* who
took the time to help me in real time so I could solve that
problem. So, thank you very much for all that, Utkarsh!
I finally have to thank Louis-Philippe, my mentor, for his
patient guidance during the whole process. His dedication and
hard work has been *instrumental* for my progress. And a special
mention for his tolerance with respect to some unforeseen
personal circumstances I had to endure during the first weeks.
When one is playing the newbie, times abound when one depends on
other people's feedback. And Debian is made of volunteers, who
have a life outside it. Every time I asked, Louis-Philippe was
there. I wouldn't have gotten here if it wasn't for him. So,
thank you *so* much for all that, Louis-Philippe!
9 Final Words
I would like to close this report with a reflection.
I have been using Debian for many, many years now, and I had been
looking for a way to contribute back to the project for some time
already. I even did some work on a non-packaging Debian project.
That being said, I never managed to deliver much, really.
So, the very existence of outreach programs as this one is, in my
humble opinion, crucial. In my case, the funding I got through
the GSoC program was instrumental in being able to allocate time
for this endeavor, and to finally get started contributing to
Debian. Plus, it has had a very positive impact on me; in many
ways, some of which I am only starting to discover now that the
project is ending.
When I put things into perspective, this project is very
important for me. Actually, it is nothing but the first step
within a long-term journey: becoming a Debian member. Hopefully,
I would like to be able to apply for Debian membership by the end
of this year.
10 Questions?
Thank you very much for your time reading this! I look forward to
hearing (or reading) your feedback. Please come and meet with the
Debian Clojure Team Moreover, I will be in the Clojure BoF on
DebConf2021. Moreover, do not hesitate to send me an email.
A Data
A.1 Task Status
• Required Tasks:
– T1: Setting up a full Debian packaging development
environment and learning the basics of Debian packaging.
∗ Successfully completed the first part during the
Application period.
∗ Successfully completed the second part during the Coding
periods.
– T2: Identifying and packaging the missing dependencies to
package clojure-cli.
∗ Successfully completed as of the end of Coding II.
– T3: Packaging clojure-cli.
∗ 90% done as of the end of Coding II.
– T4: Updating clojure to use clojure-cli.
∗ To be completed after GSoC.
– T5: Updating the Clojure Packaging Guide with information on
how to use the new clojure-cli scripts.
∗ Improved existing documentation. To be completed after
GSoC.
• Optional Tasks:
– T6: Writing Lintian tags to make Clojure packaging in Debian
more robust.
∗ To be completed after GSoC.
– T7: Working to automate Clojure unit testing in autopkgtests
using autodep8.
∗ To be completed after GSoC.
– T8: Updating older Clojure packages not built using leiningen
or clojure-cli.
∗ To be completed after GSoC.
A.2 Packages
1. Updated packages:
• tools-data-xml-clojure [3]
Stuck in experimental until the latest version of Leiningen
is packaged.
[3]: https://tracker.debian.org/news/1244465/accepted-data-xml-clojure-020alpha6-1-source-into-experimental/
2. New packages:
• tools-gitlibs-clojure -- Clojure API for programatically
accessing git libraries. (Closes #905543) [4]
In NEW [5].
[4]: https://bugs.debian.org/905543
[5]: https://ftp-master.debian.org/new/tools-gitlibs-clojure_2.3.167-1.html
• “ITP: tools-deps-alpha-clojure -- functional API for
dependency management and classpath creation” (Closes #891136)[6]
Needs to be uploaded by Louis-Philippe.
[6]: https://bugs.debian.org/891136
3. In-Progress packages:
• “ITP: clojure-cli -- upstream CLI entrypoints for Clojure” (Closes #891141)
90% done - Package completed. I only need to finish implementing
the transition from existing `clojure` scripts. To be
completed after GSoC.
[7]: https://bugs.debian.org/891141
4. Opened ITPs:
• “ITP: libtools-build-clojure -- a library for building
artifacts” (#992737) [8]
– To be completed after GSoC.
[8]: https://bugs.debian.org/992737
5. Reported bugs
• “libcore-specs-alpha-clojure: should provide (symlinks to)
.jars in /usr/share/java” (#992735) [9]
• “libspec-alpha-clojure: should provide (symlinks to) .jars in
/usr/share/java” (#992736) [10]
[9]: https://bugs.debian.org/992735
[10]: https://bugs.debian.org/992736
A.3 Other
1. Interactions with upstream in the Debian-Clojure mailing list [11]:
[11]: https://lists.debian.org/debian-clojure/
• Many productive interactions with one of the upstream
developers, Alex Miller (June [12], August [13]).
[12]: https://lists.debian.org/debian-clojure/2021/06/threads.html#00012
[13]: https://lists.debian.org/debian-clojure/2021/08/threads.html
2. Wiki page:
(a) https://wiki.debian.org/Clojure/Goals/ClojureCLIToolsInDebian
-----BEGIN PGP SIGNATURE-----
Version: Mailvelope v4.4.1
Comment: https://www.mailvelope.com
wsFcBAEBCAAGBQJhI78sAAoJED4EAsO6whyA3GcQAJlqY+GJ1Mk0g5zKjOUk
4TGJ6Sd3lJHvo9ev+bYRzHoOfQ9KbRFg0pDDnNF6JTCKXB+kBB+VW7urpyWv
jz6ddo6kTsCbIhGFbKOvn41ND0laUPXHFbVk9bVzgyzGR9EGyEpBi05xQiHZ
dDfNXqLP8HK1gyH7croZkNNd7ThJ+dJq+aHCEvPSi7o4E1gmhigQnYVV67SL
xQF/rhw17iKqCz8y3CZzO4QP+l6XtogO6AgCJj+6L+Y+euKKbWpFyapQM4hG
VCYk1r4SeGMMHe1V9VDSKFUqriTKv5AxJcW5U1PC0IIY95J5QfkQSQbd2wYm
V1X8tW5AjNjLfDnHqN0rnV3LCHHilQuaoVVnbZPhN/agRB7IybdxL8x7SaoF
ZpQWe9Pi3b3WyRPjuVdGiphT942toPWQfAJBmf8e9A0g5jsJzzK2T+C6HjJ6
jXHlwK+/qwvfnLXQ24X4aIqEjUdsBUtVAoW+5cNQu7TJraBsWE4SyBhg/W8Z
ldhUh4dbZoUIj4KxDqvQnBHdKUAjVfxZM8PXa1Cgvi/pPCNCFtJTEOEDDQwC
0Ov+mwTVOUpKMP41gUo5iW4CVBSk9zzU5F21JgTOUGM5CuVPTyt1sAfmfvFL
RlTai28ZI49eRyhpUaTMhYCCgIeq72MI/olPO89I6dV1MqUzmVQguIIS3X0E
cmbA
=wmmk
-----END PGP SIGNATURE-----
Reply to: