Bug#853915: reportbug: Retrieved base64 messages aren't decoded

[Nis Martensen <nis.martensen@web.de>]

>>>> When running e.g. `reportbug -N 853037`, a bunch of base64 is 
>>>> displayed instead of the actual content of the messages.

> Could the BTS SOAP interface be changed to return the decoded message
> body of signed messages? Being able to deal with all other kinds of
> complex MIME messages is not really necessary.

I've been looking at the tools interacting here and am not yet sure
where the bug is.

Python-debianbts, when retrieving a bug log via the BTS SOAP interface,
receives each buglog element (message) already split into header and
body [get_bug_log]. If the body is base64-encoded, it gets decoded
before the function returns the bug log. Python-debianbts also attempts
to reconstruct something resembling the original full message by using
the feedparser, and includes that in the buglog elements (dicts) it
returns. I am not sure how reliable that message reconstruction is, but
I suspect it is not perfect.

[get_bug_log]:
https://github.com/venthur/python-debianbts/blob/master/debianbts/debianbts.py#L298

Now I'd like to understand the constraints better under which
python-debianbts is operating:
What exactly is the BTS supposed to deliver via SOAP as the message body
part of the bug log? If the message is a simple text/plain email, is the
body expected to be already decoded or not? If the message is some
MIME/multipart construct, is the body then expected to be the main text
message part only or should it just be everything that is not part of
the main message headers?

I've been trying to look at the debbugs code to find the answer to these
questions, but with limited success so far. Looking at
lib/Debbugs/SOAP.pm in subroutine get_bug_log, it uses Debbugs::MIME's
parse function to split the messages into header and body:
https://salsa.debian.org/debbugs-team/debbugs/-/blob/master/lib/Debbugs/SOAP.pm#L249
`parse` in turn uses `getmailbody`, which definitely tries to extract
the main text message part and does not just dump everything that isn't
part of the primary message headers. So either something does not work
as expected there, or I'm simply looking at the wrong code and should be
looking somewhere else.

Ideas?

In the meantime I have come up with a workaround for this in reportbug,
but it would still be useful to know if everything else is working as
intended or not.