eXtplorer web file manager packaging work and SWFUpload packaging
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi everyone,
This email has 2 goals:
1/ Let you know about my past work
2/ Ask for help for packaging swfupload
Because it has been said many times in Debconf that we should tell about
things we do in Debian, I decided to write a quite long email, so you
guys understand my motivations and past work, even if it might sound
ridiculous compared to work from the {release,kernel,d-i,$whatever} team.
If you just want to help me, just skip everything until the last
paragraph. If you don't care about web apps or if you are too busy with
other RC fixes, just don't read this message...
Why did we need a server side web file manager in Debian:
- ---------------------------------------------------------
Not only because because it's cool! :)
Yes, making things easy to use is important. It's not a secret, many
users are using Ubuntu because they have the (true or false) impression
that Debian harder. We make software for end users also, and we should
never forget it.
Having a web file manager is one way to help those that are experiencing
difficulties in managing their web space. But not only! Let's say you
want to upload a big CMS like drupal to a shared host, most of the time,
your only choice is to upload all files one by one. If there's some
latencies between you and the server, it can take a long time to upload
all these small files.
Now, with the help of a good file manager, you just upload the tar.gz,
then right click on it, and select extract. As it's done on the server,
the extraction is very fast. Then you right click on the config_inc.php
file, select edit, change few parameters, hit save, and it's done.
So it's not only useful to newbies, but also it's very convenient for
everyone, IMHO.
I have seen posts in the debian-isp list from people saying that if you
want to use such software, you're a jerk, because it's never secure.
Well, running any server is insecure, and everything is about
trade-offs, IMHO. Here, clearly this adds real functionalities.
The story behind packaging eXtplorer:
- -------------------------------------
The bellow tells about my past 9 months of work with eXtplorer, giving
you a rough overview of what and why I did.
I first tried to find out which web file manager I should package, by
evaluating what was available online, and what seemed to be supported.
Thanks to some useful comments in the debian-isp list, I had few apps to
try.
My requirements where quite simple:
- - should be easy to use (Ajax, etc.)
- - support extracting of archives like zip / tar.gz / others
- - has to support multiple users with a user base separated from the system
- - accesses files on the system directly, so that it's fast for all heavy
operations
- - DFSG compiliant (open source free software)
- - has an active upstream author that reacts fast to emails
There wasn't a lot of candidate, despite what one may think at first. In
fact, writing such a web file manager that really does what a user need
isn't trivial at all. I guess it falls to the category of things you may
think as easy to write at first, then later you quickly realize that
it's a huge work.
The only software that I found usable, mature, and having enough
features, was eXtplorer, that used the (more and more famous) ExtJS
javascript library. It is also extensively used by "turnkeylinux" and
all the appliances they do. I believe that the upstream author of
eXtplorer knows what he does, his code seemed clean (apart the fact
there was embedded libraries, see below), and used by a lot of people.
In fact, eXtplorer was first developed as a plugin for the Joomla CMS.
But it also work in a standalone mode.
eXtplorer filled all the needs I had, so I started packaging. It wasn't
easy, because it eXtplorer was embedding a lot of libraries. There like
more than a dozen of them. Here's the big list, with a ! when I had to
build a Debian package (otherwise, it means it was already in Debian):
- - editarea!
- - extjs3!
- - yui
- - SWFUpload
- - php-system
- - php-service-json!
- - php-geshi
- - php-net-ftp
- - php-auth-http
- - php-mime-type!
- - php-http-upload!
- - php-webdav-server!
- - php-service-json!
That's a total of 7 dependencies that I had to build, and maybe I have
forgotten some in this above list. All the above has been removed from
upstream tarball, and it has been repackaged into a +dfsg archive
(together with a script that I maintain to do this removal).
I became DD last June, which gave me the opportunity to upload all these
new packages to SID just few weeks before Squeeze got frozen. Doing so,
I did some very bad work writing the debian/copyright file, which brings
(legitimate) hate from the ftp-masters. It was full of mistakes. I want
to take this opportunity to publicly tell that I feel sorry about my bad
work with these copyright files, and thank the ftp-masters (especially
Alexander) that did the careful review that I failed to do. Looking
backward, I think I was too focussed on removing embedded libs in
eXtplorer. I did also the same bad work when packaging xen-qemu-dm, and
didn't pay enough attention to what's required in Debian. Of course, my
packages have since been updated. I will do what I can so that it
doesn't happen again, and I will review each single file with more
attention next time. So again, my apologies here.
I thought that was it, but I was wrong. Raphael seemingly (just a wiled
guess here...) did a mass bug filling searching for swf binary files in
the Debian archive (he found some in other packages, if I'm not
mistaking), and he found that I missed the swfupload.swf file in the
package, that was embedded as well in eXtplorer. Thanks for this work.
Today, I have uploaded to SID a version without the .swf binary file, in
order to fix the RC bug. I then realized that I didn't write about the
copyright holder in debian/copyright, so I sent a bug against my own
package so that everyone knows that I'm aware of the issue (that I will
fix soon).
Sören Eberhardt-Biermann, the upstream for eXtplorer, told me that it
will continue to work without the swfupload.swf (only the file upload
function wont work), so it's not the end of the world, but I still would
like to fix this, as it is quite convenient for the average user.
That's it for what I did, now let's talk about the upload feature that I
have just removed from eXtplorer.
How to perform a user friendly upload in a web site:
- ----------------------------------------------------
Uploading a file to a website is really a basic building block for many
applications. However, it's been more than 15 years that there's still
no fully satisfying solution for this issue.
If you have a big file to upload, the <input type="file"> html tag is
a bad solution, because you wont have a clue on what's going on when you
upload a file. My (own and personal) definition of a big file in this
case is: anything that takes more than 10 seconds to upload, because
that's what I feel the critical point where feedback starts to become
mandatory. Past that limit, you need to have a progress bar. If you
don't, then you can't tell if your browser is stuck, the website not
responding, and so on: you have zero feedback until the upload is
finished, no way to tell if it's working, and you have even less clue on
how long it will take.
To add an upload progress bar in a web browser, there's 4 options
available that I know. A java applet, some Ajax on the server side that
will tell how much has been uploaded so far, using Flash, or using the
new HTML5 definition to do a drag and drop. Honestly, none of these
solutions are perfectly good, and only the Flash option seem reasonable.
Here's why.
Ajax and server side: to do this, you'd have to open the browser in a
new window, or use an iframe, so that you can render the progress while
another HTML page is loading (using a POST with some attached content).
This really is quite ugly, and not user friendly. iframes have been
removed from recent HTML standards if I am not mistaking, and opening a
window for the upload with the progress showed in another one isn't what
a designer will want.
Java applet: this works absolutely perfectly, but the issue is that you
have to have a signed applet, otherwise you can't access files on the
client hard drive. In Debian, as we only ship binaries built from
source, we could in theory have the necessary infrastructure to have the
person that builds sign the applet with a valid certificate from an
authority like verisign. It means we would be having a package that is
DFSG compliant, but you'd have to pay the certificate authority to get a
valid certificate for the applet, so that you can use it without the big
warning. And that requirement isn't there for nothing: if the applet has
rights to access the hard drive, it can literally read all files and
send them all anywhere on the web. Anyway, that really wouldn't be nice
to have a free software that you have to pay to be able to use it, so I
don't like this solution, also because it will be hard for a user to
tell if the (self-signed) applet is doing some good, or it's a hack
intended to access the full of your hard drive. Anyway, for those that
wants to have a look at what it could be like, you can try my "J'upload"
applet that I wrote *years* ago:
http://www.gplhost.com/old_stuff/rubs/softwares/jupload_test.php
as you will see, it's self signed, as I didn't want to give any
certificate money. If you don't trust me (you should even with what I
wrote above, this email is signed with my PGP key for that purpose),
then don't accept the certificate. Note that the upload applet is set to
sends to a company with a butterfly logo (just to have fun...), so send
something interesting for them, like the full of the GPL license ... :)
HTML5: this is really the best option. Unfortunately, currently only
Firefox 3.6 supports it. If you can predict if people from Redmond will
upgrade IE to support it, then please send me your crystal ball: I also
would like to predict the future. As it stands, nobody can tell what
will be the adoption rate for this feature. And even, there's sill users
with the older browsers. Ol on #debian-devel nicely gave me this URL,
which is a good example of what's going to be possible in the near
future with some browsers (and right now with Firefox 3.6 Alpha 1):
http://www.thecssninja.com/javascript/drag-and-drop-upload
Flash: this is (I'd say: unfortunately considering the security record
of this famous browser plugin) what I feel is the best solution. Flash
is installed in nearly all browsers, it is working *now* and not
sometimes later on, and on all browsers. Many big sites are using this
method, including youtube itself (AFAIK). There's no need to sign a
piece of binary either, and there's the SWFUpload library that is
released under the MIT license, is fully free. It's available here:
http://code.google.com/p/swfupload/
Note: the old swfupload.org seems to not be the home of this project
anymore (weird that they are moving away from their own domain, if you
ask me).
How to fix the SWFUpload issue:
- -------------------------------
The SWFUpload is a client side file upload tool that uses a combination
of Flash and Javascript to provide upload functionality beyond what the
basic browser provides with the <input type="file"> tag. It is shipped
with the source files of the action script source code that are needed.
But the issue here is that we need to build it using the tools available
in Debian only. As of today, the only action script compiler that I am
aware of is the "mtasc" one, that pabs packaged.
Now, my problem is simple: I don't know anything about Flash, mtasc
(that pabs maintain) or others, and I need help to do this packaging. As
I don't know Flash, I don't think it's a good idea that I become the
official maintainer of the package once it is in main.
I know I could have just sent a request for packaging, but if I don't
give the explanations that are going with it, and don't advertise,
nobody will help.
This would be a very small package and I believe it will be fast to
write it.
So, help anyone?
Thomas Goirand (zigo)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkxga+wACgkQl4M9yZjvmklUlQCfdfS7v66iHgejFy2QMbRriAk9
2tMAn1LdMv6E1TCOmpRyE28MrEsgu4mE
=shAd
-----END PGP SIGNATURE-----
Reply to: