Bug#1013290: ITP: filespooler -- Sequential, Distributed, POSIX-Style Job Queues
Package: wnpp
Severity: wishlist
Owner: John Goerzen <jgoerzen@complete.org>
X-Debbugs-Cc: debian-devel@lists.debian.org, debian-rust@lists.debian.org
* Package name : filespooler
Version : 1.2.1
Upstream Author : John Goerzen <jgoerzen@complete.org>
* URL : https://www.complete.org/filespooler/
* License : GPL
Programming Lang: Rust
Description : Sequential, Distributed, POSIX-Style Job Queues
Filespooler is a Unix-style tool that facilitates local or remote command
execution, complete with stdin capture, with easy integration with various
tools. Filespooler's capabilities:
.
It can easily use tools such as S3, Dropbox, Syncthing, NNCP, ssh, UUCP, USB
drives, CDs, etc., or pipes as transport. Basically anything that's a
filesystem or a pipe can be a transport.
.
It can use arbitrary decoder command pipelines (eg, zcat, stdcat, gpg, age,
etc) to pre-process stored packets.
.
Its storage format is simple on-disk files with locking.
.
It supports one-to-one and one-to-many configurations.
.
Locking is unnecessary when writing new jobs to the queue, and many arbitrary
tools (eg, Syncthing, Dropbox, etc) can safely write directly to the queue
without any assistance.
.
Queue processing is (by default) strictly ordered based on the order on the
creation machine, even if job files are delivered out of order to the
destination.
.
stdin can be piped into the job creation tool, and piped to a later executor at
process time on a remote machine.
.
The file format is lightweight; less than 100 bytes overhead unless large extra
parameters are given.
.
The queue format is lightweight; having 1000 different queues on a Raspberry Pi
would be easy.
.
Processing is stream-based throughout; arbitrarily-large packets are fine and
sizes in the TB range are no problem.
.
The Filespooler command, fspl, is extremely lightweight, consuming less than
10MB of RAM on x86_64.
.
Filespooler has extensive documentation.
.
This package contains the command-line tool (fspl) for interacting with queues.
Reply to: