[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Bug#649312: git: command_bidi_pipe does not set up environment for repositories not at cwd

-=| Jonathan Nieder, 19.11.2011 14:04:48 -0600 |=-
> Package: git
> Version: 1:
> Tags: upstream fixed-upstream moreinfo
> X-Debbugs-Cc: Damyan Ivanov <dmn@debian.org>
> From [1]:
> > If you are working with multiple packages maintained by the Debian
> > Perl Group, then you will probably want to use mr(1) to manage the
> > multiple git repos involved. For this workflow, you'll need the
> > following packages installed:
> >
> > git (>= 1:
> >
> >	The version in squeeze has buggy command_bidi_pipe method
> The fix is presumably v1.7.4.2~8^2 (perl: command_bidi_pipe() method
> should set-up git environmens, 2011-02-15).  It would be nice to know
> what the symptoms in the motivating case are, in order to decide
> whether this fix ought to be included in a stable update.

Thanks for the offer! :)

It seems to me that is not critical for pkg-perl mr work-flow, though, 
as the script that uses Git.pm (compare-hashes[1]) has a workaround -- 
it uses an embedded module (MY::Git), which descends from Git.pm and 
adds two new methods for working with bidi pipes, supporting also 
capturing of STDERR, and while there also fixing the missing 
application of the git environment.

 [1] http://anonscm.debian.org/gitweb/?p=pkg-perl/meta.git;f=compare-hashes

A stable fix would make this unnecessary (the STDERR capturing is not 
used in our code anyway) and simplify our code. On the other hand, the 
above documentation needs to be fixed regardless, as the git version 
requirement is no longer true, as I see it.

Perhaps somebody from the Perl Group (CC-ed) could try this on 
a squeeze system with the stock git package and confirm?

Here's the long story:

command_bidi_pipe() is used to launch any/custom git commands, feed 
their STDIN and capture their STDOUT. The pkg-perl team needs this 
functionality in a custom tool of ours, which is used as a 'skip' hook 
for mr.

Let me elaborate. The problem we are facing is caused by the sheer 
amount of repositories (2100+). With that many repositories, running 
'mr up' is slow, no mater how many parallel jobs are started or if the 
ssh connection is reused. Comparing the hashes of each repository just 
takes time (most probably doe to great amount of disk seeks). We work 
around this by keeping a special file on alioth, containing the hashes 
of all the heads of all the repositories. The file is kept current by 
PET via a post-receive hook.

So a 'mr up' first fetches that file with wget and splits it locally 
for each repository. Then, when the skip hook is evaluated the hashes 
of the local repository are compared with those from the retrieved 
file, and the repository is skipped if everything matches.

Here comes command_bidi_pipe() -- it is used to run 'git cat-file 
--batch-check' with all the retrieved hashes fed on STDIN and the 
result retrieved from STDOUT. If all the remote hashes are present 
locally, the repository is skipped.

The squeeze command_bidi_pipe didn't respect the environment given to 
Git->repository(), so it was kind of unusable. One way around this 
would be to set up %ENV before calling command_bidi_pipe, but I opted 
for the solution in the start of this mail.


Attachment: signature.asc
Description: Digital signature

Reply to: