** On May 12, Craig Sanders scribbled: > > > Very unreliable stuff and also puts some strain on the server > > > the data is fetched from - if it has a large directory tree, > > > regenerating lsR with every change to the file system takes much > > > time. With ftp the client just asks the server for the listing and > > > that's it. > > > > Uh... Let me ask you a question. What is cheaper, generating a ls-LR > > once when your archive changes or several hundred times each day for > > each mirror. Particularly when it takes a good 4-5 mins to actually do > > a ls-LR :P > > that depends. when a directory tree only changes rarely (e.g. only > during an rsync or mirror run), it's a big win to pre-generate the > ls-lR.gz files. that's what i do as part of my mirror run immediately > after i've mirrored debian (or whatever) to my local system. Then it's perfectly justified. But if you've got an ls-LR that spans the entire anonymous tree within which there's an incoming directory and every upload to that directory triggers regeneration of ls-LR - then you can soon end up with a blocked system, problems with overlapped upload/listing generation sequences and inaccuracy of the information in the ls-LR file. > when a directory tree changes often and unpredictably, you're better off > either not generating the ls-lR at all or making a decision that it's OK > for the ls-lR file to be 5 or 30 or 60 or whatever minutes out of date. I don't see a point in having and using a file that's inaccurate, that might contain false information. Then it might happen that client, judging upon contents of the ls-LR file, tries to fetch file(s) that don't exist anymore, but they're still in the listing. > btw, reiserfs would probably take a lot less than 4-5 minutes to > do the ls-lR - it's amazing how fast it is for operations that are yes, that's true, but reiserfs is still in a state of flux... > traditionally slow on ext2 and other fs. i'm using it for squid cache > spools and for an experimental news spool - no problems so far. it'll be > a good thing when reiserfs is included in the standard kernel. Hopefully finally it will happen :). > > The ftp client *should* already be using pre-generated ls-LRs. > > mirror clients generally should. i don't know of any ftp clients that > do. If it is guaranteed that the ls-LR contains accurate information, then they certainly should. marek
Attachment:
pgpL3twzHNJzv.pgp
Description: PGP signature