Thilo Six wrote: > Excerpt from Bob Proulx: > >>> # rsync -av /mnt/backup/etc/ /etc/ > >>> # rsync -n --delete -av /mnt/backup/etc/ /etc/ > >>> # rsync --delete -av /mnt/backup/etc/ /etc/ > >> > >> I believe this not to be a good idea. When you do this you mess around with > >> config files under the control of dpkg. With all the mess that creates. > > > > Thank you for the review comment! However I respectfully disagree. > > I gather we do have different opinions and workflows in this regard which is > absolutely fine. > Maybe i am too worried about this rsync method. Probably due to past > experiences. But it is just too easy to overlook s.th. see below. At one level the problem is that there are a lot of files in /etc. Too many to keep completely in human brain memory all at one time. Plus files will either exist or not exist based upon what packages are installed on system making the list impossible to enumerate statically. But I actually think we are closer to the same than different. More in a moment. > > And apparently you do as well because you immediately suggested > > etckeeper which does exactly the same thing of managing files in /etc. > > It is the same thing but simply using a different tool. > > In this regard i do have an other POV then you. I do believe that > etckeeper does not change any file in /etc without manually > intervention. (Obviously files in '/etc/.vcs' are excluded from > this rule.) So i think this is different from the rsync operation > you suggested. Well... I would claim that rsync does not touch any files other than by manual intervention too. One must invoke the command for it to do anything. The issue in my mind is not the tool and the implication of how sweeping of changes it can make at one time but rather whether it is okay to make any changes at all. If it is okay to make changes (which it is) then whether the modifications are done small scale or large scale is simply a matter of scale. Rsync is definitely a large scale autoautomatic sweeping tool. Whereas etckeeper as I unerstand it is more of a controlled manual targeted tool. But I acknowledge and concede your point. /etc contains a large and unknown set of items. Some of those unknown items should not be restored from backup. Then using a large sweeping tool like rsync to sweep *all* of the files into place is undoubtedly going to sweep in something that should not be restored from backup. * The new system is brand new. The task is to make it look like the old system. Therefore there is no problem with restoring /etc/apache2 completely from backup for example. And if a completely restoration is the task then exactly that is needed. Make the new system look exactly like the old system. * But the new system is a brand new install, not a clone, and some files must remain unique. Such as UUIDs in /etc/fstab, /etc/lvm, /etc/mdadm, /etc/udev/rules and others as you and I have both pointed out. > Let me tell you why i suggested etckeeper. > Here i use it to commit /etc into $VCS on a regularly basis. This helps me to > keep track of what is going on in /etc. > Being able to compare my modifications with the original is a feature you surely > will not miss anymore, after you get used to it. This is where our strategy of attack on the problem is different. I started from the idea that there are a lot of files in /etc but only a few of them need to be excluded from the restoration. Therefore identify the files that should be excluded. Then restore everything *except* for the files that should be excluded. Whereas I think you were started from the idea that there a lot of files in /etc and only the known okay files to restore should be restored. Identify the files that should be restored and only restore those files. Basically one strategy is include all but those excluded and the other strategy is exclude all but those included. I think we are in agreement that some files must be restored and some files must be excluded. We differ on the technique to get there. > > Most files in /etc are "conffiles". They are within the realm of > > control of the user. Changes to them are understood and respected by > > the package manager. There isn't any "mess" caused by interaction > > with dpkg. > > I do have to commit that i did not spent time yet to understand that "conffiles" > system deeply. There is also two strategies to attack this problem too. Either install the desired configuration from backup first and then install the packages selecting the exising conffiles. Or install the packages without any existing configuration and after installation restore the previous files from backup. I had proposed the first with restoring the /etc files first. But either should result in the same end system. > I do have this snipped in my '/etc/apt/apt.conf.d/99local' though: > // --force-confold: > // do not modify the current configuration file, the new version is installed > // with a .dpkg-dist suffix. With this option alone, even configuration files > // that you have not modified are left untouched. You need to combine it with > // --force-confdef to let dpkg overwrite configuration files that you have not > // modified. > > // --force-confdef: > // ask dpkg to decide alone when it can and prompt otherwise. This is the > // default behavior of dpkg and this option is mainly useful in combination > // with --force-confold. > > // --force-confmiss: > // ask dpkg to install the configuration file if it’s currently missing (for > // example because you have removed the file by mistake). > > Dpkg::Options { > "--force-confdef"; > "--force-confold"; > "--force-confmiss"; > } > credits for this go to: http://raphaelhertzog.com That controls the behavior of dpkg when it would normally stop and interact with the user admin who is installing or upgrading a package about the handling of a conffile. Should you install the new package file, keep the existing file, or merge them? Or see the differences between them? The above answers those questions globally so that it never stops to ask a question. I fully agree with confmiss. I routinely set that too. But I have seen heated arguments from people who have argued (apparently successfully because the change went in) that removing a required /etc file is a desired configuration and that it must be preserved. (Regarding the /etc/network/interfaces file.) This adds yet another logical state to be considered. I use a system configuration management system similar to cfengine, puppet, chef that I have programmed to configure the system. I have directed it to read-modify-write various configuration files to set them as desired. In that type of environment it is advantageous to me to use confnew. With confnew I will always get the package version of the file with the package default configuration. Which will then immediately be reconfigured into my desired configuration by the system management tool. (The cfengine, puppet, chef tools could do the same.) I choose to do this so that my /etc files are always fresh and as close as possible to a freshly installed system. I have machines that have been upgraded many, many times. If I were to preserve the original file then I would have files hanging around that were from Potato 2.2 days which would be very ancient now. Certainly for a Stable system only installing security upgrades the choices for confold should always be fine since there should not be any differences in the files in Stable in /etc. But when upgrading across major upgrades I very much prefer to freshen them with packaged versions and then merge in any desired deviation from the pristine. Because major changes will bring major behavior changes and many times for best results I will want to change how the configuration is done. Often the world improves and I can remove my customizations entirely which I always consider a good thing. > > However that isn't true of all files. And in particular there are > > uuids associated with lvm and md. The /etc/lvm and /etc/mdadm > > directories and any other UUIDs in /etc/fstab contain UUIDs and other > > information specific to the new machine and new filesystems. Those > > would need to be avoided when restoring from backup. Otherwise the > > new system would have the UUIDs from the old system and that wouldn't > > work. > > > > * Exclude /etc/fstab > > * Exclude /etc/lvm > > * Exclude /etc/mdadm > > + /etc/initramfs-tools/conf.d/resume Ha! I always use lvm and so for me the names in the /etc/initramfs-tools/conf.d/resume file are always logical names and would be the same across systems. But point taken. If that is a UUID then it should be excluded from the restore. Plus there is your note about ethernet addresses. In addition to the 70-persistent-net.rules file that may also affect people using "guessnet" or other such networking packages. > ..and maybe some more. This is why i would look into the difference prior to > rsyncing. After i had become confident i would use rsync, too. At the center between the strategies of include and exclude is the need to identify which files must be included in the restoration and which files must be excluded. Having that list then we would restore what should be included and avoid those that should be excluded. I think you and I would both identify these files on-the-fly by seeing the files, by looking at the file contents, by other dynamic decision making. But I fear that for many people that would be beyond their level of skill. Our discussion shows that it isn't trivial. It takes some experience. Can we document what we have learned from this discussion? Would someone be able to follow that documentation and successfully peform a restoration as described? Is it possible to create a script that would analyze the backup and the new system and would then identify the files that should be included and excluded? I think we can do all of this but am leaving it as an open question for the moment. Bob
Attachment:
signature.asc
Description: Digital signature