On Thursday 11 September 2008 14:44:18 Marco Amadori wrote: > Il giovedì 11 settembre 2008 07:58:39 Joseph Rawson ha scritto: > > After taking a long look at ubiquity, I decided to try another route. > > [..] What I wound up doing is trying to make a debian package out of > > partman. I did this by checking out the partman directory in the svn > > repository, and building the udebs. I then extrated the udebs into a > > directory, and the templates into a subdirectory of that directory. I > > then used regular debian tools to install the result into a deb. > > Why not build the deb directly from the sources? > I haven't tried that. I'm building them from source, but I haven't been making a deb out of each one of them. > In any case in a regular deb, maybe most parts of the udeb postinst should > be moved to the runtime, since in udebs most of the action happens in > postinst. > I didn't know that. I've just been using the debconf templates, since that's all I saw as being necessary. There's also the problem of these udeb's writing into /usr/lib at runtime, which shouldn't be allowed, but they seem to be able to get away with it on a short-lived ram system. This shouldn't be done on a more permanent system. > The chroot approach you tried before seems nicer to me, since it could > require less diverge of d-i code, but I didn't tried your route yet. > I like it better too, but the chroot seems to act funny at times, and I'm having a hard time figuring out where it goes wrong. it also takes a lot more memory (at least 15M) than just making the deb(s), since you have to extract the initrd into tmpfs on a live system. I think that extracting the initrd during the creation of the live image would be a good way to solve this problem. (this reminds me, the live installer doesn't seem to copy the live system on the hard drives for the net images, but I've only tried it once, and haven't had time to look into it further) > > I took a look around the web, and it seems that the state of automated > > partitioning is about the same as it was in 2003 when I started making > > paella, with the exception of partman (and maybe anaconda, I'd almost > > forgotten about that one). I guess that there aren't very many people > > interested in this. > > Partman is nice after all, it could be scripted with recipes, it supports > raid, lvm, dmraid and crypted volumes. > I agree, it's very nice. That's why I've been spending so much time in trying to get it to work outside the confines of the d-i environment. > > It seems that after looking around, the perl script in > > fai is still the best choice for partitioning disks and creating > > filesystems on a regular system (with the possible exception of anaconda, > > which I'm going to look into once I'm done writing this). > > I don't know fai partitioning scripts, in which ways are they superior to > partman? It's not superior in any way, except that I can automatically partition a disk and make filesystems on it very easily on a regular system. Partman does do this better, but only does it's work in the d-i environment. I chose to use the fai script in paella because it was being actively maintained, and at the time, partman wasn't even around (the work on d-i had just begun back then). I also think that the people who are using fai would also do better with partman. I'm sure that they would've already switched to using it if partman was available as debs instead of being udebs only. After writing the email last night. I took some time to look at anaconda. It's written either mostly or entirely in python, which is good for me since that's the language that I'm most comfortable in. The partitioning in anaconda depends on the pyparted extension, found here: http://dcantrel.fedorapeople.org/pyparted/ This python extension has absolutely zero documentation and seems to only be used in anaconda. I've already made a deb out of this, and I'm going to spend the day making a pure python wrapper around the extension with some docstrings to help document it. The anaconda code seems to support raid, dmraid, lvm, and apparently the beginnings of crypto (I haven't looked at that part much yet). There is a lot of red hat specific code in there, which I won't be using. There is also a lot of support for architectures that I don't recognize, and I'm probably not going to be using those parts of the code either. I may not use any of it, if it proves to be more burdensome then hacking at d-i. Regardless, I'm probably going to maintain the pyparted code and document it better. The code seems to be in good shape (much better than that old python-parted package that has been removed from debian), and a good package with documentation and examples may help inspire other developers to actually use it. The problem in the past with these types of extensions is that they have been poorly implemented and poorly documented. The pyparted code seems to be well implemented and will probably be maintained upstream for as long as anaconda is maintained. That's half of the problem solved. Hopefully I'll be able to handle the other half. The biggest problem that I've faced with the debian-installer code is the assumption that it would only be used in that particular environment. And that environment is minimal, to say the least. While the reuse of the debian installer code sounds really nice on the surface, it's only when you dig into more that you see how pervasive that assumption is, and how it affects the way the code is run. I am talking from the standpoint of already having the code available and trying to get it to run in another environment, instead of being at the standpoint of having worked with the code to get it to run in its natural environment, so I'm probably not aware of many of the important details that I should be knowing about (such as including the postinst scripts you mentioned above). The only success that I've seen in running the d-i code outside of that environment has been with ubiquity. I messed with that for a while, but following the way that the code is written hasn't been easy for me. There also seems to be more problems with implementing ubiquity in debian, rather than just learning how they managed to integrate the debian-installer. I'll admit that I get scared when I see autotools being used extensively, and a lot of recursive makes going on. It makes things harder for me to follow. Using yet another version control program just adds to this, else I'd probably spend more time figuring out how things are done with ubiquity. I'm willing to continue working on getting the debian installer to work in a regular environment in the future, if anybody else is interested enough to help. It's become obvious to me that I can't really do it very well alone. I really want to get paella up and running in better shape than it has been in the past, and the main thing that it lacks is what partman does well. I don't really like pulling parts out of anaconda to do this, but at this point it seems that this is the best option. The code is written in python (as is paella) and it's easier for me to work with. I know that I probably won't be getting any help with hacking on the debian installer until after lenny is released. As far as that is concerned, it's not a problem. I'm willing to switch gears and work on it as soon as somebody else is. However, my financial situation won't allow me to wait for this for the purposes of having better partition/filesystem code in paella, so I can't let the grass grow under my feet there, and I will have to pursue using parts of anaconda to fullfill that requirement. Even so, I won't be able to support the vast array of systems and options that either d-i or anaconda does, but I'll at least have something that I can work with. I don't want to spend too much effort here, either, if there's going to be future interest in getting d-i to run on a regular system. Technically, paella can do everything partman does, since you can hook a script to replace any step of the process, but without the script, the default behavior is very limited, and probably not appealing. I wish to make it a little more appealing without spending too much effort in doing so. Being able to preseed partman with how to do this would eliminate a lot of the problems here, and keep people from having to write extravagant scripts to support raid, etc. and also keep me from having to write a lot of code to support just a small subset of what partman is capable of. This email seems to have run away from me! :) I'm sorry it's so long. I wanted to be detailed on what I have been doing, and what the future looks like for me and paella. -- Thanks: Joseph Rawson
Attachment:
signature.asc
Description: This is a digitally signed message part.