The following are mostly ideas exchanged in the wild about what could
be done in cdebconf to reduce the impact of the increasing number of
translations on the memory requirements of D-I:
17:07 < bubulle> fjp: I have been playing silly with cdebconf stuff in D-I to see how we could reduce the impact
of translations on memory requirements
17:07 < bubulle> first test: I edited /var/lib/cdebconf/templates.dat and removed all Description-* and
Extended_description-*
17:07 < bubulle> except the ones for French
17:08 < fjp> Raaku: You'll have to say what you need help with...
17:08 < bubulle> I expected that only French translations would be available and not others...but indeed others
are still here
17:08 < bubulle> Means they're in memory, certainly
17:08 < bubulle> but, then, what's the point of the giant /var/lib/cdebconf/templates.dat file?
17:09 < cjwatson> I think the right fix for that is to mmap cdebconf's databases
17:09 < cjwatson> I started working on that a while back but never completed it
17:09 < cjwatson> it's silly that they're on the filesystem and then it reads them all in as well
17:09 < fjp> bubulle: #329743
17:10 < fjp> Yes, that would be a first huge step.
17:10 < bubulle> cjwatson: yep. Morefunnily, I just removed that giant file and D-I still works
17:10 * fjp doubts it's guaranteed to remain working.
17:11 * bubulle doubts it also...just trying to continue and see what happens when new packages are loaded
17:12 < cjwatson> cdebconf will probably write it straight back out again at the next save
17:13 < cjwatson> losing performance too - at the moment it avoids saving the db if it exists and isn't dirty
17:13 < cjwatson> (saving the template db takes a while)
17:14 < bubulle> yep, it is written bak again
17:16 < bubulle> sounds like we're losing about 10MB memory here
17:17 < cjwatson> I don't think nuking the db off the filesystem is the appropriate fix, really
17:17 < bubulle> of course..:-)...this is just a convenient way to see what amount of memory we're losing..:)
17:18 < cjwatson> templates.dat could be gzipped, but then you wouldn't be able to mmap it
17:18 < cjwatson> so long run that would probably be a loss
17:18 < bubulle> dropping off all translations but English and the current language in templates.dat squeezes it
down from 9.4MB to 1.2MB
17:19 < cjwatson> and then you switch language ... ;-)
17:19 < bubulle> sure, going that way would mean you cannot switch languages
17:20 < bubulle> but we could have a special lowmem mode where it wouldn't be possible to switch languages while
still having translations
17:20 < bubulle> after all, users don't switch languages that often....the only case that comes to my mind is the
user error at the first screen
17:20 < cjwatson> lowmem already trims the templates file
17:21 < bubulle> yep, it trims all translations...
17:21 < cjwatson> unless you mean trimming just after localechooser runs and slightly less aggressively
17:21 < bubulle> I was thinking about an intermediate mode
17:21 < bubulle> yes, this
17:21 < cjwatson> I'd suggest maybe trimming it at the start of the anna run tto allow a bit of margin for error
17:22 < cjwatson> memory use doesn't really spike until then anyway
17:24 < bubulle> fjp: I propose pasting this discussion to #329743
--
Attachment:
signature.asc
Description: Digital signature