...
The trouble here seems to be, at least in part, that Adobe feels at liberty to extend the PDF spec without caring about users with non-Adobe readers. Adobe probably won't alter its approach unless it feels a significant community backlash is brewing.
Another problem is that there is an increasing number of programs with PDF output that are not strict about even some fairly well-established parts of the spec. The developers of these programs probably test the output only in Acrord32, if they test it at all. That's like declaring a web site well-coded if it looks okay in IE.
I'm not sure what to do about either of these problems, except to suggest greater community investment in cross-platform FLOSS PDF readers, writers, validators and plug-ins.
Sam