This sounds tempting. Upstreams of quite a bunch of packages are well-behaved and could be trusted to produce non-breaking updates, at least for patch and minor versions. What worries me are license changes and security issues. While the former could be formally detected (licensecheck), the entry threshold for potential backdoors would be lowered by auto-updates (man-in-the-middle and the like). I'd trust only GPG-signed release tarballs.
Best,
Andrius