Adding the first user to group audio shouldn't break any ACL management that is done by systemd-logind.
logind hide/ show the audio device from the user trough the ACL. The audio group will always show all audio devices. I think these two settings are conflicting. floris