Actually the problem is probably quite the opposite.
It is likely that you have installed the drivers from matrox over the
top of the ones in the debian xfree86 packages. These files are not
conffiles, thusly your changes to them are not preserved.
If you install the drivers from matrox, you get mga_hal.o as well as
mga.o.
mga_hal is not in the debian xfree86 packages, because it is
gratuitously non-free, and only available in binary form.
When xfree86 is next upgraded, mga.o is overwritten with the version
in the xfree86 package. mga_hal.o is untouched, since it is not in any
package. You now have different versions of mga.o and mga_hal.o; this
invariably causes the server to segfault.