problem with xmodmap

[Luis Gustavo Madrigal Salazar <tavo@scratchy.emate.ucr.ac.cr>]

I'm trying to get my Intellimouse Explorer working with XFree 4.0 My
XF86Config
file has this:  
Section "InputDevice" 
        Identifier "Mouse0" 
        Driver "mouse" 
        Option "Protocol" "ExplorerPS/2" 
        Option      "Device" "/dev/mouse" 
        Option "Buttons" "7" 
        Option "Resolution" "200" 
        Option      "ZAxisMapping" "6 7"
EndSection

I think this works well, but my problem arrives when I put this  on my
.xsession file:
xmodmap -e "pointer = 1 2 3 6 7 4 5" 

With this stuff I can't login with kdm....it kicks me out!!
How can I fix this?