You are confusing limited fair use rights (which only exist in some jurisdictions) with substantial rights to copy and modify a work.
WelltThat may be true, but in the US one cannot commit copyright infringment by simply modifying a tangible copy of a work, only by copying it. (After all, if you bought the book, why should the author be aple to prevent you from annotating it? Fist sale should also allow you to sell your commented copy.) Also in general (classical copyright law) there are no real restrictions on private copying, except in the case where you dispose of the original. In that case you must dispose of the copies in the exact same way, or destroy said copies.