there is a different in reading our partner thought than "guessing" our partner thought based on past life experience that you have been through together. and for some people it is not only about privacy, but its more about belief that the mind reader is actually controlling you and you will start questioning yourself whether it is inhuman or not.
however you have a good point of view there, there will always be two sides of a coin