A Quick Clarification

When inviting somebody to something, there are subtleties that ought to be noticed. One that bothers me is the wording of the invite. Yes, there is a difference. When you hear "Do you want to go?", it is different than when you hear "Do you want to come with?" Although both qualify as invitations, one is slightly more...oh nevermind. If you can't figure this one out, shame on you.