Don't be too sensitive. Thinking too much will hurt yourself. The speaker is unintentional, but the listener is intentional. Don't you feel tired if you have to think about a casual word? Many things are remembered by the listener, but the speaker has long forgotten.
