It generally isn't. Except for if you happen to say that your public address is the same as the address of a wanted criminal group. To avoid that you could use a sub-address or better yet, another wallet (with a different key).
If you are being targeted, with a high risk profile, someone could try to link your activity on different sub-addresses with an EAE attack. That's why it's better to isolate activities with different wallets instead of just sub-addresses.
Please keep up the good work!
Ignore the downvotes, they aren't from this instance