SnooPets20

I have recorded a video, it's in the OP.

Ok, I just did. It's (global-display-line-numbers-mode). Now.... why???? It's odd as hell.

I just tested, it reports that I'm dragging the mouse and setting a region, but it's weird because if the issue is my clicking being sloppy, when it should happen in emacs with no config as well, and it doesn't.

      ;; mouse-drag-region
                 ;; mouse-set-region

This is when I click and the button doesn't get pressed.

   ;; mouse-drag-region
                   ;; Info-mouse-follow-nearest-node

And this is when it does register.

I can never reproduce the issue on emacs without config.

Nope! And when I tried with my config the very first click had that issue. It's very weird tho, I have no idea what could be causing it...

Nope, vanilla emacs in that respect.

Have you tried just mounting the remote folder with sshfs? Then you can just access it as a regular file, not just with emacs, with anything.