I came home from work one day to be greeted by two cats in the window waiting for me. Aww.
Upon entering ‘my’ house I found two more cats on ‘my’ sofa that I had never seen before. And they didn’t move, just lifted their heads to acknowledge my presence and then went back to sleep…
This is fine.
Upon entering the kitchen there were two more cats on the rear window board looking out in to the back garden. They turned to look at me and then ignored me.
Fine.

After feeding them all ‘my’ two cats went out in to the garden and were quickly followed by the rest of them. About an hour later the usual two cats came back in with one of the newer ones and they all curled up on the sofa with me.
Fine.

The new one stayed for a few days until I saw a post about a missing cat. Contacted them and, yep, he was from a few streets away.
Never had that happen again.