Fair points you make there Neo and I believe this is the case for you, I have seen similar reports elsewhere too. Still I say that the FF technology itself doesn't slow you down, atleast for me it gives the tiny bit of more info which allows me to setup the car more aggressively and thus faster laptimes.
There are fine wheels in the market, G25 included, these games are designed to drive with Force Feedback on and one should not let the FF go just because of fear that its slower way around the track
After all some like Father more than Mother and vice versa