Potter’s troubled side crashed to a seventh defeat in their last 10 games in all competitions.
Chelsea fell behind to a goal from their former winger Willian in the first half at Craven Cottage.
Kalidou Koulibaly equalised after the break but Felix was dismissed for a reckless studs-up lunge on Kenny Tete before Carlos Vinicius headed the late winner after Kepa’s mistake.