The 54-year-old chef - who has children Megan, 22, twins Jack and Holly, 20, Matilda, 19, and Oscar, 19 months, with wife Tana - is tired of people thinking his youngest child actually belongs to his older son and that he's the grandparent when they are all out together.
He moaned: "The worst thing is when I'm walking with Tana and Jack, and everyone is looking at Jack as if Oscar is his son. 'You must be so proud to be a grandad.' 'F*** off, he's ours.'
"If I get called 'grandad' one more f******time, I'm going to hit the roof."
Gordon is known for his passion for fitness and he's delighted his children have followed in his footsteps when it comes to endurance events.
He told the new issue of Britain's Men's Health magazine: "My son Jack did his first Ironman 70.3 last year. I got all the kids entrance into the London Marathon, raising money for Great Ormond Street Children's Hospital. Tana is going to commit to her 10th marathon. I've got my 15th London coming up, two LAs and five ultras. It's starting to rub off on them."
The Hell's Kitchen star finds cycling very uncomfortable - and admitted he was left fruitlessly searching for a treatment after misunderstanding some advice from Olympian Mark Cavendish.
[[nid:502832]]
He said: "I remember saying to Mark Cavendish, 'F*** me, that piece between my ball bag and my a*******! I've never got over how painful it is on the saddle.'
"He said, 'You've got to get rhinoceros skin.' So, there's me thinking there's a gel out there called 'Rhinoceros Skin.'"
When he's training for an event, Gordon would never tuck into protein bars for fuel because he thinks they are "f****** disgusting".
He said: "I'm very lucky that I can train and still eat pretty much what I want. How can a chef get into protein bars and say they taste delicious, when they taste f****** disgusting?"