I'm sure you have often heard people putting down the United States by saying that it was founded by old white men. However, when the Declaration was signed, the median age of the signers was 44, so half of them were below 40, and a few were in their 20s. Only 17 of them would be eligible to join this forum, and only two were older than me. You might have to zoom in your screen to read it, but here it is.
Charts are charts and images are images. It probably has to do with the power of images. Just look at Pine's or Trumbull's paintings. What they show is a meeting of old men most of them with grey wigs and all dressed in an old-fashioned way. And even those with their natural hair color would not come across as "young". Viewers would not check their age either. In addition, politics that is not politics of the street is always thought to be done by old men. All of those are coming together and convey the impression you describe.
It makes me want to know more about the ones I never heard about like the two "kids" at 26 years old like, namely Edward Rutledge and Thomas Lynch.
In a way, since the life expectancy was around 36 years of age in the North American 1700’s, I guess the majority really were kinda old. From what I have read, most children didn’t even get named until they were around 2 years of age.