Not all young people are small and agile. Some are huge and/or bulky.
But the idea is ok, just not in the way you explained it.
Personaly the main difference should be, that old people take you more seriously when your old, but young people accept you sooner when your in their age.