Probably the greatest amount of research on the long-term effects of soy has to do with soy infant formulas. Some kids can't tolerate regular formula and get switched to formula derived from soybeans and they'll drink this stuff from infancy to one-year of age and beyond. Some kids are raised on soy milk instead of cow's milk their whole lives. A 2004 Italian study of kids raised on soy protein formulas showed no gynecomastia, no early puberty, no changes in their bones and no other signs of screwed up hormones.
I should point out that searching "soy and gynecomastia" in the National Institute of Health database only spews out three articles, but you'll be happy to know that the reproductive tracts of male macaques were not adversely affected by ingesting soy. There is one interesting case report of a 60-year-old man who developed big ol' hooters and presented to a gland clinic for an answer. His gynecomastia eventually went away when they discovered he was drinking 3/4 of a GALLON of soy milk A DAY and told him to cut it out. With all the millions of people who ingest soy daily (there are countries where soy is a staple of the "normal" diet) the fact that there is one case report of one guy whose breast tissue was sensitive to soy products is reassuring to me.
By Dr. Steve at Men's Fitness - http://www.mensfitness.com/nutrition/339