Maintenance of vigorous exercise habits from young to old age is considered protective against hip fractures, but data on fracture risk in lifelong vigorous exercisers are lacking. This longitudinal cohort study examined the hazard of hip fractures in 1844 male former athletes and 1216 population controls and in relation to exercise volume and intensity in later years. Incident hip fractures after age 50 were identified from hospital discharge register from 1972 to 2015. Exercise and covariate information was obtained from questionnaires administered in 1985, 1995, 2001 and 2008. Analyses were conducted using extended proportional hazards regression model for time-dependent exposures and effects. During the mean (SD) follow-up of 21.6 (10.3) years, 62 (3.4%) athletes and 38 (3.1%) controls sustained a hip fracture. Adjusted hazard ratio, HR, indicated no statistically significantly difference between athletes and controls (0.84; 95% CI, 0.55, 1.29). In subgroup analyses, adjusted HRs for athletes with recent high (≥ 15 MET-h/wk) and low (< 15 MET-h/wk) exercise volume were 0.83 (95% CI, 0.46, 1.48) and 1.04 (95% CI, 0.57, 1.87), respectively, compared with controls. The adjusted HR was not statistically significant between athletes with low-intensity exercise (< 6 METs) and controls (1.08; 95% CI, 0.62, 1.85). Athletes engaging in vigorous-intensity exercise (≥ 6 METs at least 75 min/wk) had initially 77% lower hazard rate (adjusted HR, 0.23; 95% CI, 0.06, 0.86) than controls. However, the hazard ratio was time-dependent (adjusted HR, 1.04; 95% CI, 1.01, 1.07); by age 75 years the hazards for the athletes with vigorous-intensity exercise reached the level of the controls, but after 85 years the hazards for these athletes increased approximately 1.3-fold annually relative to the controls. In conclusion, these data suggest that continuation of vigorous-intensity exercise is associated with lower hazard rate of hip fracture up to old age.