As genetic factors can hardly explain the changes taking place during short time spans, environmental and lifestyle-related factors have been suggested as the causes of time-related deterioration of male reproductive function. However, considering the strong heterogeneity of male fecundity between and within populations, genetic variants might be important determinants of the individual susceptibility to the adverse effects of environment or lifestyle. Although the possible mechanisms of such interplay in relation to the reproductive system are largely unknown, some recent studies have indicated that specific genotypes may confer a larger risk of male reproductive disorders following certain exposures. This paper presents a critical review of animal and human evidence on how genes may modify environmental effects on male reproductive function. Some examples have been found that support this mechanism, but the number of studies is still limited. This type of interaction studies may improve our understanding of normal physiology and help us to identify the risk factors to male reproductive malfunction. We also shortly discuss other aspects of gene-environment interaction specifically associated with the issue of reproduction, namely environmental and lifestyle factors as the cause of sperm DNA damage. It remains to be investigated to what extent such genetic changes, by natural conception or through the use of assisted reproductive techniques, are transmitted to the next generation, thereby causing increased morbidity in the offspring.