Studies in both pathological and healthy samples have suggested altered functional connectivity between orbitofrontal cortex (OFC) and amygdala as a possible cause of anger and aggression. In patient populations presenting with pathological aggression, there is also evidence for changes in structural connectivity between OFC and amygdala. In healthy samples, however, the relationship between white matter integrity and aggression has not been studied to date. Here, we investigated the relationship between trait aggressiveness and structural OFC-amygdala connectivity in a large sample (n = 93) of healthy young men. Using diffusion tensor imaging, we measured the distribution of fractional anisotropy and mean diffusivity along the uncinate fascicle bilaterally. We found no differences in either measure between participants high and low in physical aggressiveness, or between those high and low in trait anger. Our results therefore argue against a direct relationship between structural OFC-amygdala connectivity and normal-range trait aggressiveness.