Pediatric bipolar disorder (BD) and unipolar disorder (UD) share common symptomatic and functional impairments. Various brain imaging techniques have been used to investigate the integrity of brain white matter (WM) and gray matter (GM) in these disorders. Despite promising preliminary findings, it is still unclear whether these alterations may be considered as common trait markers or may be used to distinguish BD from UD. A systematic literature search of studies between 1980 and September 2013 which reported WM/GM changes in pediatric and adolescent BD/UD, as detected by diffusion tensor imaging and voxel-based analysis was conducted. Of the 34 articles judged as eligible, 17 fulfilled our inclusion criteria and were finally retained in this review. More abnormalities have been documented in the brains of children and adolescents with BD than UD. Reductions in the volume of basal ganglia and the hippocampus appeared more specific for pediatric UD, whereas reduced corpus callosum volume and increased rates of deep WM hyperintensities were more specific for pediatric BD. Seminal papers failed to address the possibility that the differences between unipolar and bipolar samples might be related to illness severity, medication status, comorbidity or diagnosis. UD and BD present both shared and distinctive impairments in the WM and GM compartments. More WM abnormalities have been reported in children and adolescents with bipolar disease than in those with unipolar disease, maybe as a result of a low number of DTI studies in pediatric UD. Future longitudinal studies should investigate whether neurodevelopmental changes are diagnosis-specific.