Gleason grading is the best independent predictor for prostate cancer (PCa) progression. Recently, a new PCa grading system has been introduced by the International Society of Urological Pathology (ISUP) and is recommended by the World Health Organization (WHO). Following studies observed more accurate and simplified grade stratification of the new system. Aim of this study was to compare the prognostic value of the new grade groups compared to the former Gleason Grading and to determine whether re-definition of Gleason Pattern 4 might reduce upgrading from prostate biopsy to radical prostatectomy (RP) specimen. A cohort of men undergoing RP from 2002 to 2015 at the Hospital of Goeppingen (Goeppingen, Germany) was used for this study. In total, 339 pre-operative prostatic biopsies and corresponding RP specimens, as well as additional 203 RP specimens were re-reviewed for Grade Groups according to the ISUP. Biochemical recurrence-free survival (BFS) after surgery was used as endpoint to analyze prognostic significance. Other clinicopathological data included TNM-stage and pre-operative PSA level. Kaplan-Meier analysis revealed risk stratification of patients based on both former Gleason Grading and ISUP Grade Groups, and was statistically significant using the log-rank test (p < 0.001). Both grading systems significantly correlated with TNM-stage and pre-operative PSA level (p < 0.001). Higher tumor grade in RP specimen compared to corresponding pre-operative biopsy was observed in 44 and 34.5% of cases considering former Gleason Grading and ISUP Grade Groups, respectively. Both, former Gleason Grading and ISUP Grade Groups predict survival when applied on tumors in prostatic biopsies as well as RP specimens. This is the first validation study on a large representative German community-based cohort to compare the former Gleason Grading with the recently introduced ISUP Grade Groups. Our data indicate that the ISUP Grade Groups do not improve predictive value of PCa grading and might be less sensitive in deciphering tumors with 3 + 4 and 4 + 3 pattern on RP specimen. However, the Grade Group system results less frequently in an upgrading from biopsy to the corresponding RP specimens, indicating a lower risk to miss potentially aggressive tumors not represented on biopsies.