Acute caffeine intake can delay sleep initiation and reduce sleep intensity, particularly when consumed in the evening. However, it is not clear whether these sleep disturbances disappear when caffeine is continuously consumed during daytime, which is common for most coffee drinkers. To address this question, we investigated the sleep of twenty male young habitual caffeine consumers during a double-blind, randomized, crossover study including three 10-day conditions: caffeine (3 × 150 mg caffeine daily), withdrawal (3 × 150 mg caffeine for 8 days, then switch to placebo), and placebo (3 × placebo daily). After 9 days of continuous treatment, electroencephalographically (EEG)-derived sleep structure and intensity were recorded during a scheduled 8-h nighttime sleep episode starting 8 (caffeine condition) and 15 h (withdrawal condition) after the last caffeine intake. Upon scheduled wake-up time, subjective sleep quality and caffeine withdrawal symptoms were assessed. Unexpectedly, neither polysomnography-derived total sleep time, sleep latency, sleep architecture nor subjective sleep quality differed among placebo, caffeine, and withdrawal conditions. Nevertheless, EEG power density in the sigma frequencies (12–16 Hz) during non-rapid eye movement sleep was reduced in both caffeine and withdrawal conditions when compared to placebo. These results indicate that daily caffeine intake in the morning and afternoon hours does not strongly impair nighttime sleep structure nor subjective sleep quality in healthy good sleepers who regularly consume caffeine. The reduced EEG power density in the sigma range might represent early signs of overnight withdrawal from the continuous presence of the stimulant during the day.