Magnetic Particle Imaging is a tomographic imaging technique that measures the voltage induced due to magnetization changes of magnetic nanoparticle distributions. The relationship between the received signal and the distribution of the nanoparticels is described by the system function. A common method for image reconstruction is using a measured system function to create a system matrix and set up a regularized linear system of equations. Since the measurement of the system matrix is time-consuming, different methods for acceleration have been proposed. These include modeling the system matrix or using a direct reconstruction method in time, known as X-space reconstruction. In this work, based on the simplified Langevin model of paramagnetism and certain approximations, a direct reconstruction technique for Magnetic Particle Imaging in the frequency domain with two- and three-dimensional Lissajous trajectory excitation is presented. The approach uses Chebyshev polynomials of second kind. During reconstruction, they are weighted with the frequency components of the voltage signal and additional factors and then summed up. To obtain the final nanoparticle distribution, this result is rescaled and deconvolved. It is shown that the approach works for both simulated data and real measurements. The obtained image quality is comparable to a modeled system matrix approach using the same simplified physical assumptions and no relaxation effects. The reconstruction of a 31 × 31 × 31 volume takes less than a second and is up to 25 times faster than the state-of-the-art Kaczmarz reconstruction. Besides, the derivation of the proposed method shows some new theoretical aspects of the system function and its well-known observed similarity to tensor products of Chebyshev polynomials of second kind.