Ordinal time series analysis is a new approach to the investigation of long and complex time series, which bases on ordinal patterns describing the order relations between the values of a time series. In this paper we consider ordinal time series analysis from the conceptional viewpoint. In particular, we introduce ordinal processes as models for ordinal time series analysis and discuss the structure of ordinal pattern distributions obtained from them. Special emphasis is on the relation of ordinal time series analysis to symbolic dynamics and to a transformation extracting the whole ordinal information contained in a time series. Finally, we consider invariance properties of ordinal time series analysis.