Event recognition is probably the ultimate purpose of an automated surveillance system. In this paper, hidden Markov models (HMM) are utilized to recognize the nature of an event occurring in a scene. For this purpose, object trajectories, which are obtained through a successful track, are written as a sequence of flow vectors that contain instantaneous velocity and location information. These vectors are clustered by K-means algorithm to obtain a prototype representation. HMMs are trained with sequences obtained from usual motion patterns and abnormality is detected by measuring distances to these models. In order to specify the number of models without user interaction, a novel. approach is proposed in which the clues provided by centroid clustering are utilized. Promising simulation results are obtained for this approach, which is. applicable to any surveillance application.