We present a video shot boundary detection algorithm based on the novel graph theoretic concept, namely dominant sets. Dominant sets are defined as a set of the nodes in a graph, mostly similar to each other and dissimilar to the others. In order to achieve this goal, candidate shot boundaries are determined by using simply pixelwise differences between consequent frames. For each candidate position, a testing sequence is constructed by considering 4 frames before the candidate position and 2 frames after the candidate position. Proposed method works on a weighted undirected graph, where the graphs are established by using the frames in the testing sequence. Each frame in the sequence corresponds to a node in the graph, whereas edge weights between the nodes are calculated by using pairwise similarities of frames. By utilizing the complete information of the graph, its dominant set is detected. The simulation results indicate that the proposed algorithm can be a promising approach for abrupt shot boundary detection.