This study focuses on the damage potential of earthquake ground motions based on the inelastic dynamic response of equivalent single degree of freedom structures. Their yield resistances are selected in accordance with seismic design codes. An index accounting for the accumulation of damage due to inelastic excursions is used to represent structural damage. A set of 94 ground motions are employed for this analysis, which are all scaled to the same peak ground acceleration of 0.4 g. Earthquake ground motions are classified with respect to the peak velocity. The effect of peak velocity on damage potential is investigated by using sensitivity analysis and probabilistic techniques. It is concluded that peak ground velocity significantly influences the damage potential of earthquake ground motions, although it is not directly represented by the spectral definitions of earthquake excitations in seismic design codes.