Engineers usually focus on the performance of structural members, whereas the occupants of a residential building are affected mostly by the performance of infill and partition walls in buildings after a moderate earthquake. This often creates controversy and discussion regarding the post-earthquake use of buildings. Seismic rehabilitation codes for existing buildings offer sophisticated measures in rating the seismic performances of structural components, whereas performance measures suggested for infill and other partition walls are crude by comparison. Furthermore, seismic design codes for new buildings totally disregard such disparity, since their force-based approaches are built on single-level performance targets specified implicitly for the entire building under a design level, that is, a rare earthquake. In this paper, performance levels of buildings after an earthquake of moderate intensity are discussed from the viewpoints of engineers and building occupants. Suggestions are made for achieving uniform performance in structures where the seismic forces are resisted by structural members as well as the infills and partition walls coupling with the structural system although the contribution of such walls to seismic resistance and their performance is not usually considered in design.