Agile software development methodologies have been widely adopted by the software industry during the last decade. Agility assessment is an approach to measure the success of this adoption as well as to satisfy the further demands. In response, a number of agile maturity self-assessment surveys have been developed. However, software organizations do not widely utilize existing self-assessment surveys. In this study we aim to identify the existing surveys and evaluate their strengths and weaknesses in agility assessment. Firstly, an exploratory case study is performed by using one of the surveys in order to determine the expected features of an agile maturity self assessment survey. Then, a comprehensive case study is conducted to measure the sufficiency of the mostly referenced agile maturity self-assessment surveys according to the determined features. The study results reveal that current surveys do not meet the expected features comprehensively.