In this study, the effect of surface roughness on convective heat transfer in two-dimensional parallel plate microchannels is analyzed numerically for steady-state, single-phase, developing, and laminar air flow in the slip flow regime. Slip velocity and temperature jump at wall boundaries are imposed to observe the rarefaction effect. The effect of triangular roughness elements on Nusselt number are compared to cases with smooth surfaces. The results indicate that increasing surface roughness reduces heat transfer in continuum. However, in slip flow regime, an increase in the Nusselt number with increasing roughness height is observed; this increase being more pronounced at low rarefied flows. It is also found that the presence of axial conduction and viscous dissipation have increasing effects on heat transfer in smooth and rough channels, compared to cases where they are neglected.