IEEEE-Health is becoming a vital industry and human activity recognition (HAR) is one of the most popular research areas of its scope. Although there are various studies on HAR, most of them come up with complex models that are not compatible with portable and wearable devices due to their limited computing capabilities. In this study, a new approach to data representation is presented with convolutional neural network architectures for high accuracy and lightweight activity detection. An anomaly detection framework is presented, which uses ECG data for the prediction of cardiac stress activities. The novel approach to data representation and the proposed deep learning model are tested on the MHEALTH dataset with two different validation techniques for accuracy and three different complexity metrics. The experimental results show that the proposed approaches can achieve up to 96.92% and 97.06% accuracy for the HAR and cardiac stress level, respectively. In addition, the models proposed for inertial data and ECG-based prediction are lighter than the existing approaches in the literature with sizes of 0.89 MB and 1.97 MB and complexities of 0.06 and 1.04 Giga FLOPS, respectively.