Multi-frame knowledge based text enhancement for mobile phone captured videos

Ozarslan S., EREN P. E.

Conference on Mobile Devices and Multimedia - Enabling Technologies, Algorithms, and Applications, San-Francisco, Costa Rica, 3 - 05 February 2014, vol.9030 identifier identifier

  • Publication Type: Conference Paper / Full Text
  • Volume: 9030
  • Doi Number: 10.1117/12.2040606
  • City: San-Francisco
  • Country: Costa Rica
  • Middle East Technical University Affiliated: Yes


In this study, we explore automated text recognition and enhancement using mobile phone captured videos of store receipts. We propose a method which includes Optical Character Resolution ( OCR) enhanced by our proposed Row Based Multiple Frame Integration (RB-MFI), and Knowledge Based Correction (KBC) algorithms. In this method, first, the trained OCR engine is used for recognition; then, the RB-MFI algorithm is applied to the output of the OCR. The RB-MFI algorithm determines and combines the most accurate rows of the text outputs extracted by using OCR from multiple frames of the video. After RB-MFI, KBC algorithm is applied to these rows to correct erroneous characters. Results of the experiments show that the proposed video-based approach which includes the RB-MFI and the KBC algorithm increases the word character recognition rate to 95%, and the character recognition rate to 98%.