Protection of visual privacy has become an indispensable component of video surveillance systems due to pervasive use of video cameras for surveillance purposes. In this paper, we propose two fully reversible privacy protection schemes implemented within the JPEG architecture. In both schemes, privacy protection is accomplished by using false colors with the first scheme being adaptable to other privacy protection filters while the second is false color-specific. Both schemes support either a lossless mode in which the original unprotected content can be fully extracted or a lossy mode, which limits file size while still maintaining intelligibility. Our method is not region-of-interest (ROI)-based and can be applied on entire frames without compromising intelligibility. This frees the user from having to define ROIs and improves security as tracking ROIs under dynamic content may fail, exposing sensitive information. Our experimental results indicate the favorability of our method over other commonly used solutions to protect visual privacy.