We report in situ neutron background measurements at the Kuo-Sheng Reactor Neutrino Laboratory (KSNL) by a hybrid neutron detector (HND) with a data size of 33.8 days under shielding configurations identical to those used during the neutrino physics data taking. The HND consists of BC-501A liquid and BC-702 phosphor powder scintillation neutron detectors, which are sensitive to both fast and thermal neutrons, respectively. Neutron-induced events for the two channels are identified and differentiated by pulse shape analysis, such that the backgrounds of both are simultaneously measured. The fast neutron fluxes are derived by an iterative unfolding algorithm. Neutron-induced backgrounds in the germanium detector under the same fluxes, due to both cosmic rays and ambient radioactivity, are derived and compared with the measurements. The results are valuable to background understanding of the neutrino data at the KSNL. In particular, neutron-induced background events due to ambient radioactivity as well as from reactor operation are negligible compared to intrinsic cosmogenic activity and ambient gamma activity. The detector concept and analysis procedures are applicable to neutron background characterization in similar rare-event experiments.