By using the first order autoregressive time series model, rotation histories of accretion-powered X-ray pulsars are modeled yielding a torque noise strength S(in) and the average secular spin-up term [OMEGA]. We find that Her X-1, Cen X-3, A0535 + 26, and Vela X-1 can be modeled with white torque-noise leading to random walks in pulse frequency; for the remaining sources the random walk model is also acceptable. Using this model we calculate the noise strengths in the frequency derivative and estimate the remaining secular angular accelerations. Subsequently we compare noise strength parameters with the X-ray luminosities L(x) and the secular angular accelerations. We do not see any distinction in these correlations that separate wind accreting sources from disk accreting sources. In the absence of any clear evidence for the two types of accretion, we suggest a simple scenario for the pulse frequency fluctuations in terms of episodes of mass transfer with positive and negative torques. The data shows that sources with L(x) > 10(37) erg s-1, spin-up on the average even though they experience a large amount of reverse torques. We do not see any significant secular spin-up/down behaviour for most of the sources with L(x) < 10(37) erg s-1.