In this paper, we propose an approach for modeling claim dependence, with the assumption that the claim numbers and the aggregate claim amounts are mutually and serially dependent through an underlying hidden state and can be characterized by a hidden finite state Markov chain using bivariate Hidden Markov Model (BHMM). We construct three different BHMMs, namely Poisson-Normal HMM, Poisson-Gamma HMM, and Negative Binomial- Gamma HMM, stemming from the most commonly used distributions in insurance studies. Expectation Maximization algorithm is implemented and for the maximization of the state-dependent part of log-likelihood of BHMMs, the estimates are derived analytically. To illustrate the proposed model, motor third-party liability claims in Istanbul, Turkey, are employed in the frame of Poisson-Normal HMM under a different number of states. In addition, we derive the forecast distribution, calculate state predictions, and determine the most likely sequence of states. The results indicate that the dependence under indirect factors can be captured in terms of different states, namely low, medium, and high states.