© 2016 IEEE.In this paper, we provide a comprehensive analysis of the error floor along with code optimization guidelines for structured and regular non-binary low-density parity-check (NB-LDPC) codes in magnetic recording (MR) applications. While the topic of the error floor performance of binary LDPC codes over additive white Gaussian noise (AWGN) channels has recently received considerable attention, very little is known about the error floor performance of NB-LDPC codes over other types of channels, despite the early results demonstrating superior characteristics of NB-LDPC codes relative to their binary counterparts. We first show that, due to the outer looping between the detector and the decoder in the receiver, the error profile of NB-LDPC codes over partial-response (PR) channels is qualitatively different from the error profile over AWGN channels-this observation motivates us to introduce new combinatorial objects aimed at capturing decoding errors that dominate the PR channel error floor region. We call these objects balanced absorbing sets (BASs), which are viewed as a special subclass of previously introduced absorbing sets (ASs). Aided by these new objects (BASs), we develop a method that combines analytical equations and biased simulations to predict the error floor performance of NB-LDPC codes over PR channels without the need to execute extensive Monte Carlo (MC) simulations. We show that explicitly incorporating the inter-symbol interference of MR channels into our prediction method makes the accuracy of the error floor estimate within 0.2 of an order of magnitude from the traditional MC simulation. In addition, we prove that, due to the more restrictive definition of BASs (relative to the more general class of ASs), an additional degree of freedom can be exploited in the code design for PR channels. We then demonstrate that the proposed code optimization aimed at removing dominant BASs offers performance improvements in the frame error rate in the error floor region by up to 2.5 orders of magnitude over the unoptimized designs. Our code optimization technique carefully, yet provably, removes BASs from the code while preserving its overall structure (node degree, quasi-cyclic property, regularity, and so forth). The resulting codes outperform the existing binary and NB-LDPC solutions for PR channels by about 2.5 and 1.25 orders of magnitude, respectively.