JOURNAL OF HIGH ENERGY PHYSICS, no.6, 2018 (Journal Indexed in SCI)
A search for lepton flavour violating decays of the Higgs boson in the mu tau and e tau decay modes is presented. The search is based on a data set corresponding to an integrated luminosity of 35.9 fb(-1) of proton-proton collisions collected with the CMS detector in 2016, at a centre-of-mass energy of 13 TeV. No significant excess over the standard model expectation is observed. The observed (expected) upper limits on the lepton flavour violating branching fractions of the Higgs boson are B(H -> mu tau) < 0.25% (0.25%) and B(H -> e tau) < 0.61% (0.37%), at 95% confidence level. These results are used to derive upper limits on the off-diagonal mu tau and e tau Yukawa couplings root vertical bar Y-mu tau vertical bar(2) + vertical bar Y-mu tau vertical bar(2) < 1.43 x 10(-3) and root vertical bar Y-e tau vertical bar(2) + vertical bar Y-tau e vertical bar(2) < 2.26 x 10(-3) at 95% confidence level. The limits on the lepton flavour violating branching fractions of the Higgs boson and on the associated Yukawa couplings are the most stringent to date.