Lennon's side dominated large parts of the first half but were unable to convert their ascendancy into goals.
Paulo Sergio's Hearts then struck twice after the break at Tynecastle, while Celtic midfielder Kris Commons was handed his marching orders.
"I am disappointed with our play in the final third today," the manager told BBC Radio Scotland.
"We were in control of the game when Hearts scored. We missed simple clear-cut chances in the first half. We lost the goal and then the discipline and I cannot accept that."
Celtic trail Scottish Premier League leaders Glasgow Rangers by ten points.