Defender Jordan Obita, with his first goal for 30 months, gave Reading the lead with a clinical strike after nine minutes.
As the home side dominated, striker Sam Baldock made it 2-0 in the 37th minute with another clean effort from distance.
Millwall reduced the deficit in the second half when Jed Wallace cleverly slid home his sixth goal of the season.
Bowen replaced the sacked Jose Gomes last month and had led Reading to a 1-0 home win over Preston North End and a 2-2 draw at QPR.
Gary Rowett took over as Millwall boss after the long-serving Neil Harris resigned last month.
Rowett's first match in charge produced a 2-0 win over Stoke, his former club, at The Den.
Millwall made the early running, with midfielder Shaun Williams at the heart of most of their forward moves.
From a precise Williams corner, Millwall captain Shaun Hutchinson should have done better – only to head weakly wide.
Reading weathered the pressure and went in front in the ninth minute.
John Swift's corner was deflected for a throw-in and, from skipper Liam Moore's long throw, the Millwall defence partially cleared it.
Obita, the left wing back only recently returned from almost two years out with knee problems, unleashed a superb 20-yard angled drive that flew past goalkeeper Bartosz Bialkowski.
Millwall replied through Wallace but he blazed wildly over from long distance.
Reading soon resumed control and Swift could have extended their lead from a free kick way out on the left.
He audaciously went for goal but, with Bialkowski struggling to reach it, the ball struck the crossbar and rebounded to safety.
Bialkowski kept Millwall in contention with a stunning save to keep out Matt Miazga's glancing header from a Swift corner.
But the keeper stood little chance when Baldock, set free by Ovie Ejaria, rifled home from just outside the area eight minutes before the break.
Rowett brought on striker Jon Dadi Bodvarsson, who moved from Reading to Millwall during the summer, at half time.
But the home side maintained their tempo from with first period, with Baldock scuffing an effort wide after good work by Swift.
Andy Rinomhota then ran through from midfield but drove over the bar.
Millwall gradually reasserted themselves and were rewarded in the 63rd minute when they reduced the gap.
Wallace weaved his way past several poor challenges on the right, cut inside and then beat keeper Rafael with a low shot.
Millwall pressed towards the end but could not find an equaliser.