Spurs were reduced to 10 men on the half-hour mark when Kyle Naughton was shown a straight red card for handball inside the box, but the hosts failed to take full advantage as Mark Noble rolled his penalty wide.
The game seemed to be heading for a goalless draw until deep into stoppage time, when Spurs defender Eric Dier marked his debut in style by rounding the keeper and slotting the ball home.
Here, Sports Mole takes an in-depth look at an action-packed 90 minutes in East London.
On target: 4
On target: 4
Was the result fair?
It is tough to truly analyse a match that saw a late goal and two players sent off, but over the whole 90 minutes this was not a fair result. West Ham created the best chances and could have put the game to bed before half time.
Not only did the Hammers squander a penalty, they also missed a number of other clear chances before the break as they looked to take advantage of their extra man. Tottenham, meanwhile, were left holding on at times and would have been delighted to reach the break all square.
Even after the Collins red card, the hosts had chances, with Hugo Lloris brilliantly denying Stewart Downing shortly before the winner. Dier's goal was by far Tottenham's best opening of the match, and it was that clinical nature that proved to be the difference this afternoon.
West Ham's performance
The board called on Sam Allardyce to provide a more entertaining spectacle with some free-flowing football, and there were glimpses of that this afternoon. As usual, the hosts' biggest threat came aerially as Spurs struggled to deal with crosses into the box, but there were times when West Ham passed the ball nicely as well.
Of course, that may have been aided by having an extra man for more than half an hour, and during that period of time the Hammers really should have done enough to win the match. They created enough chances, but just lacked the finishing touch that Spurs applied late on.
To be fair to West Ham, they did push for an opener towards the end of the first half and were on top before Collins's red card evened things up again. They are unfortunate to have lost this game and there are plenty of positives for Allardyce to take, but they only have themselves to blame for not winning today.
Despite the result, Tottenham fans may not be too excited by what they saw from their side today. Mauricio Pochettino had his Southampton side playing some lovely football last season, but there was not a lot of that from his new charges at Upton Park this afternoon.
Admittedly, a match in which they were down to 10 men for more than an hour is not the best opportunity to judge the team, but even before the sending off they were under the cosh. They were able to get back into the match a bit following Collins's red, but will consider themselves lucky to have come away with all three points.
In fairness, Pochettino was missing a lot of players and the team is likely to adapt to his philosophy as the season progresses. Overall it was not a great performance this afternoon, but the fact that they have come away with three points when well short of their best is encouraging for Spurs.
Sports Mole's man of the match
Stewart Downing: A menace all afternoon for West Ham, bombing down the right flank and giving Danny Rose a really tough time in the process. Downing put a number of very dangerous deliveries into the box, but was let down more than once by poor finishing.
The former Liverpool winger had a chance of his own late on, only to be denied by a fabulous stop from Lloris. He did not deserve to be on the losing side this afternoon but can feel justifiably happy with his own display.
This one has to go to Noble for his penalty miss. Usually so reliable from 12 yards, the midfielder seemed to have second thoughts on his run-up and simply rolled the ball wide of the mark. He had sent Lloris the wrong way but couldn't find the bottom corner as Spurs were let off the hook. Dier's late goal makes the miss even more costly.
It was a busy afternoon for Chris Hoy on the opening weekend of the Premier League season. He showed two red cards, both of contrasting fairness. Naughton's dismissal was very harsh as the Tottenham man was too close to Kevin Nolan to have moved his arms out of the way of the shot, although it appeared to be the linesman who had the biggest say in that spot kick and red card.
The Collins second yellow was more straightforward as the defender stepped across the path of Adebayor to block him off. It is debateable whether the Tottenham striker would have collected the ball on the other side, but Collins didn't give him the chance and was deservedly booked for a second time. His first yellow was also the correct decision.
West Ham: Next up for the Hammers is another London derby as they travel to Selhurst Park to take on Crystal Palace.
Tottenham: Spurs, meanwhile, are back in European action against AEL Limassol in the Europa League this Thursday.