Costa netted twice - the first of which came after just 49 seconds - to drag the game into extra time, when Saul Niguez and Koke both scored to wrap up a 4-2 victory over the Champions League holders.
The triumph was Atletico's first European final win over their city rivals - having lost two Champions League finals to them in the past five years - and Costa talked up the importance of the result.
"With the desire and the 'cojones' that we have, we can fight against any team. It was a victory that we had been looking for a long time, I am very happy," the former Chelsea striker told reporters.
"They won two finals against us, we wanted to win a European final against them. It was very important to get confidence ahead of the season.
"Atletico have always had an ambitious project, we can get anywhere with work, humility. We can fight with any team, we have signed very strong and I hope we keep growing."
Costa's opening goal in Tallinn was the quickest ever scored in the 43-year history of the UEFA Super Cup.