Derby succumbed to a 4-1 defeat at home to Leeds United on Saturday evening, but the Rams will be the favourites to prevail when they face League Two side Oldham on Tuesday night.
Lampard is expected to make changes for the fixture at Boundary Park, but the Chelsea legend has suggested that he wants his team to have a prolonged run in a cup competition this season.
The 40-year-old is quoted by the Daily Mail as saying: "I've watched Oldham and know they will be well-organised, physically good and if we aren't at it, they can hurt us, without a doubt. It's a potential banana skin and it's up to us to approach the game with the right attitude and how we play to make sure we get the right result.
"We know a cup run is good for the club, where it can potentially take us through to play Premier League teams and see where we really are at. The further you can get in it, the better for our confidence. We will give it everything to try and win."
Lampard won the EFL Cup on two occasions as a player.