For me, it's definitely Canary Mary. On those last two races in Cloud Cuckoland, she made my thumb get all numb and got several scratches on it. Even though I can now beat her on the first tries, I still loath her to death. However, I like an excellent challenge. Although...
Even though I absolutely hate her, I suggest Rare to put her back in the game to annoy me once more. That human-like bird really makes my day by giving the hardest racing challenge I had ever encountered in a game.
She's one of the reasons to why I love Banjo-Tooie so much, in all honesty.
But please... For the love of dear God, I absolutely forbid Rare to make me press a little, retarded A button for like, 10 minutes straight. Instead, make it 20. With allowed pauses as much as I like for a good rest... Purty' please?