I've gotten 100% completion on it and I don't think it's the time limit that makes it difficult. The time limit is very generous when you slow things down. However it is a sequel to Ocarina of Time and the dungeons seem to be a progression in difficulty from that game. For example I think the Great Bay temple is tougher to figure out than the Water Temple.
Also since it's so heavy on side quests, many of the rewards that benefit you during the story are locked around figuring the side stuff out.
I would agree that Majora’s Mask is the hardest, even the 3DS version. The 3-day mechanic does make it hard, but what makes that harder is that it applies to dungeons as well (where in Ocarina of Time, time doesn’t move when you’re inside a dungeon). But this wouldn’t have mattered if it weren’t for collecting the missing fairies in a dungeon that make a Great Fairy. This leaves you with no room for error.
Twilight Princess. I first started playing it in the 6th grade, and I've been stuck in the first half of the game the whole time. I've only done the first three dungeons, but they were all VERY difficult for me. Don't even get me started on the Lakebed Temple... Not to mention all the backtracking and the overall amount of content in the game. I don't foresee myself beating this game any time soon.
I agree that Majora's Mask is also pretty difficult, given the 3-day time limit, but it wasn't a huge problem for me. I struggled with the Snowhead and Great Bay temples, but surprisingly found the Stone Tower Temple to be fairly easy. I've only done the main quest though, and I haven't touched most of the side quests, so I'd imagine 100% completion would be pretty daunting.
Ocarina of Time was the easiest 3D Zelda game for me. While I did get stuck on certain parts of the game, it was smooth sailing otherwise. I actually beat the game (the 3D version, at least) five times.
I haven't played Wind Waker, and I didn't advance very far in Skyward Sword, so no comment for either.
I haven’t played these myself, but according to my hubby (who is a big fan & has played them all), his answer is Twilight Princess. When I asked him why, he simply answered… “because it’s hard.” A man of few words, that one.