Halo 2, no contest. The other games have tough sections or a level or two that's annoying or hard to play on Legendary. For example:
Halo 1 has the slog that is The Library and the Carrier-infested canyon sections of Keyes.
Halo 3 has Cortana and the first part of Halo.
Halo 4 has teleporting bullet-sponge Knights, and horrible gunner AI that makes vehicle sections a pain.
ODST has Buck's horrible gunner AI that loves to shoot dead Grunts instead of live Ghosts.
Reach (in the MCC at least) has the ridiculously aggressive Seraphs in Long Night of Solace and the Beachhead section of Exodus.
Out of all of these games, though, Halo 2 incorporates everything you could find annoying in a Halo game into one horribly unfair package. You have:
-bullet sponge enemies (Brutes and Flood)
-deadly-accurate and fast-shooting enemies that kill extremely quickly (Jackal Snipers)
-annoyingly long levels filled with both of the above (Sacred Icon and Gravemind especially)
-an overload of boss battles that go on WAY too long (Regret and Tartarus)
All of this combined creates one of the most frustratingly unfair experiences in any FPS game, possibly in all of gaming.