The Moa was the only wingless bird that ever existed.
The moa were hunted to extinction by 1500 by the Maori in New Zealand. They were the only species of birds with no wings. But wait, you say, what about kiwis, emus, and ostriches? Well, these flightless birds, a group of birds called ratites, actually do have wings (some of them vestigials).
Oh, and one more thing. I mentioned New Zealand - have you ever asked yourself where is Old Zealand? New Zealand is actually named after Zeeland, a major seafaring province of the Netherlands, by Dutch navigator Abel Tasman in 1642 (yup, the island of Tasmania is named after him). Captain James Cook misspelled it New Zealand and the name stuck ever since.