There is a scene near the end of Animal Crackers (my favorite movie!) where the boys sing "my old Kentucky home", and I think you can hear Harpo singing a muffled harmony along with Chico on the last "so far away", while Groucho and Zeppo are holding another note. He is standing behind the others and even looks like he is trying to sing without moving his lips!