The short answer is that our brains are programmed to see the world in three
dimensions instead of two
. There are more details at Discover magazine, as well as a gallery of colorful optical illusions. For example, these two Rubik's cubes do NOT have the same colors. The "blue" squares in the left picture and the "yellow" squares in the right picture are gray.
I took samples from each and put them on a white field to make sure. Link