I've never done this on a Microsoft keyboard, but on most keyboards I have used, you can carefully prise up a key and it will come off and you can clean underneath it. Then it will click back into place. remember there are 2 shift keys on most keyboards. You may find bits of dried food, baby drool etc under there.
I know this is being wise after the event, but letting babies play with keyboards (or computers in general IMHO) is a bad idea, not least because if you can get the keys off, so might the baby if one was a bit loose, and then it could choke either on the key or the plastic parts underneath. Also nice keyboards are not cheap. If you don't want to come over as mean to the baby, pitch the choking idea to the baby's mother. Most moms don't fancy the idea of a baby turning blue and a scary trip to the emergency room and will see the sense of choosing more appropriate playthings for the little one.