I live in a 102-year-old house, on a small acreage, which helps explain why I had a problem and needed to know how to get rid of mice in the walls. (It sounded like they were having a party in there!)
The young lady that works where I shop recommended peppermint oil for mice. She'd heard from her grandma that all you need are peppermint-soaked cotton balls scattered about, and mice will split!
So I placed small, peppermint-soaked cotton balls all around the kitchen. I unscrewed two wall plugs and carefully slipped some in behind the sockets and, like magic, no more sounds of mice. Twenty-four hours later, no sign of the mice at all.