Higher mortgage rates are putting a damper on the housing market. Applications for a mortgage to purchase a home are down 21% from a year ago as fewer and fewer people can afford to buy.