Given the housing market, this might be all one could afford after paying student loan debt and high rent. (Photo: Shutterstock)

The high cost of living is getting in the way of 45 percent of millennial homebuyers achieving their goal. So they're resorting to all sorts of ways to raise cash—including raiding their retirement accounts.

A new Bankrate survey finds that to scrape together the money for a down payment, millennials are moving in with relatives, selling off their stuff and even taking money out of their retirement accounts. Even with those efforts, it still takes them an average of 3 years to stash enough money to cover the down payment—while it takes GenXers 2 years and 9 months to cobble the payment together, and 2 years and 6 months for boomers to have managed it.

Oddly, many don't even know what the required down payment is, the survey finds, but they have a lot of company: 51 percent of Americans say they don't know how much they need as a down payment in the current market to buy a home.