Of course, some sponsors just can't stand their providers; 31 percent told Brightwork they made the shift because they were dissatisfied with their old provider.

Brightwork founder Ron Bush said the balance of those surveyed, 18 percent, cited both reasons for their decision.

Changing 401(k) plan providers is a big deal, often taking about three months, and sponsors are generally loathe to go through the process.

Speaking Tuesday at the 2014 SPARK conference here, Bush said the promise of better service was the leading reason (27 percent) cited by sponsors for making a switch.

Those sponsors told Brightwork they wanted a provider that was easier to work with, that they needed more administrative, reporting and compliance support and needed greater customization of educational materials, among other things.

Service problems included lack of responsiveness, mistakes, poor communication, unwillingness to do what the sponsors needed, mishandling of loans and withdrawals, and mishandling of distributions.

Cost was cited by just 13 percent of the respondents as their primary reason for switching. More specifically, 6 percent said they wanted to reduce the investment costs in their plans and just 1 percent wanted a provider who could allocate fees "fairly across all participants."

See also: Retirement plan costs fall once again

On average, plans stayed with their provider for eight years.

Providers hoping to hold onto sponsors might take heart in one other finding in the Brighwork survey: 66 percent said they remained with the provider after it made changes they had requested.

More specifically, 35 percent of the largest sponsors said they stuck by their provider after it offered greater choices of investment managers and funds.

Brightwork surveyed sponsors with more than $1 million in assets who switched, or seriously considered switching, providers in the past two years.

See also: The tricky business of finding a 401(k) advisor