A list of the top tax dodging hotspots includes many cities in California, as well as some on the East Coast.
Beverly Hills, Calif. tops the list, followed by Newport Beach, Calif., New Carrolton, Md., and College Park, Ga.
The National Taxpayer Advocate came up with the list by analyzing the IRS's confidential tax return data.
They determined that there are more than 350 communities in 24 states where tax cheats occur most.
One-third of them were in California, mostly centering around Los Angeles and San Francisco.
Others were in communities by Houston and Atlanta and in Maryland suburbs.
The fewest tax cheats occurred in the Midwest and Northeast.
The list of the least likely tax cheats includes the Aleutian Islands in Alaska, West Somerville, Mass., Portersville, Ind., and Mott Haven, N.Y.
All tax returns are given a score, and when that score is higher, the IRS is more likely to audit the tax payer.
Nina Olsen helped come up with the list after she was granted access to the data to learn why some taxpayers were more likely to cheat than others.
Olsen said some people are more likely to cheat on their taxes than others, like those who own construction companies or real estate rental firms.
President and CEO of the Newport Beach Chamber of Commerce, Steve Rosansky, said business owners are more likely to be targeted by the IRS because of their higher incomes.
"I imagine it's just a matter of them going where they think the money's at," Rosansky said. "I guess if I was running the IRS I'd probably do the same thing."
Only about one percent of tax returns are audited each year, and the IRS tends to chose returns that are more likely to yield additional tax money.
This year, the IRS is using a confidential computer program that determines the chances of collecting more money from an audit.
Each return is run through the computer program and produces a score called the Discriminant Inventory Function, or DIF. While a high DIF doesn't necessarily mean the person is cheating on their taxes, the IRS said it is reliable in determining who is.
"If your return is selected because of a high score under the DIF system, the potential is high that an examination of your return will result in a change to your income tax liability," an IRS publication said.
While it is not known how the system works exactly, a few veteran tax preparers believe they know ways your DIF score could increase.
"If you're reporting $8,000 of charitable contributions when you're only making $50,000, that's a red flag," Bob Meighan, vice president of TurboTax, said.
"Likewise, if you're reporting business or employee expenses that are out of the ordinary for your income range, that would attract the interest of the IRS as well."
Experts agree that the way to increase a DIF score is to have an unusually large amount of deductions for an income.