BOSTON -- While the rest of us filled out tax forms and waited in post office lines to pay our share to the state, a couple who lives in a $800,000 North Andover home continued their annual tradition -- not paying.

Carl Vissering and his wife are somewhat of a mystery to their neighbors who say they rarely see them and have no idea what they do for a living.

Video: Massachusetts' Biggest Tax Delinquents

But the Visserings are well-known to the Massachusetts Department of Revenue. Carl Vissering has spent years in and out of the top spot of the DOR's list of worst tax delinquents.

"It's, I think, over $4.2 million," said DOR Commissioner Navjeet Bal.

That's what Vissering alone owes. The top 10 tax delinquents collectively owe the state almost $20 million, enough money to hire more than 400 teachers at the state's average salary.

"It should offend people," said Bal. "I think it's a question of fundamental fairness."

Team 5 Investigates' Sean Kelly asked the state why these tax deadbeats aren't in jail.

"Our goal as a Department of Revenue is to collect money so I think in some ways it's more effective for us to continue on the civil side and actually collect the money rather than putting them in jail," Bal said.

In Vissering's case, there appears to be little money to get. Team 5 Investigates made half a dozen visits to his home over the last year to find out why he hasn't paid since 1999.

"Can you tell me where you are on the $4 million you owe?" Kelly asked Vissering outside his front door.

"They processed it and I never even got a notice," Vissering said.

State officials tell Team 5 Investigates they contacted Vissering many times over the last eight years about his multimillion-dollar debt. In August, a superior court judgment ordered him to pay his taxes, penalties and interest in full.

How did Vissering get in so deep? Day trading. He told Team 5 Investigates that he took major losses in the market flipping stocks daily. But those losses did not offset his enormous capital gains taxes. He claims his problems are an accounting error yet he's shown no urgency to correct it.

"You couldn't pay it a year ago, you still can't pay it now right?" asked Kelly. "Oh yeah, no," said Vissering. He has not hired a lawyer. "I can't afford one. They confiscated all my money." Vissering said.

The state is also prepared to go after the proceeds of Vissering's home if he ever sells it. It's a house in shambles inside because of a pipe burst two years ago that flooded the entire first floor.

"Do you think you'll ever see that money or even half of it?" Kelly asked the DOR commissioner. "You can't get blood from a stone so you have to be realistic," Bal said.

Last year the commonwealth hired 87 new collectors and auditors at the Department of Revenue, and in his budget Gov. Deval Patrick is banking on that newly expanded team to generate $60 million worth of new revenue.

But Team 5 investigation found that in many cases that money will be tough to get, especially from the top 10 tax delinquents. Their tax debt dates back 30 years in one case. Several others haven't paid since the early 1990s and Massachusetts is still waiting.

"Sometimes these proceedings can drag out," Bal said. "In some cases we found that people hide their assets so it's hard for us to get at them."

State officials think they'll have an easier time going after people and companies who owe smaller amounts of back taxes. According to the DOR, Massachusetts is owed a total of $800 million in back taxes.