Real estate prices in Hamptons soar 40%

Hamptons' sand is turning into gold.

The average sale price for real estate in the Hamptons soared by 40 percent, to $1.7 million, in the first quarter, one of the biggest year-to-year increases in recent history.

Fueled by a continued surge of wealthy vacation-home buyers paying all cash, the number of deals in the first quarter jumped by 52 percent and the median price rose by 19 percent to $880,000, according to a report from Douglas Elliman and Miller Samuel.

The Hamptons has always been a high-priced sandbox for the rich, of course. But the new numbers suggest that even in the high-end home market, the eastern Long Island is defying economic gravity.

Read More Connecticut estate sells for a record $120 million

Among the mansions that sold in the first quarter was a six-bedroom stone and stucco waterfront manor home in North Haven for $31.5 million. The average sale price in North Haven—known for its outsized homes—hit $17.4 million in the first quarter.

A boxy, modern home in Sagaponack, which is not on the water, but has 2.4 acres, sold for $20 million.

"The whole second-home market is very strong right now, and especially the Hamptons," said Dottie Herman, CEO of Douglas Elliman.

Granted the year-to-year numbers were exaggerated somewhat by an artificially low first quarter of 2013. That quarter saw a steep drop in sales and prices after the rush of buying at the end of 2012 as homeowners sought to sell before the increase in capital gains tax rates.

And the $1.7 million average price is still below the $2.1 million peak in the fourth quarter of 2012.

Yet the Hamptons boom stands in stark contrast to the rest of the American housing market, which seems to be slipping into a spring slump as homes become less affordable to everyday buyers. Like many wealthy, second-home markets, the Hamptons is on a tear, as the growing population of millionaires and billionaires turns to real estate as an alternative investment

Read More Higher prices 'choke' housing

The continued strength in the Hamptons is also surprising given the volatile stock market in the first quarter. Traditionally, the Hamptons market has been tightly correlated to stocks. But while stocks were basically flat in the first quarter, the bull market for sales and prices in the Hamptons took off.

Jonathan Miller of Miller Samuel said that while Wall Street bankers are still important to the Hamptons market, today's buyers are coming from other areas of finance—private equity and hedge-funds—as well as from other industries and countries.

"The Hamptons has diversified," Miller said.

The growth in the first quarter was driven mainly by the very high end—homes priced at $5 million or more. The average sales prices for the so-called luxury market in the Hamptons—the top 10 percent of sales—soared 56 percent to $7.4 million. The median price jumped 53 percent to $5.6 million.

Read More Is Miami the next Monaco?

Miller said that homes sold for $5 million or more accounted for 7 percent of the market in the first quarter, up three-fold from a year ago.

"The market was more skewed toward the high end," Miller said.

—By CNBC's Robert Frank