Sotheby's most profitable business isn't art auctions, and that is worrying some people

The most profitable business at auction house Sotheby's is its finance arm, and the company's temptation to grow this has one analyst worried.

Sotheby's boosted the size of its credit facility nearly three times this summer, to about $1.34 billion.

Those borrowings will enable Sotheby's to double its portfolio of loans to $1.3 billion, according to ratings agency Moody's.

In a downturn for the art market, that could mean trouble for the auction house.

"The potential $1 billion borrowing to support growth at the finance segment would materially weaken Sotheby's credit metrics and its corresponding credit profile during the next cyclical downturn, compared to the last recession - potentially putting the company's ratings under pressure in this scenario," Moody's senior vice president Margaret Taylor said in a note on Monday.

"Although the finance segment earnings generate the highest margins and are poised to continue growing rapidly, Sotheby's is sacrificing its balance sheet to support its loan portfolio."

The company has seen shares drop about 25% so far in 2015, despite it buying back hundreds of millions of dollars of its own stock. Shares of Sotheby's fell for about 2.5% in morning trading Monday.

The drop has come even after a pair of hedge fund activists have applied months' worth of pressure on the embattled New York company, ripping into lavish board lunches and forcing the March resignation of its CEO.

Moody's envisions three scenarios depending on how much of its credit facility Sotheby's is willing to use. The more it lends the higher the earnings and the greater the indebtedness.

Business Insider has reached out to Sotheby's for comment and will update this story if we hear back.