Little St. James was part of the Danish West Indies. In 1917, Denmark sold the United States Virgin Islands to the United States. Little St. James was included as one of the "adjacent islands" in the deal.

Little St. James is a private island. In 1997, it was owned by venture capitalist Arch Cummin and was for sale for $10.5 million. In April 1998, a company called L.S.J. LLC purchased it for $7.95 million. Documents show that Jeffrey Epstein was the sole member of L.S.J. In 2019, the island was valued at $63,874,223. Little St. James was Epstein's primary residence, and he called the island "Little St. Jeff". The main house on the island was renovated by Edward Tuttle, a designer of the Aman Resorts.

Before Epstein bought the Island, several buildings were already present, including the main house, the guest house, oval-shaped pool, and three cabanas. They were seen in the booklet promoting the island for sale and appeared in the March 1996 edition of House & Garden magazine and "The Villa Report". In 2005, Epstein hired the Virgin Islands Water and Power Authority to install a combination power and fiber optic cable between St. Thomas and Little St. James, giving the island dedicated data and electric connections, which eliminated the need for generators.


Little Saint James Island in the Virgin Islands, while owned by Jeffrey Epstein in June 2013
In 2008, Epstein's estate on Little St. James had 70 staff. According to a former staffer, Epstein insisted on discretion and confidentiality from his employees. In 2016, the Department of Planning and Natural Resources received complaints about Epstein, who had begun to clear land without a permit. In his will and testament (a 21-page pour-over will), signed two days before his death, all of Epstein's holdings were transferred into the "1953 [the year of his birth] Trust".

In March 2022, Little St. James and the neighboring Great Saint James were listed at $125 million. A lawyer for Epstein's estate said money obtained from the sale would be used to settle lawsuits. Bespoke Real Estate, the agency jointly overseeing the sale, said that further information on the listing was available only to prospective buyers.

In May 2023, billionaire Stephen Deckoff, under his firm SD Investments, announced the acquisition of Great St. James and Little St. James for $60 million.