The oldest part of the house, two rooms up and two rooms down, was built for Reverend Hale in 1694. In 1745, Colonel Robert Hale, Jr. (John Hale’s grandson), added the front section of the house with the gambrel roof. He excavated a cellar (where evidence of a buttery has been found), added the present staircases, and built an addition of two rooms down and two up, along with the gambrel roof that allowed rooms on the third floor. The property at that time included about 100 acres and extended to the ocean.

In 1881, Robert Hale Bancroft of Boston, who had inherited Hale Farm from his mother, converted the house into a summer estate for his family. He added a wing for a new kitchen and laundry in 1881, and servant quarters in 1898. Bancroft died in 1918. In 1937, his daughters sold the property to the Beverly Historical Society.