Originally a small villa for a local textile maker, Hayfield was massively extended by the second owner to create a very pleasing house which bore many hallmarks of the Picturesque architectural style also seen at houses such as Deepdene in Surrey.
The house was built by James Hartley, local worsted cloth spinner, who sold the house to his business partner, John Cousen Horsfall, in the 1870s. Horsfall took the house and created a tumbling series of extensions in the Italianate-style around the pyramid-roofed tower in the centre. The picturesque bay windows, arcades and windows created a more organic image than the strict Georgian or Palladian houses. It has been suggested that John Kirk of Huddersfield may have been the architect.
Sir John Donald, Bt., lived in Hayfield in 1922. Following World War II it was abandoned and left vacant with local people venturing into it to remove souvenirs and items for use as garden ornaments.