Just after the arrival of the First Fleet in 1788, a fleet of French ships captained by Jean-Francois Lapérouse also arrived and anchored at what is now known as Frenchmans Bay. After making landfall, they set up a stockade and planted European vegetable seeds in a plot which became known as 'the Frenchman's Gardens'. The French fleet, however, departed after just six weeks. An 1824 description noted the garden was still visible: 'An enclosure in which Lapérouse had vegetables sown is still there, it has kept the name of the French garden. It is surrounded by a hedge but the inside is almost uncultivated; some vegetables saved by the detachment perished because of lack of water'.
The location of the Frenchman's Gardens is uncertain but was described in 1824 as 'located at 300 paces' distance' from the Macquarie watchtower.