Kyu Shibarikyu Tei-en has an origin in the Rakuju-en garden, attached to a detached residence in Shibakanasugi. This garden was granted by the shogun Tokugawa Ietsuna to Okubo Tadatomo, Roju or senior councilor and later appointed as a lord of Odawara domain in Sagami Province. Based on the passages in Rakuju-en ki (Records on Rakuju-en Garden) handed down by the Okubo family, we can assume that the gardening works were completed by 1686.

The garden is a so-called stroll style garden which has a large pond with a largely folded shoreline; in the pond, three small islands, called respectively Oshima, Nakanoshima and Ukishima, are arranged. Further, long banks are built on the east and the west of Nakanoshima. Sand bars are provided on the north and the west of the shore of the pond, while stones are arranged so as to represent dry waterfall on the south side. An artificial mountain in stones on Nakanoshima, called Horaijima, is allegedly a structure from the time of an original gardening. The area for the garden was expanded in 1696, by landfilling and receiving a grant of a plot 27 m long from east to west and 226 m long from north to south, in an area close to the sea on east side. The Rakuju-en garden was transferred in 1818 to Hotta Clan, the lords of Sakura domain, who then used the garden as a lord's own residence in Edo. Thereafter, the garden was owned by Shimizu Clan, one of Gosankyo, three branches of the line of Tokugawa Yoshimune, and later by Tokugawa Clan of Kii Province who used the place as their residence in Shiba. Finally the garden was transferred to the Ministry of the Imperial Household in 1875 and renamed Shibarikyu in the following year.