In the early 19th century the painter's grandfather Giuseppe Fagioni, an Italian sailor from the vicinity of Genoa, suffered a shipwreck near Cavtat. While his boat was being repaired, he fell in love with the young Ana Kličan, got married and decided to settle permanently in Cavtat. They had four sons. One of them, Agostino, a sailor and merchant, married Mariette Perić from Cavtat and they had four children: sons Jozo and Vlaho and daughters Ana and Gjorgja.
Jozo was a merchant, like his father, the daughters got married and established their own families in Cavtat and on the island of Korčula, while Vlaho became a painter. In 1877 Vlaho decided to change the family name Fagioni into the Croatian translation Bukovac (word faggio in Italian means beech). Vlaho used the surname Bukovac until the end of life.
In 1892 Bukovac married the young Jelica Pitarević who came from a wealthy Dubrovnik family . They had four children: a son Ago and daughters Marija, Jelica and Ivanka. Jelica and Ivanka from their young days lived and painted in their father's studio and in 1921 they began studying in their father's class as some of the first female students at the Prague Academy of Fine Arts.
Jelica Bukovac Radosavljević (1897 - 1967) and Ivanka Bukovac Javorsky (1899 - 1978) were also painters, daughter Marija Bukovac Houdek (1895 - 1975) was a pianist, and son Ago Bukovac (1893 - 1961) was a diplomat.
Museum was established in painter's birthplace in Cavtat thanks to the special devotion of Vlaho's successors.