The Divine Word Missionaries, having opened a minor seminary in Girard, PA in 1912, were eager to expand their presence on the East Coast. They strongly considered Boston and the surrounding towns due to the city’s reputation for its strong Catholic faith. The bishop was already familiar with the SVDs from their work establishing a Black seminary in the south. When Father Peter Janser met with him in 1922 to request permission to open a minor seminary, the bishop not only approved, but suggested Miramar, his summer residence in Duxbury, MA as the site. After approval from Rome, the property was purchased by the Society in June of that year. 

Thus, Miramar made room in October 1922 for the first academic class of high school boys. The barn was converted into a classroom, gym and wash-up area. The pioneers included Divine Word Revs. Anthony Hullin as rector, Joseph Murphy and William Ross as academic staff, and Divine Word brothers Patrick Dunnigan and Bernard Husmann in charge of kitchen, maintenance and vehicles. The seminary was dedicated to St. Francis Xavier, but most of the SVDs called it Miramar. 

Beginning in 1922 the barn that was part of the original farm served as a dormitory and multi-purpose school building. Between 1931 and 1934, due in large part to the vision and determination of Rev. Hugo Aubry, SVD, a three-story school building was erected between the faculty house and the old barn. Financial difficulties plagued the community of Miramar at this time as the Great Depression was in full swing. They were having difficulty making their loan payments to the bishop for the property as well as financing this construction project.  

The problem of finding sufficient finances to support the continuation and expansion of the new foundation was not the only, or even the main, problem facing the young community. The interconnected challenges of recruiting sufficient students and the number of those who simply did not continue with their studies after leaving Miramar. Staff shortages were also a recurring issue. 

But the completion of the new school building in 1933–1934 led to an increasing number of students. In 1935, there were 100 students at Miramar, the high-water mark at the time. The first Miramar Summer Camp was held in 1940 and served as a home away from home for literally hundreds of boys each summer until it closed in 1968. 

The residence at 184 Beacon Street in Boston, purchased from Emerson College on December 31, 1953, was part of the Divine Word apostolate to create and sustain mission awareness among Catholics along the Eastern Seaboard. The house on Beacon Street was sold in 2006 and a condominium at 9 Olive Street in Brighton, MA was purchased. Revs. Donald Skerry SVD and Kenneth Feehan SVD continued their apostolate of creating mission awareness and collecting funds until 2010. 

St. Francis Xavier Seminary closed its doors as a high school in 1957 and reopened as a two-year junior college for those aspiring to the Divine Word missionary priesthood. It served this role for seven years until Divine Word College in Epworth, IA became the site of the junior college in 1964. 

Between 1965 and 1982, the main buildings were leased to various educational institutions and service groups. The Miramar community continued to do parish ministry in the area while seeking a new focus for the future. Once the leases were up on the buildings, they were sold and developed into condominiums in 1983. The remaining community members moved into the retreat house.

While the educational aspect of Miramar had concluded, work continued with the retreat apostolate. The retreat house was renovated after the sale of the seminary buildings and was dedicated in 1986 as “Miramar Retreat House.” Their work with the retreat apostolate continues to this day.