Northern missionaries and abolitionists Laura Towne and Ellen Murray founded Penn School in 1862 for the education of newly freed Sea Island slaves. It was one of the first schools built in the America for former slaves. The Penn School, established as part of the "Port Royal Experiment", was a model school in the South. In 1901, it became known as the Penn Normal Agricultural and Industrial School after adopting the Hampton Institute and Tuskegee industrial arts curriculum. The school operated through the end of World War II when education was then provided by the Beaufort County School system.

The school preserved much of the Island's African-American history and culture through collections, historical documentation, oral histories, musical recordings and handicrafts (especially sweetgrass baskets). The facility also served as a meeting place and retreat for Dr. Martin Luther King, Jr. and the Southern Christian Leadership Conference during the 1960's. The Penn Center, Inc. owns one of the premier scholarly collections of African American history from post-Civil War to today that is reposited at the University of North Carolina-Chapel Hill.

The Penn Center is open for tours, educational enrichment, conventions and group enrichment year-round and welcomes over 40,000 visitors annually. The institution will commemorate its 150th Anniversary with a multi-year calendar of programs and exhibits beginning in April 2012 to November 2014. A long-standing celebration on the site is the Annual Heritage Days Celebration held the second weekend of November, featuring Gullah performers, artists, crafters, and food.

Tour groups are always welcome to come to the Penn Center, the Lowcountry's most historic African American site for museum exhibits, cultural performances and workshops, historical presentations, educational programs for children, walking and island tours, outdoor recreational activities, environmental studies and more in a rustic retreat setting.