The museum was founded in 1971 by the Pacificulture Foundation, which purchased “The Grace Nicholson Treasure House” from the City of Pasadena. Grace Nicholson donated the structure to the city for art and cultural purposes in 1943 and was a dealer in Native American and, later, Asian art and antiques. It houses some 13,000 rare and representative examples of art from throughout Asia and the Pacific Islands. In 2013, the museum became part of the University of Southern California. The building was temporarily closed beginning June 27, 2016 until December 2017 for a seismic retrofit and renovation. It reopened as of December 8, 2017. The building, which is listed in the National Register of Historic Places, was built in 1926 and designed by the architectural firm of Marston, Van Pelt & Maybury.

In 2022, the University of Southern California established USC Museums, comprising the USC Fisher Museum of Art (est. 1939) and the USC Pacific Asia Museum (est. 1971 and acquired by USC in 2013), and the University Collections Management Compliance Program (est. 2022). 

Under this new structure, each museum maintains its distinct identity, individual collections, site-specific staff, and mission. Together, USC Museums envisions becoming USC and South Los Angeles’s most innovative arts laboratory and a model for experiential learning and co-creation in the museum field. By positioning ourselves as “co-creators,” we aim to unite the resources of our museums, campus, and communities, placing art at the core of our collective efforts to address shared challenges. In democratizing our practices, opening our processes, and making collaboration central to everything we do, we can offer our audiences the access, connection, and representation they desire.