The name of the settlement was changed from Sveta Trojica v Slovenskih Goricah (literally, 'Holy Trinity in the Slovene Hills') to Gradišče (literally, 'fortified hillfort') in 1952. The name was changed on the basis of the 1948 Law on Names of Settlements and Designations of Squares, Streets, and Buildings as part of efforts by Slovenia's postwar communist government to remove religious elements from toponyms. The name was then changed from Gradišče to Gradišče v Slovenskih Goricah (Gradišče v Slovenskih goricah) in 1953. The name Sveta Trojica v Slovenskih Goricah was restored in 1992.
The local landmark is the parish church in the settlement from which the village gets its name. It is dedicated to the Holy Trinity (Slovene: sveta Trojica) and belongs to the Roman Catholic Archdiocese of Maribor. It is a large Baroque church with three belfries that also appear in the municipality's coat of arms. It was built between 1636 and 1643 and expanded between 1735 and 1740. There is also a Franciscan monastery in the village.