The church was built in the IX century to the desires of the Andrearda or Giulia families, but accounts also attribute the construction to other families. It is believed that this place of worship was governed by the Patriarchs of Grado who had moved to Venice in a palazzo close to Campo San Silvestro for health reasons. Here the priest Vittore could be found, son of the Doge Orso I Partecipazio, who rose to the patriarch of Grado in 884.
The church underwent various restorations, the last being during the Nineteenth century to the designs of the architects Santi and Meduna. During one of these redevelopments many precious masterpieces were sold, except the Baptism of Christ by Jacopo Tintoretto.
The church was home to the Guild of Gondoliers who gathered here to discuss the activities of the profession.
The famous painter Giorgio Barbarelli, known as Giorgione, lived in a house located in the Campo of San Silvestro while, on the 16th of April 1703 in the the parish of San Silvestro, the poet Luisa Bergalli was born, wife of Count Gasparo Gozzi.