Garoppolo spent the first four seasons of his career in New England