Smith’s time in Washington is reportedly over