Hundreds gathered for the opening of Swindon's 32nd Festival of Literature, despite a damp start.