In
Settings > Discussion
Uncheck An administrator must always approve the comment
Check Comment author must have a previously approved comment
This way comments from people (identified by combination of name, email and site) who have previously approved comments will not require moderation. Rest of comments will.