Is there any good way to use SVN for uploads directory?

You could hook into the action add_attachment and perform an SVN commit.

add_action( 'add_attachment', 'svn_commit_attachments' );

function svn_commit_attachments( $attachment_ID )
{
    // collect data about the attachment, for example its path, the user and so on.
     shell_exec ( '$ svn commit -m "$user added file $filename."' )
}