Externals, definitely. Set up your own SVN repo, then make WP as an external in the repo. I put it into a /wp directory. You can access the WP SVN as an external here: http://core.svn.wordpress.org/
I tend to use trunk. You can use a branch, if you really, really want. Branches get auto updated with minor releases (/branches/3.0 held 3.0, then 3.0.1, 3.0.2, etc), new branches get made for major releases.
Alongside it, I make a /name-content directory, where I’ll put my custom themes and such. “name” is whatever you want it to be. Then I can use a custom /wp-config.php to change the location of the wp-content directory into my custom one, like so:
define( 'WP_CONTENT_DIR', $_SERVER['DOCUMENT_ROOT'] . '/name-content' );
define( 'WP_CONTENT_URL', 'http://example.com/name-content');
For the themes, they go directly into the repo (since they’re probably custom to the site) under /name-content/themes.
Any plugins I use I make a /name-content/plugins directory for, then set them to be externals from the http://plugins.svn.wordpress.org/pluginname/trunk repo. If you don’t like running trunk (dangerous and that), then you can use the /tags/1.1 or whatever for them instead.
With a little bit of custom .htaccess logic (with the .htaccess file in the root of the repo) to make /wp the “root” of my domain, then the whole thing is setup. Example .htaccess file to move the root:
Options -Indexes
RewriteEngine on
RewriteCond %{HTTP_HOST} ^(www.)?example.com$
RewriteCond %{REQUEST_URI} !^/wp/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ /wp/$1
RewriteCond %{HTTP_HOST} ^(www.)?example.com$
RewriteRule ^(/)?$ wp/index.php [L]
This allows WordPress in the /wp directory to be at the root of your site, with it pointing to the name-content directory for the content aspects. The .htaccess won’t redirect those requests (since the files exist) and thus everything works perfectly.
Advantage: When you want to update, a simple svn up updates the whole site, after you fiddle with the externals and such to point to the newer versions (if you’re too chicken to run on trunk).