WordPress Codex has quite extensive installation instructions.
I see there’s a .zip file and a .tar.gz file — which should I use?
No difference. Archive format is merely for convenience, contents are the same.
I want the site to be at http not at www., but I do want either the site to also be visible at www. or for the www. to redirect to http — where should I upload the WordPress files? Into public_html?
It depends on how your domain is set up, but usually both www and non-www version point to same thing. public_html is probably right directory.
You will only be able to pick www or non-www link when configuring WordPress. Redirecting other version will need a bit of .htaccess tweak, google it.
Do I unzip wordpress-3.2.1.zip and upload those files, or do I upload the .zip to the server?
Depends on how comfortable you are with server-side manipulation. Uploading unpacked files is easier, uploading archive and unpacking on server (using SSH or cpanel for example) is faster.
Note that there is wordpress directory in archive which you don’t need, files whould be unpacked to the root of the site.
Once the files are uploaded, is there anything else to do before it’s ready to go?
See link to Codex above, you should have database created, etc.
How do I get the username and password to login to the admin panel so I can build the site? And what url do I go to to login?
Trying to visit site with freshly uploaded WordPress files will redirect you to install.php that will guide you through initial configuration. Initial setup should have link to admin area in theme sidebar, you can also always go there by visiting /wp-admin/ URL.