Git beginner: Keeping track of on-server changes

Can i, with Git, checkout the most
current repo from my live server to my
local machine?

No, unless you also add the wp-content folder under version control.

You can instead look at other tools, such as rsync.

As for the database, you could make the live server accept remote MySQL connections, so that you could work on it directly from your dev machine, although I don’t think that’s such a good idea.