your problem is not with the admin pages themself, they are simply not big enough to be hugely impacted by slow connection, but with JS and CSS files. You can set an expiry date for them which will signal to the browser it may cache them until that time.
To cache them for a week in your .htaccess file add (idea taken from askapache)
# 1 WEEK
<FilesMatch "\.(jpg|jpeg|png|gif|swf|js|css)$">
Header set Cache-Control "max-age=604800, public"
</FilesMatch>
The max age parameter specifies the time in seconds after which the url have to be fetched again.
If this is not good enough, maybe because your browser cache is full with other files which you access more frequently, then you should install a caching proxy on your PC, configure it to always cache JS and CSS from your site, and configure the browser to use the proxy.