How can I add a field to make link nofollow to the WordPress Link Inspector Window?

You could add the advlink plugin in to tinyMCE. I don’t think I can attach the code, so to do it you’ll need to download a copy of tinyMCE:

http://tinymce.moxiecode.com/download/download.php

Then copy over the advlink directory (from the plugins folder) to your WordPress plugins folder, and open up the link.htm file.

In there edit the 4 script tags at the top from:

    <script type="text/javascript" src="https://wordpress.stackexchange.com/questions/7336/tiny_mce_popup.js"></script>
<script type="text/javascript" src="../../utils/mctabs.js"></script>
<script type="text/javascript" src="../../utils/form_utils.js"></script>
<script type="text/javascript" src="../../utils/validate.js"></script>

to:

    <script type="text/javascript" src="https://wordpress.stackexchange.com/questions/7336/wp-includes/js/tinymce/tiny_mce_popup.js"></script>
<script type="text/javascript" src="../../../wp-includes/js/tinymce/utils/mctabs.js"></script>
<script type="text/javascript" src="../../../wp-includes/js/tinymce/utils/form_utils.js"></script>
<script type="text/javascript" src="../../../wp-includes/js/tinymce/utils/validate.js"></script>

Lastly add this filter function to your theme’s functions.php file:

function tiny_mce_advlink($plugins) {
   $newPlugins=array('advlink' => WP_PLUGIN_URL.'/advlink/editor_plugin.js' );
   return $plugins+$newPlugins;
}
add_filter('mce_external_plugins', 'tiny_mce_advlink');

Then you should have a more advanced dialog when you click on a link. In the Advanced tab you’ll notice a drop down that says ‘Relationship page to target’ and you can select the ‘No Follow’ option.

You can of course use this tinyMCE plugin as the basis to write your own if you wanted.