Hey, based on your requirements I’d suggest you consider just using Custom Post Types with custom fields and defining your own layout. That way you won’t be painted in to the limitations of a plugin that doesn’t meet you needs exactly, and it sounds like Custom Post Types will fit your requirement pretty much spot on. Here’s another answer I previously wrote that give that discusses Custom Post Types.
In addition I’ll point you to a 2nd answer I wrote where I described How to Recreate CrunchBase in WordPress using Custom Post Types. After all, there is really very little technical implementation difference between creating a directory of companies and a directory of people, right?