I figured out the answer myself and I’ll post it here in case anyone else has this problem. Basically in WP 3.9.1, wp_insert_post() checks the current logged in user for account capabilities (at least this is what I assume). Since I was connecting over cURL, there is no logged in user so the capability checks fail. All I had to do was send the cookie WP uses to remember logged in users:
$curl = curl_init( 'http://www.domain.com' );
$login_cookie="wordpress_logged_in_[hash]=xxx"; // This cookie can be found once a user is logged in.
curl_setopt( $curl, CURLOPT_COOKIE, $login_cookie );
So I simply set up a new admin account for my external application, logged it as that account (using /wp-admin), copied the cookie that WP made for me, and added it to my application.