What am i doing wrong here?

Many, many things, not all performance related. Lets begin with the critical parts, then conclude on your performance issues and what can be done to mitigate and help

AJAX API

Firstly, you’re not using the AJAX API, and reinventing the wheel. It’s quite simple:

JS:

$.post( do_mail_sending.ajaxurl, {
    action: 'do_mail_sending',
    post_id: post_id

}, function(response) {
    alert( response );
});

PHP:

wp_localize_script( 'do_mail_sending', 'do_mail_sending', array( 'ajaxurl' => admin_url( 'admin-ajax.php' ) ) );
add_action( 'wp_ajax_do_mail_sending',  'do_mail_sending' );
add_action( 'wp_ajax_nopriv_do_mail_sending',  'do_mail_sending' ); // and for logged out users
function do_mail_sending() {
    $post_id = $_POST['post_id'];
    die( 'response data' );
}

Closing PHP Tag and Trailing Space

PHP can be used as a scripting language, but you’ve veered into app development, so I’d recommend not having a closing PHP tag at the end of the document. Your code snippet above does this and it has a new line break space afterwards, which can cause issues.

Security

It would appear that although you’re running a prepare statement, not once are you checking if the user is actually allowed to do this.

E.g. I can flood all your subscribers, and bring down your server with the following javascript snippet:

while ( true ) {
    jQuery.ajax({
        async: false,
        type: 'get',
        url: nieuwsbrief.sendmail_url,
        data: 'postid=' + post_id,
            success: function(data) {
                return false;
            }
    });
}

You don’t check that:

• the user is logged in
• the user has the necessary roles and capabilities
• that the request came from your site
• that the request came from the page with the email send form
• make use nonces
• check that the post itself exists

Performance

Your performance woes are a result of what you’re doing. Namely:

• An expensive SQL query including multiple joins
• Sending an email

Both are inherently expensive, and the last one you’re doing a lot of. Emails themselves are costly to send server-wise, as it has to contact the SMTP server, and negotiate, and any remote requests are going to take time.

There is little you can do to avoid this expense, as the core parts that make this up are inherently expensive.

But that’s okay. You aren’t the first person to have to do something expensive, so your goal should be mitigation and redesign. You cant reduce the cost of the process, but you can modify the process so that it appears fast for the UI, and still gets done. Banks solved this in the early days by doing their hard number crunching overnight while the branch was closed in batch runs.

Here are some ways of mitigating it:

• Split the process up into multiple steps, instead of 1 AJAX request, do several, 1 to count how many emails need to be sent, then several afterwards, using a paging style technique to send them in batches of 10 or 20 depending on your servers response times.
• Dont send emails at all, instead calculate what work needs to be done, then save it in the database as a task that needs doing. Then have a cron job run at a regular interval that sends the emails in the background.
• Add notes to a queue of emails to be sent, and send them one by one, removing the item from the queue. Your newsletter may not be sent all at once, but it will be sent, and it won’t bring your server down while it happens. It also enables you to create a status page showing the mailing progress in realtime.

Data Note

That you need an SQL statement like that makes me wonder if your categories table and query cant be replaced with a custom taxonomy and a call to WP_Query. This way you could take advantage of the built in object and query caching