Automatically delete post in a single category when a new one is created

Why not. The API of WordPress has the function wp_delete_post to delete a post. If the trash is active, on default true, then this function moves the post to the trash.
You need to add a logic after publishing a new post to delete the old one. To firte your function you should use the hook publish_post.

So a simple starter see below, need the logic to get the ID of the old post.

add_action( 'publish_post', 'post349197', 10, 2 );
function post349197( $ID, $post ) {

    // &oldID = Need logic to get the last post.
    wp_delete_post($oldId);
}