Remove plugin function and add it back in via functions.php

You can use has_term to check if a post has a ‘category’ then you can return your function early if it does not or ideally you would only add this function to posts of your chosen ‘category’ however you have not shown code where you add this action so I won’t comment on that.

Try something like:

public function display_button_on_single() {
    global $wp_query;

    if ( ! has_term('category_3', 'taxonomy_name')) {
        return;
    }

    $post_ids = array();
    // the rest of your function...

https://developer.wordpress.org/reference/functions/has_term/