apply filters only to specific post listing without check the url parameters

Yes, you’re right. There is a clever way. post_row_actions filter can accept also second param $post from which you can get it’s post_type. See the code:

add_filter( 'post_row_actions', array( $this, 'remove_row_actions' ), 10, 2);
public function remove_row_actions( $action, $post ) {
    if ( 'slider' === get_post_type( $post ) ) {
        unset $action['view'];
        return $action;
    }
    return $action;
}