Cant get paginations on single.php to work with my custom post types

You will need to add %link to the first parameter ($format) of next_post_link and previous_post_link

This will generate the html links to the respective posts

EDIT 1

Just on your statement about custom queries, did you had a look at how pre_get_posts can be used to alter the main query as needed. It is never a good idea replacing the main query with a custom query.

Here is an example on adding your custom post type to your homepage

function include_post_type( $query ) {
    if ( $query->is_home() && $query->is_main_query() ) {
        $query->set( 'post_type', array('post', 'my_post_type') );
    }
}
add_action( 'pre_get_posts', 'include_post_type' );

EDIT 2

Just on your comment. The posts are displayed according to post date, either ASC or DESC depending on your blog settings.

You can set both links to only page to the next/previous post within the same term. Have a look at the third parameter ($in_same_term) and fifth parameter ($taxonomy)

By default $in_same_term is set to false and $taxonomy is set to category. You can set this accordingly.

Example:

next_post_link( '%link', 'Next post in types', TRUE, ' ', 'types' ); 

This will page to the next post within the same term that this particular post belongs to within the types taxonomy