Is it possible to get all term items from a custom taxonomy regardless of post attachment status?

If I understand right, you need all attachments (attached or not) with specified term?

$args = array(
    'post_type'     => 'attachment',
    'my_taxonomy' => 'my_taxonomy_term'
);

$my_query = new WP_Query( $args );

if( $my_query->have_posts() ):
    while ($my_query->have_posts()) : $my_query->the_post();

        // your stuff here

    endwhile;
}
// Don't forget to reset
wp_reset_postdata();

WP_Query Taxonomy Parameters.