How can I query all available public post types and taxonomies?

There are several ways to answer that question, depending on what it is you are attempting to accomplish. To generalize, you need to create two queries: one to get all public posts and one to get all public taxonomies and terms. Getting both in one array is not practical as term objects and post objects have different formats – however you can construct your own array from the two that are produced.

WordPress queries can be constructed in a number of different ways using WP_Query parameters. Alternately, various other functions perform pre-built queries for you.

In short, you are probably after something similar to (untested – may need some work):

$public_taxonomies = get_taxonomies( Array( 'public' => true ) );
$terms = get_terms( $public_taxonomies ); // Get ALL terms from all public taxonomies

$public_post_types = get_post_types( Array( 'public' => true ) );
$posts = get_posts( Array(
  'posts_per_page' => -1                  // Get ALL posts...
  'post_type'      => $public_post_types, // ...of the public post types...
  'post_status'    => 'publish'           // ...that are published.
) );

Please note that this has some pretty hefty performance implications, particularly on large sites… Returning virtually all the content on a site in one go is no small task. Better to break it up into several queries with pagination parameters, or caching just the important bits from the resulting arrays using the Transients API, if at all possible.