How do I display certain products via their category on a section of a page using PHP?

you can still use a shortcode inside of php if you aren’t customizing the output. see https://docs.woocommerce.com/document/woocommerce-shortcodes/#scenario-5-specific-categories and https://developer.wordpress.org/reference/functions/do_shortcode/

<?php
echo do_shortcode('[products category="new-arrivals"]');
?> 

Alternatively, If you need to customize the output of the products then just use wc_get_products to get a list of products and iterate through it. See https://github.com/woocommerce/woocommerce/wiki/wc_get_products-and-WC_Product_Query

<?php
$args = array(
    'category' => array( 'new-arrivals' ),
);
$products = wc_get_products( $args );
foreach ($products as $product) {
    echo $product->get_title() . ' ' . $product->get_price();
}
?>