get_page_children() return only titles

You can just return the IDs, then get each title by the respective ID. And to restrict the query to four posts only, you can use posts_per_page.
I modified and optimized your above code a bit, so here it is:

<h3>Recent Projects</h3>
<ul class="footer-list"><?php
    $projects_page = get_page_by_title('Projects');

    $args = array(
        'post_type' => 'page',
        'post_parent' => $projects_page->ID,
        'ignore_sticky_posts' => true,
        'posts_per_page' => 4,
        'fields' => 'ids',
    );
    $projects = new WP_Query($args);

    foreach ($projects as $project) {
        ?><li>
            <a href="https://wordpress.stackexchange.com/questions/97799/<?php echo get_permalink($project); ?>"><?php echo get_the_title($project); ?></a>
        </li><?php
    }
?></ul>