How do I make reusable content blocks for header and footer when using WordPress headless with another front-end?

ACF has the ability to create a ‘Theme Options’ page which you can then use to conveniently centralise options for things like headers and footer etc..

ACF Options Page information

As Jacob Peattie rightly points out, how you integrate this into Gatsby is outside of the scope of WPSE and may require custom development. However, providing you can inject custom code into these areas, via a function, shortcode or custom widget etc., using an ACF Options Page to control that content works nicely and is worth exploring.