admin-post.php is perfect for that.

You have to do three things:

1. Set action for your form to admin-post.php

<form method="post" action="<?php echo admin_url('admin-post.php'); ?>">

2. Add hidden input with name=action

<input type="hidden" name="action" value="my_plugin_action" />

3. Register your callbacks for admin_post_{$action} and admin_post_nopriv_{$action}:

add_action( 'admin_post_my_plugin_action', 'my_form_processor' );
add_action( 'admin_post_nopriv_my_plugin_action', 'my_form_processor' );
function my_form_processor() {
    // your code that will process form data
}