WordPress uses PHPMailer class and has an action hook that you can use to configure it. For example:
add_action( 'phpmailer_init', 'cyb_smtp_config' );
function cyb_smtp_config( $phpmailer ) {
$phpmailer->IsSMTP();
$phpmailer->Host="smtp.mailer.com";
$phpmailer->Port = 25;
$phpmailer->Username="username";
$phpmailer->Password = 'password';
}
Be sure that you need a custom SMTP connection instead of configuring the SMTP service in your hosting. If you are not sure, contact with your hosting provider.