There are at aleast two ways you can block other user agents and allow only a few. This can be done by editing the .htaccess file in the root directory of your WordPress website.
Option 1
Using mod_rewrite, add the code below at the top of your .htaccess file.
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} !(Google|Bing)
RewriteRule ^(sitemap_index\.xml)$ - [F,L]
</IfModule>
Option 2
This option uses the mod_setenvif module with SetEnvIfNoCase block incase mod_rewrite isn’t available.
Add the code below at the top of your .htaccess file.
<IfModule mod_setenvif.c>
SetEnvIfNoCase User-Agent .*google.* allowed_user_agents
SetEnvIfNoCase User-Agent .*bing.* allowed_user_agents
Order Allow,Deny
<FilesMatch "^(sitemap_index\.xml)$">
Deny from all
Allow from env=allowed_user_agents
</FilesMatch>
</IfModule>
You could also use robots.txt file, but I haven’t tested this one yet.