Original Answer
Add the code below to your current theme’s functions.php:
function wpse_add_nb_mime_type( $allowed_mimes ) {
$allowed_mimes['nb'] = 'application/mathematica';
return $allowed_mimes;
}
add_filter( 'mime_types', 'wpse_add_nb_mime_type', 10, 1 );
Remove define, mentioned in your question, from wp-config.php. Now you should be able to upload .nb files.
Plugin alternative: to make the code above, immune to theme changes, it’s better to put it into a plugin. Create new file wpse-addnbmimetype.php, with this content:
<?php
/*
Plugin Name: WPSE Add NB Mime Type
Description: Adds 'nb' => 'application/mathematica' to allowable mime types
*/
function wpse_add_nb_mime_type( $allowed_mimes ) {
$allowed_mimes['nb'] = 'application/mathematica';
return $allowed_mimes;
}
function wpse_prepare_add_nb_mime_type() {
// this filter can be added as early as in init action
add_filter( 'mime_types', 'wpse_add_nb_mime_type', 10, 1 );
}
add_action( 'init', 'wpse_prepare_add_nb_mime_type' );
Put this file into ‘/wp-content/mu-plugins/’ folder. Now you can switch themes, without losing ability to upload .nb files.
Analysis
Read comments to @majick’s answer, first. After all corrections, his filter will look like this:
add_filter('upload_mimes', 'modify_upload_mimes', 10, 2);
function modify_upload_mimes($t, $user) {
if ($user == null) {$user = wp_get_current_user();}
if (in_array('administrator', $user->roles)) {
$t['nb'] = 'application/mathematica';
}
return $t;
}
Does it work? Yes, but it does restrict uploads of files with .nb extensions to ‘administrator’ role only. Authors and editors will not be able to upload such files. Let’s change it, by replacing line:
if (in_array('administrator', $user->roles)) {
with:
if ( $user->has_cap( 'upload_files' ) ) {
Now, authors, editors, and administrators can upload .nb files.
Checks against user’s role should be discouraged, and should be made against user’s capabilities, instead. If one day, we decide to add ‘upload_files’ capability to contributor’s role ( all contributors ), or to the individual user, our filter will work, without changes.
Why do we have two hooks, ‘mime_types’ and ‘upload_mimes’, to choose from? It is recommended, to use ‘mime_types’ for adding mime types, and ‘upload_mimes’ for removing ( to unset ) them.
There are two functions in ‘/wp-includes/functions.php’: first –
get_allowed_mime_types( $user = null ), introducing ‘upload_mimes’ filter hook, which was intended for arbitrary removal of ‘swf’, ‘exe’, and conditional removal of ‘htm|html’, mime types, and second – wp_get_mime_types(), introducing ‘mime_types’ filter hook, intended for adding mime types.
Conclusion
Technically, we could use either hook for our solution, but according to recommendations, ‘mime_types’ should be used. No checks for user’s capabilities are necessary, as both hooks will be triggered for users with ‘upload_files’ capability, only.