How to override shortcodes.php core file?

You can’t “override” whole files or functions, except in special circumstances. That is basic PHP.

WordPress has code in place that will let you override particular files and functions. For example, child themes can override parent theme files. That is because of the way the WordPress Template loader works. The override only works if the files load via that mechanism. Those files are meant to be overridden. Core files are not.

WordPress also has a number of functions in pluggable.php that are wrapped in a if ( !function_exists('some_function_name') ) : conditional. If you declare a function with an identical name before that file loads, then your function will be used and not the default. Again, those are meant to be overridden.

And some hooks let you replace callback functions.

Otherwise you can’t override files and functions. You need to find another way to do what you want. Maybe that involves hooks, but you will need to did through the source to find the hooks you need.

For shortcodes in particular, you can remove shortcodes individually with remove_shortcode, and then replace them with your own but you can also clobber/hijack another shortcode just by registering another one with the same “tag”.

Proof of concept:

function test_sc($atts,$content) {
  return 'Howdy';
}
add_shortcode('testsc','test_sc');

function clobber_test_sc($atts,$content) {
  return 'Hi';
}
add_shortcode('testsc','clobber_test_sc');

However, shortcodes.php doesn’t actually contain shortcodes. It contains the shortcode support functions– the API. Those are not meant to be meddled with, and there are no hooks that I spotted.

As far as “Build an array and send data differently to other functions”, I don’t even know what that means but my guess would be that you can’t do it.