I want to show category name when using [category] shortcode

Put the code below in your functions.php file:

function register_theme_shortcodes() {
    add_shortcode( 'category',
        /**
         * category shortcode function.
         *
         * @param array $atts
         *
         * @return string
         */
        function ( $atts ) {
            $atts = shortcode_atts(
                array(
                    'id' => '',
                ),
                $atts
            );

            /**
             * @var int $id Optional Post ID to get categories from.
             */
            extract( $atts );

            if ( $id ) {
                $post_id = $id;
            } else {
                global $post;

                if ( $post instanceof WP_Post ) {
                    $post_id = $post->ID;
                }
            }

            if ( empty( $post_id ) ) {
                $output="";
            } else {
                $output = [];

                if ( $post_categories = get_the_category() ) {
                    /**
                     * @var WP_Term $category
                     */
                    foreach ( $post_categories as $category ) {
                        // Builds the category name with its link.
                        $output[] = "<a href="" . get_term_link( $category->term_id ) . "">{$category->name}</a>";

                        // Build just the category name.
                        // $output[] = $category->name;
                    }
                }

                $output = implode( ', ', $output );
            }

            return $output;
        }
    );
}

add_action( 'init', 'register_theme_shortcodes' );

Make sure you pay attention to the comments along the code as you can have the category names output with their links or not.

And if you want to print the category of any other post, outside the post, the code above also supports the following format: [category id=1234].