Using Bash regex match (=~) where regex includes quotes (” characters)

Actually, your second attempt works for me in bash 3 and 4:

$ echo "$BASH_VERSION"
3.2.51(1)-release
$ echo "$foo"
"Hello World!"
$ [[ "$foo" =~ \".*\" ]] && echo $BASH_REMATCH
"Hello World!"

$ echo "$BASH_VERSION"
4.3.18(1)-release
$ echo "$foo"
"Hello World!"
$ [[ "$foo" =~ \".*\" ]] && echo "${BASH_REMATCH[0]}"
"Hello World!"

However, to talk theory for a second, it all has to do with how bash interprets the expression as a whole. As long as the regular-expression characters themselves aren’t quoted, the rest of the expression can be quoted without side-effects:

$ [[ $foo =~ '"'.*'"' ]] && echo $BASH_REMATCH
"Hello World!"

but perhaps the easiest way of all is to use a second variable to hold the regex itself.

$ exp='".*"'
$ [[ $foo =~ $exp ]] && echo $BASH_REMATCH
"Hello World!"