Actually, your second attempt works for me in bash 3 and 4:
$ echo "$BASH_VERSION"
3.2.51(1)-release
$ echo "$foo"
"Hello World!"
$ [[ "$foo" =~ \".*\" ]] && echo $BASH_REMATCH
"Hello World!"
$ echo "$BASH_VERSION"
4.3.18(1)-release
$ echo "$foo"
"Hello World!"
$ [[ "$foo" =~ \".*\" ]] && echo "${BASH_REMATCH[0]}"
"Hello World!"
However, to talk theory for a second, it all has to do with how bash interprets the expression as a whole. As long as the regular-expression characters themselves aren’t quoted, the rest of the expression can be quoted without side-effects:
$ [[ $foo =~ '"'.*'"' ]] && echo $BASH_REMATCH "Hello World!"
but perhaps the easiest way of all is to use a second variable to hold the regex itself.
$ exp='".*"' $ [[ $foo =~ $exp ]] && echo $BASH_REMATCH "Hello World!"