How to list all record from one custom taxonomy start with only one letter i.e A

You can use preg_match function:

foreach($stores as $store)
    // if $store starts with A or B
    if( preg_match( '/^(A|B|a|b)/', $store ) )
        //I've not checked this one
        $groups[mb_strtoupper(mb_substr($store->name, 0, 1))][] = $store;