Python throws ValueError: list.remove(x): x not in list

You should not remove items from a list you are looping over. Create a copy instead:

for a in aliens[:]:

and

for b in bolts[:]:

Modifying a list while looping over it, affects the loop:

>>> lst = [1, 2, 3]
>>> for i in lst:
...     print i
...     lst.remove(i)
... 
1
3
>>> lst
[2]

Removing items from a list you are looping over twice makes things a little more complicated still, resulting in a ValueError:

>>> lst = [1, 2, 3]
>>> for i in lst:
...     for a in lst:
...         print i, a, lst
...         lst.remove(i)
... 
1 1 [1, 2, 3]
1 3 [2, 3]
Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
ValueError: list.remove(x): x not in list

When creating a copy of the lists you are modifying at each level of your loops, you avoid the problem:

>>> lst = [1, 2, 3]
>>> for i in lst[:]:
...     for i in lst[:]:
...         print i, lst
...         lst.remove(i)
... 
1 [1, 2, 3]
2 [2, 3]
3 [3]

When you have a collision, you only need to remove the b bolt once, not in the loop where you hurt the aliens. Clean out the aliens separately later:

def manage_collide(bolts, aliens):
    for b in bolts[:]:
        for a in aliens:
            if b['rect'].colliderect(a['rect']) and a['health'] > 0:
                bolts.remove(b)
                for a in aliens:
                    a['health'] -= 1
    for a in aliens[:]:
        if a['health'] <= 0:
            aliens.remove(a)
    return bolts, aliens