How can I make a list of lists in R?

You can easily make lists of lists

list1 <- list(a = 2, b = 3)
list2 <- list(c = "a", d = "b")
mylist <- list(list1, list2)

mylist is now a list that contains two lists. To access list1 you can use mylist[[1]]. If you want to be able to something like mylist$list1 then you need to do somethingl like

mylist <- list(list1 = list1, list2 = list2)
# Now you can do the following
mylist$list1

---

Edit: To reply to your edit. Just use double bracket indexing

a <- list_all[[1]]
a[[1]]
#[1] 1
a[[2]]
#[1] 2