You install a new version of R from the official website.
RStudio should automatically start with the new version when you relaunch it.
In case you need to do it manually, in RStudio, go to :Tools -> options -> General.
Check @micstr’s answer for a more detailed walkthrough.