How can I find the hook I need to rewrite a function?

BuddyPress defines a class named BP_Group_Extension which can be used by plugins or themes as an API for creating group extensions. This base class is located in buddypress/bp-groups/bp-groups-classes.php, and there is documentation on how to use it in the file at the beginning of the class definition (starting at line 1742).

bbPress uses this BuddyPress API to modify the BuddyPress functionality. In bbPress, the class where you found the create_screen function is named BBP_Forums_Group_Extension and it extends the BP_Group_Extension class. If you want to create a plugin to modify this further, you can extend the class further and redefine the create_screen function.