When to use the Filesystem API? Should I use it at all?

There is a bit of practical split about where WP can write files and if Filesystem API is invoked. It might be easier to see this divide not as technical, but as administrative.

There is user space.

Users must be able to do things like create attachments and must not need to have admin access credentials.

For these requirements uploads directory typically requires lax permissions and code that works with it is usually not using Filesystem API.

Rest is admin space.

Administrators/developers must be able to (over)write anything (plugins, themes, WP core itself) and have admin credentials.

For such functionality code is using Filesystem API and will prompt for credentials, if it can’t accomplish task without them.

So rule of a thumb for development distributable code is roughly – uploads directory is one and only place in WP installation where you can expect to perform writes without using Filesystem API.