PHPUnit would come to mind, if the WP test suite wasn’t so broken, and if WP had been designed and written in a way that it could actually be tested properly. 😉
More seriously, you can test your plugins all you want from their functional standpoint with unit tests and the like. The issue is that these tests won’t guarantee that they’ll catch subtle chances introduced by WP upgrades, let alone that they’ll continue to work once plugged into a customized WP install.
Among the colorful things I’ve seen happen:
-
A subtle change in the WP API affects your plugin’s functionality, e.g. the hook you’re on used to get a term id and it’s now getting a term taxonomy id. (Chances are good that your test terms conveniently have the same id for both).
-
A subtle change in the WP API results in your receiving a
WP_Errorobject instead of the previously expected value offalseas bad input. -
Your plugin is added from within the mu-plugins folder, resulting in a subtly different code flow.
-
Your plugin worked fine until memcached or some other persistent store got enabled.
-
Your plugin worked fine until the despised switch_to_blog() got called.
-
A plugin changes the hook which it resides on when called, and unknowingly interrupts it as a side effect.
-
A plugin (un?)knowingly messes around with your input or output data to the point where things look broken even though you’re not at fault.
I could expand the list on and on, but those would be the key items that broke my own plugins. The two items are arguably catchable with unit tests. The next two as well, if you’re patient enough, but I’d argue that WP should not change the way things work when they occur. No amount of testing will work around the buggy implementation of switch_to_blog(). And the last two are hopelessly untestable.
Oh, and… don’t even get me started on the onslaught of attachments, auto drafts, revisions, menu items and what not that end up stored in the posts table.
Good luck… 🙂