I’m pretty sure you’d define the custom post formats in functions.php, then they’d reference their own PHP file which contains the loop you’d need.
So for example, Text format would go to post-text.php, Image format post-image.php, etc.
You could add your match filters into a function within functions.php, and then call the function from within the post format PHP files.
Some info here: http://en.blog.wordpress.com/2013/01/15/post-formats
How-to is here: http://wp.tutsplus.com/tutorials/proof-using-post-formats.