Put your code in the template file category.php.
Remove all the part before the loop: once in category template, you don’t need to get the category, get the paged, run again the query with query_posts…
So your category.php should simply appear like so:
if ( have_posts() ) : while ( have_posts() ) : the_post();
$image = wp_get_attachment_image_src( get_post_thumbnail_id( $post->ID ), 'single-post-thumbnail' );
?>
... post markup here
<?php endwhile;
else: ?>
<p><?php _e('No posts found'); ?></p>
<?php endif; ?>
<!-- Pagination Part -->
<div id="pagination">
<div class="next"><?php next_posts_link('next »') ?></div>
<div class="prev"><?php previous_posts_link('« previous') ?></div>
</div>
To force that template to show only 4 fosts per page, in functions.php use:
add_action('pre_get_posts','four_post_per_cat');
function four_post_per_cat( $query ) {
if ( ! is_admin() && is_main_query() && is_category() ) {
$query->set('posts_per_page', 4);
}
}
After that numberposts and posts_per_page are synonyms, but numberposts is deprecated. Setting different values for them make numberposts do nothing (or posts_per_page do nothing, I can’t remember… however, use one of them).
If your scope is limiting the total posts reached (in all the pages), use the post_limit filter, in functions.php add also:
add_filter( 'post_limits', 'cat_post_limits' );
function cat_post_limits( limit ) {
return ( is_category() ) ? 'LIMIT 0, 50' : $limit;
}
Following my tips, not also you’ll solve your issue, but also improve performace: because query_posts is very bad regarding performance: never use it.
A note: if you have added any rewrite rule, be sure to flush rules. In you dashboard go to Settings -> Permalinks and click “Save Changes”.
PS: If you get at max 50 posts, 50 is not divisible by 4, so last page will have 2 posts.. why don’t set limit to 52 or 48?