Is it possible to display a DIV to not logged in users and then change it for users who are logged in?

Yes it’s possible, just use the is_user_logged_in() function as descibed in the codex for this.

It would go like this:

<?php if ( is_user_logged_in() ) { echo "you are logged."; } else { echo "You are not logged"; } ?>