Should I worry about SQL injection when using REST API?

Yes. This is not secure at all. You’re putting user input directly into a database query. You need to use $wpdb->prepare() if you’re inserting user input into SQL:

$post_slug = $request->get_param( 'slug' );

$query = $wpdb->prepare(
    "select * from wp_posts where post_name=%s and post_status="publish" limit 1;",
    $post_slug
);

$results = $wpdb->get_results( $query );