It isn't my responsibility to judge outsiders, but it certainly is your responsibility to judge those inside the church who are sinning. God will judge those on the outside, but as the scriptures say "You must remove the evil person among you."
We should never associate ourselves with believers whom still continues to do evil. "For a rotten tomato placed with other non-rotten tomatoes will surely rotten them all". Thus, we must stay away from believers who still continue to do evil so that we may not become like them.