When you remove a higher power such as god, and his associated rules there is a void in man. Thus man sees none with dominion over him, and thus all is permitted.
This is why christianity has been the most prosperous religion, as it kept degeneracy to a minimum and provided a strong higher power to enforce man into delaying gratification for higher pursuits. All this without the negative set back side effects you see with Islam or Judaism.