Almost all drugs (including alcohol, nicotine and caffeine) can cause damage to us. Whether people should be allowed to take drugs comes down to a more general argument of whether we should be allowed to take risks. The current attitude of society towards risk is completely illogical.
A good example is driving. Once you have passed your test, you are allowed to drive vehicles at sigificant risk to yourself, but more importantly to others as well. Taking a drug can only cause risk to yourself and yet many drugs are illegal.
The argument that drugs cause harm to others because of their involvement with crime is complete bullshit. This is simply because they are illegal. Any product that is prohibited automatically becomes profitable for criminals.
Anyone involved in 'The War on Drugs' needs to think about some simple economics. Destroying the supply is useless if there is a demand.
The extreme of the risk argument is whether suicide or euthanasia should be legal.