Upon the death of the human body, the more spiritually evolved aspect of the soul crosses into the light, which I believe is a portal into a higher dimension of consciousness. The vibration or frequency of that dimension is much higher than that of the Earth plane. Part of the soul that is "left behind" when the body dies is known as the astral body, which contains the memories of the life experiences it gained on Earth. These negative (low-vibrating) emotions and thoughts may get stuck in the Earth plane, and be seen as ghosts or apparitions. It may also fuel itself on the negative emotions, thoughts, and habits (such as substance abuse) of a human host through that person's astral field/body, to which it attaches itself. If the entity gains enough fuel or energy, it can begin to move objects or cause problems for other humans who may not even realize this is what is happening. The good news is that this negative aspect of the soul can still learn lessons, change its ways, and prepare for reincarnation while in this state of afterlife. If it achieves enough insight (positive energy that does not come from siphoning from humans) to raise its vibration or resonate at a higher frequency, it can move into higher dimensions and join with higher evolved aspects of itself. Many people are skilled in helping these Earthbound souls cross over.
My information comes from my personal experience with dis-incarnate spirits as well as from Ethan Vorly's work and The Spirit's Book by Allan Kardec. I hope that brings a little more clarity to this mystery of souls and the afterlife.
You might enjoy reading my book, More Than Meets the Eye True Stories about Death, Dying, and Afterlife. Purchase paperback on Amazon.com. It's also on Amazon as an e-book for those who have Kindle or Sony Readers. The audio book is now available!