As succinct background, per wikipedia, St. John of the Cross was born Juan de Yepes Alvarez into a Jewish converso family in Spain.
On the night of 3 to 4 December 1577, when he was 35 years of age, following his refusal to relocate after his superior's orders and allegedly because of his attempts to reform life within the Carmelite order, he was taken prisoner by his superiors, and jailed in the city of Toledo, where he was kept under a brutal regimen that included public lashing before the community at least weekly, and severe isolation in a tiny stifling cell barely large enough for his body. He managed to escape nine months later, on 15 August 1578, by escaping through a small window in a room adjoining his cell. (He had managed to pry the cell door off its hinges earlier that day.)
Mystic St. John of the Cross is considered one of the foremost poets in the Spanish language. Although his complete poems add up to less than 2500 verses, two of them — the "Spiritual Canticle" and "Dark Night of the Soul" are widely considered to be among the best poems ever written in Spanish, both for their formal stylistic point of view and their rich symbolism and imagery.
Dark Night of the Soul (in Spanish: La noche oscura del alma) is a treatise. It has become an expression used to describe a phase in a person's spiritual life, a metaphor for a certain loneliness and desolation. It is referenced by spiritual traditions throughout the world.
In the Christian tradition, one who has developed a strong prayer life and consistent devotion to God suddenly finds traditional prayer extremely difficult and unrewarding for an extended period of time during this "dark night." The individual may feel as though God has suddenly abandoned him or that his prayer life has collapsed. In the pronounced cases, belief is lost in the existence of God or in the validity of his religion.
Rather than resulting in devastation, however, the dark night is perceived by mystics and others to be a blessing in disguise, whereby the individual is stripped (in the dark night of the senses) of the spiritual ecstasy associated with acts of virtue. Although the individual may for a time seem to outwardly decline in his practices of virtue, in reality he becomes more virtuous, as he is being virtuous not for spiritual rewards (ecstasies in the cases of the first night) obtained and only out of a true love for God. It is this purgatory, a purgation of the soul, that brings purity and union.
Entering this dark night of the soul is commonly referred to in Buddhism as "raising the Great Doubt."
The most important and influential teaching of Japanese Zen Master Hakuin Ekaku was his emphasis on, and systemization of, koan practice. Hakuin deeply believed that the most effective way for a student to achieve insight was through extensive meditation on a koan. The psychological pressure and doubt that comes when one struggles with a koan is meant to create tension that leads to awakening. Hakuin called this the "great doubt," writing, "At the bottom of great doubt lies great awakening. If you doubt fully, you will awaken fully." Only with incessant investigation of his koan will a student be able to become one with the koan, and attain enlightenment.
Or, as Mark Epstein and Jonathan Lieff wrote, quoted in Ken Wilber's book Transformations of Consciousness:
…when the perceptual capacity to discriminate very fine changes in moments of consciousness is developed, regression in service of the ego has become transmuted to inspection in search of the ego. A period characterized by the subjective experience of dissolution is entered where traditionally solid aspects of the personality begin to break up, leaving the meditator no solid ground to stand on. This is traditionally the time of spiritual crisis, characterized by “a great terror,” the “Great Doubt” in Zen, and the struggle to allow a transformation or “decathexis” of the self.In online Buddhism, arhat Daniel Ingram is one who speaks and writes about Dark Night. In the 25th chapter in his hardcore-dharma book, Mastering the Core Teachings of the Buddha, he writes,
There are two basic things that happen during the Dark Night. One is that our dark stuff tends to come bubbling up to the surface with a volume and intensity that we may never have known before. Remembering what is good in our life can be difficult in the face of this, and our reactivity in the face of our dark stuff can cause us staggering amounts of needless suffering. On top of this, we also begin to directly experience the fundamental suffering of duality, a suffering that has always been with us but which we have never known with this level of intensity or ever clearly understood. We face a profound and fundamental crisis of identity as our insight into the Three Characteristics begins to demolish part of the basic illusion of there being a separate or permanent us. This suffering is a kind of suffering that has nothing to do with what happens in our life and everything to do with a basic misunderstanding of all of it.In a video, "Dark Nights of Meditation Practice," Ken Wilber responds to a young woman struggling with her Integral Life Practice. She's been experiencing discomfort, anxiety, and fear for her sanity while meditating. Wilber explains the "switch-points" between the major states of consciousness, each marked by its very own Dark Night: a death and rebirth of identity that can be scary to experience.
Dealing with either of these two issues, i.e. our dark stuff and our fundamental crisis of identity, would be a difficult undertaking, but trying to deal with them both at the same time is at least twice as difficult and can sometimes be overwhelming. It goes without saying that we tend not to be at our best when we are overwhelmed in this way.
The knee-jerk response often is to try to make our minds and our world change so as to try to stop the suffering we experience. However, when we are deeply into the Dark Night, we could be living in paradise and not be able to appreciate this at all, and so this solution is guaranteed to fail. Thus, my strong advice is to work on finishing up this cycle of insight and then work on your stuff from a place of insight and balance, rather than trying to do it in the reactive and disorienting stages of the Dark Night! I cannot make this point strongly enough. ...
[The fruit of Dark Night is] the first attainment of ultimate reality, emptiness, nirvana, God or whatever you wish to call it. In this non-state, there is absolutely no time, no space, no reference point, no experience, no mind, no consciousness, no nothingness, no somethingness, no body, no this, no that, no unity, no duality, and no anything else. Reality stops cold and then reappears. Thus, this is impossible to comprehend, as it goes completely and utterly beyond the rational mind and the universe. To “external time” (if someone were observing the meditator from the outside) this lasts only an instant. It is like an utter discontinuity of the space-time continuum with nothing in the unfindable gap.
The initial aftershocks, however, can go on for days, and may be mild or spectacular, fun or unsettling or some mixture of these.