Hey b,
I won't make an attempt to describe how distortion sounds using words, but I could write hate mail about it for days. Keep in mind that low frequencies are produced by a combination of driver and enclosure and distortions can be produced by either.
According to this old 4T (My definition)
Distortion = any tone or lack of tone that is added to, subtracted from, or in any way different from the original. I also expand that to include any deviation from the original shape of the waveform caused by numerous compromises made during manufacture of a so called low frequency transducer system.
To me, the most annoying distortion and the most audible is InterModulation (IM) distortion, which means that a tone is added to a complex signal as a result of two tones reproduced at the same time causing a third tone to be heard which is not musically related to the other two. This is usually caused after the amplifier, since most amps have decent IM specs.
The same is true with harmonic distortion (THD) except that it occurs with some harmonic consistancy and is musically related to the original tone. The second harmonic of a tone is exactly 2x the frequency of the original and follows a simple arithmetic progression (multiples: 2x,3x,4x,5x). this doesn't sound too bad as long as the 2nd is softer (less power) than the fundamental and the 3rd is softer than the 2nd and so forth, like a musical instrument. Musical instruments are usually harmonic on the even order and we are used to those. Mostly high order harmonics can become louder in a distorting driver and that increases fatigue on the listener and is unnatural sounding. Most people seem to hear odd order harmonic distortion sooner than any other type, since it so non-musical.
Nonlinearity distortion comes in many flavors. The term usually describes the inability of a driver to respond to rapidly changing voltages in the signal. A driver sometimes moves in one direction "easier" or "farther" than in the other and this causes all sorts of problems. The driver must also stop moving when there is no signal voltage, but many will continue to "ring" when they are supposed to shut up.
Power compression* is also a form of distortion, in my book, and is audible on powerful peaks as it lops off the peaks of the waveform by being too stiff to actually keep up with the changes in the signal voltage. This causes a kind of mechanical clipping and is very common. Generally power compression is heat related and increased power causes more heat and more compression. Any respectable driver should reach its Xmax without any power compression.
For that matter, any decent driver should not introduce any distortion within its rated parameters, but this is not reality. Truly excellent drivers can add as much as 5% distortion and be still be considered acceptable by current standards. Many high excursion drivers add as much as 15% or even 25% of their own anomalies to the sound. That is one of the reasons I tend to use powerful large diaphram Musical Instrument drivers with large magnets and small Xmax, in multiples, instead of one large Xmax driver to achieve the desired SPL.
Of course, it is much more complicated than this in practice, but that's a brief description of some of the basics. These types of distortion are the most noticeable to me and my ears, which have many years of use and abuse on them, but are still very sensitive for an old 4T and were well trained at one point in time.

*MOD:
Just to be more clear, power compression is an inability of a driver to get louder as you turn it up. Any driver will reach a point where they can't increase acoustical ouput predictably no matter how much more amplifier power you put into them. The rules are: if you double the power input, the driver output will increase by 3dB. A driver that is in compression due to overpowering (heat increases resistance in the voice coil) or its mechanical limits (Xmax) will no longer follow the rules. Unfortunately many drivers (Power) compress long before the rated Xmax is reached.
I think I fixed all of my typos.