Hi, Dana. I was struck by your comment "I'll admit I struggle with both jazz and classical as a listening choice I just figure I'm not smart enough to get it . . .". I guess this post is a rambling "response" to that (it is not meant as an uninvited suggestion on how you listen to music!).
Like many born in the late 1950s, the music I heard the most growing up was rock music, and a lot of that was blues-based, and I have never felt a need to "study" this music - it just seems to make sense to me because it was just part of the environment I grew up in. For example the
1970 self-titled album "Leon Russell." 
When I starting to listen to jazz, I usually only liked a very limited percentage of the songs I listened to. The songs I liked were enough to keep me exploring (why explore if you don't like
any of it!), and over time I began to appreciate more of it - I don't think that reflects intelligence or study. Instead, for me it is just exposure to more music in the genre. I generally move slowly through the music by following favorite musicians or sub-genres that slowly expanded my exposure.
In writing this, I went back to the first recording I bought with sax player Sonny Stitt -
Sonny Stitt, Bud Powell, J.J. Johnson on Prestige (with a very strange cover!):

When I first heard this, I liked the introductions, but then got lost in the middle of each song until the melody of the introduction got repeated at the end. I liked enough of it to keep looking for other recordings by him. Revisiting this recording tonight for the first time in many years, it seems to make sense now, I think just because I have heard more. Glad I revisited it!