Re: I've tried to like delay but it's such a finicky effect in my experience...
There are hundreds of ways to use delay 'correctly'. It's hard to even talk about delay as a singular effect; analog, 'tape' and digital delay produce effects as different as a flanger and a chorus. They have a very similar effect but the subtle differences really shape the way you use them. 
I generally use Digital (specifically older/Lo-Fi style Digital) delay on solos, and on pieces with a lot of notes. I generally set it to one-two repeats, with a delay time around 300ms ish and the mix turned down. It helps fatten up the sound of a solo, and the lack of tonal variation (no modulation, fairly close reflection of the original signal) help it sound more in tune and stops the solo from getting too crowded.
I like Analog Delay for two main purposes: Out of control oscillation and slapback. A little touch of modulation helps fatten up the sound or get you into the weird extremes of pitch vibrato that tape delay can't really do. Analog is pretty versatile, you can set the mix really low and get that dark reverby sound, you can turn the delay time all the way down and get a dirty springy slapback, you can open it all up and have crazy, messy repeats that feed infinitely into one another and crackle out into a fuzzy mess that distorts and breaks as you play with the pitch via the delay knob. Yeah it can get pretty weird, but I've used it in live sets in our more.. 'free' songs. Even for the slapback I generally like setting the repeats a bit higher so you get a twangy almost feedback-like edge that sounds really nasty when put into a pushed amp. Analog can be the most subtle and user friendly.. but I like it for the weird.
'Tape' delays refer to the multitude of digital reproductions of tape devices on the market now. They're really good, really cool. Ironically, my favorite way to use 'tape' delays is a setting not possible on older tape machines, with the delay time set really high. The limit on most of these older tape delays was maybe 300 ms, but I generally like anywhere between 400 and 1000 for those really spacey, warbly ambient sounds. I think these are also the hardest to use without your music sounding like an absolute mess, because most of them tend to run the delay through a filter which cuts out the bass (as old tape decks tended to do naturally) meaning that the delay sounds are really bright and present. It means that the repeats will always stand out, making it more important to really lock into a rhythm or risk your song sounding like a crowded mess. 
Delay is probably my single favourite effect, every other effect on my board has it's tasteful use, but I'm almost always looking for an excuse to throw a little delay into a song. My small travel board has had the carbon copy on it for years, just because it can cop most/all the flavours I listed above, and it can do the weird stuff too. I remember feeling the same as you when I was 'learning' delay. The best way to stop it from feeling finnicky is to consciously step away from your board. Turn the delay on and go and stand somewhere that you wouldn't be tempted to play with it. Just pick a setting and sit there for 20 minutes and see what (if anything) you can come up with. Delay in and of itself isn't finnicky, you are as a player. You can also check out different gear channels (JustNick, Proguitarshop, Mike Hermans, Pete Thorn) and just listen to their reviews of delay pedals to get more of the possible sounds in your head. Pete thorn just did an awesome one recently where he NAILED the van halen delay sound with a Boss DM-2. 
And if at the end of all this you decide that delay still isnt for you, that's cool. Just because delay fits me like a glove doesn't mean we have the same shaped hand!