Right now I'm trying to move from playing songs to coming up with my own material. The only issue is there are so many different ways to improvise that I'm kind of blinded by the options. More or less here's what I've done in order
1: Applying rythmns and embellishments to cowboy chords (eg, suspending a chord only for a beat or two before a chord change). Also I'll group in this step, basic arpeggios by hitting chords one string at a time in various orders.
2: Repeat step one but using various CAGED positions for each chord
3: Creating simple lead lines by quickly switching through triads based off those CAGED positions.
What was you guy's paths and where do I go from here?
Well done! There are many factors that come together with a great improvised lead, however, one of the most important is seeing the chord tones. It makes melodies far more accessible and tends to limit rambling. You are already way, way ahead of most! keep the thinking cap and your ear going and you will go far.
Speaking of rambling, when you're starting off, I suggest avoiding pentatonics. They are certainly great spice once you know what you're doing, and I've got some students that arent going anywhere else, so at least they can play something
But since you're taking your time, first learn your major scales in all positions. This is much easier than it sounds because unlike other instruments, guitar patterns simply move up and down the fretboard. I won't go into the detail, because you can find this on hundreds of websites, but you have essentially three fingerings off the e string and one off the a string that will cover a good portion of the neck.
And then when you need them, the pentatonics in those positions are easy.. just leave out the four and the seven. It's much easier to remove notes, then to add the notes that your fingers aren't used to playing.
There's one more form that helps you complete most of the neck.. The mixolydian mode, which is also the dominant mode.. it's much, much easier than people make out. The one you need to start is exactly the same as the "under" major scale except the 7th is lowered. Again it's really easy, here's how that would work.
The under major scale starts with your pinky on the 5th fret and your first finger on the second fret of the e string.. let's talk about the first string for this example. In this position, you get an A scale if you do the do re me thing down from the 5th fret. If you play the same intervals from the f sharp, you are playing the relative minor, another extremely useful scale, so two scales in one.
So once you know that, you can shift that exact same pattern up to the 12th fret. Your pinky is now on the octave e and the only difference between the A major scale you just learned and the e dominant mode is the d note. Because e is the 5th of a, it's the dominant of a and requires the lowed seventh that were used to seeing with dominant chords.
So you come down the scale from e to d, a whole step, as opposed to the half step coming down from the major scale. This isn't gobbledygook if you go put your fingers on a fretboard and do it a few times. once you have the majors and this dominant mode, you have covered most of the neck.
There are tons of other patterns and scales and modes that I use to connect everything I just mentioned... But they are all gravy... you can do just about anything you want if you have your majors and this one dominant mode. Combined with your knowledge of chords, you may already have the neck at this point.
Someone already mentioned stealing riffs... This is actually one of the best ways to find good pentatonic movements, especially in jazz and blues.
And then get your hands on every melody you can possibly think of. When you play a song melody, you will note that it crosses over many of the chords that you can already see, but most also favor scales and you be able to see the passing tones that are in between and the inharmonic notes that are not within the scale. Since you will already understand scale fingerings, you can easily remember that I need to "flatten the second" for this particular lead. The more melodies you play, the more you will see.
When I was a kid, one of the best bands to copy was Boston. Schultz's leads are not hard to play, the complexity is really due to the harmonized leads that are overdubbed. Pick any one of his riffs or even one of his big melodic leads, like the middle of don't look back, and learn it by ear, or off of YouTube, or reading tabs...whatever it takes. Combined with your knowledge of chords, you will quickly see that he uses a lot of major scales... Again it's easy to simplify to two note riffs and Pentatonics once you've have the scales but if you push into this for a couple of months the fretboard will come alive.
Last but not least, continue using your ears and mind. Approximately half my students come in saying I don't want to learn theory, I only want to play by ear and the other half come in saying I mainly want to understand theory. Of course, reality is smack in the middle of the two. We need both over time to be good so you might as well learn both from the beginning
So keep up the good work and let us know how it goes!