The ticking is from an exhaust leak that I just found. It sounds just like the one in the video I posted. Not very noticeable until you get up to 4th and 5th gear. I couldn't hear it when I bought the car because it had a large exhaust leak after the downpipe to give you an idea of the volume of it.
I also just double checked and it doesn't sound any different when idling at a stop with the clutch in versus clutch out. That is the part that has me most stumped.
I've read that a short shift kit can sometimes make noise and the previous owner did have one installed so I'm going to check that now and grease everything back up.