If the rotary valve seal is bad, the engine will pull in extra air (through the transmission, from the transmission vent), which would make it lean. That might make it not want to run well at higher RPM if it is really lean, otherwise it would likely make it run better (up until it seized). It will also pull in oil from the transmission, which will make it smoke significantly more that normal, especially if it is bad enough to affect the running at higher RPM. Just a slight leak at the seal would just make it difficult to start and idle and smoke a little extra, not affect higher RPM.
You do need the exhaust system to make it run right, having the baffle out might make it run a little better, but will also lean it down just a bit.
It does sound like you would have caught nearly anything I can think of external to the engine.
This could be an embarrassing question, but how much different is your weight now than 20 years ago? A 100 will perform much differently with a skinny 16 year old than with a normal 40 year old.