Here's what is going on when the blade thickness and the insert thickness don't match.
Start with 1/8 blade and 1/16 insert as an example.
First cut, no issues, 1/8 material removed, insert glued in, everything looks great.
2nd cut, 1/8 material removed. The key here is that 1/8 thickness of the insert is removed as well since you have to cut through it.
Put 1/16 insert in and glue up. Since you removed 1/8 thick of the original insert, the two halves of the original insert will be slightly offset when you glue the material back together with the new insert.
Every cut after this will run into the same problem and will affect all inserts that are cut through.
NOTE: If you are using straight inserts and cutting 90 degress across the blank, then the blade/insert thickness really won't matter. It's cutting at an angle where this comes into play.
The larger the differences in sizes between blade and insert, the more obvious the error is. With the small sizes we use in pen making, even a small difference can be extremely obvious.
And I fully agree with Bad. Straight lines on a scrollsaw are actually more difficult than the curved lines. A bandsaw or tablesaw will give you a lot more accuracy.
Btw my first name is Randy too so I started to get confused by who was talking to who at first.