If you really JUST want to get into basic pen and razor handle turning (with some seam rippers and keychains on the side) the Harbor Freight Mini Wood Lathe (8x12) for 130 dollars is probably the best deal you'll see. Wait till there's a 25% off coupon and you can take your lathe home for 100 dollars + tax.
This 1/3 HP mini wood lathe comes with electronic variable speed control ... only one belt and pulley, just turn a knob and you have between 750 and 3200 RPM, though I think the low speed on mine is half the stated value, around 300 RPM. I have more than enough power to turn razor handles, pens, keychains, even bottle stoppers ... and if I feel like it, small bowls and other little projects.
In order to turn the small items, you'll need a mandrel and the kits ... may as well get all that from PSI. I highly recommend the MT1 mandrel saver kit, which will come with 5 7mm bushings, the MT1 mandrel, the MT1 mandrel saver, and the 7MM brad point drill bit. After that, pick up the razor handle kit of your choice ... perhaps toss in some wood or acrylic blanks, and if you feel up to it, grab a starter kit for some pens or keychains too!
Your wife or mom (or some other lady in your life) will likely appreciate a hand made seam ripper for sewing projects too ... and handmade turned Christmas ornaments aren't all that difficult!
As for lathe tools ... Harbor Freight, 8-pack of lathe chisels for around 15 dollars. They aren't very sharp right out of the box, so unless you have a decent sander already, grab one of the combo sanders from Harbor Freight and that should help you keep a good edge on your tools. You'll want to keep the same bevel angles, to start with, and just put a good sharp edge on them. You'll also want to pick up your sandpaper in packs from Harbor Freight as well ... decent quality at a good price.
While you're there ... their tubes of superglue make an excellent finish for wooden items, especially if you are making razor handles! Always make sure to seal the ends of your razor handles as well, to keep the ends from absorbing water and swelling. I put 9 coats of CA on my cocobolo safety razor handle, and it's held up well for over a year now!

Still shines like the day I finished it, chrome and all.
The mini wood lathe has #1 Morse Taper attachment points, as I mentioned above. The spindle is threaded for 3/4" x 10 (10 threads per inch, 3/4 inch wide). You'll want to ensure that any accessories you get for it will fit or have an attachment to let you fit it to this lathe. Most lathe accessories you get WILL be made for a midi lathe or larger, with a 1" x 8 spindle attachment, but 90% of them will ALSO come with an adapter that will attach internally to a 3/4" x 10 spindle, adding barely half an inch to your spindle length. I have no problems working with my 4-jaw chuck.
The lowest cost way to obtain a chuck of some sort for this lathe is to order one online through Harbor Freight Tools. They have a 3-jaw scrolling jaw chuck with a #1 Morse Taper built in. It's only about 30 dollars + shipping. The nearest competitor is PSI at over 50 dollars + shipping, and the quality is pretty much exactly the same. (edit - forgot PSI's 3/8ths drill chuck with #1MT ... it's only around 15 bucks, and is an improvement over the handheld drilling, so get one when you can!)
I generally just turn my blanks round between centers (spur center and live center), and then grip it in the 3-jaw chuck (automatically centers round objects) to cut it to length ... Once I have the length I want, then I put it right back into the 3-jaw chuck and drill it. I took the 3-jaw keyless chuck off of a cheap 12-dollar Harbor Freight corded 3/8ths drill and put my 7mm drill bit in it ... and HELD IT BY HAND to drill my hole through the blank as it was spinning on my lathe. Make sure you have a center hole and hold the bit steady as you drill .... the spinning blank will NATURALLY self-center the hole. Just don't pull it to the side as you drill.
After that ... superglue the barrel into the blank, put it on the mandrel between the bushings, and turn it to your satisfaction .... sand, finish, polish, give it a coat of wax for extra protection if you like, and enjoy!
Now ... if you EVER decide to move up to a larger lathe in the future ... the only items you likely won't be able to re-use is anything with a 3/4" x 10 spindle thread or #1MT. In my case, I'll still be able to use my 4-jaw scrolling chuck from PSI, but I'ld have to replace my mandrel and mandrel saver as well as my drill chuck #1MT and my 3-jaw scrolling chuck with #1MT. All in all, that's not that big a replacement cost ... we're talking less than 70 bucks lost.