Can I get away with the least expensive one?
That depends on what you want to do with it. I'm using the GoPro Hero HD, which is their second generation camera. GoPro has subsequently released the Hero 2, Hero 3 (and 3+), and Hero 4 series (with multiple cameras in each). The Hero 5 is rumored to come out in October. Each of the cameras (and it varies within a generation as well as between them) supports different resolutions, angles of view, and frame rates. Newer cameras have better image quality, especially if filming underwater. Newer cameras also support wi-fi streaming and remote control and have different battery run times.. I'm tempted to pick up a Hero 3+ Silver to get a wider field of view and better image quality. I don't need 4k resolution yet - 1080p is still good enough for me. Using the Hero HD with battery backpack, shooting 1080p 30fps, I can record 3-3.5hrs of continuous video (approximately 24gb) before the batteries run out.
Are you planning to record action footage, or lathe operation? If the latter, you may be better off with a small camcorder. They're less expensive and typically have long zoom lenses. The GoPro cameras are fixed focal length at a very wide angle. They're meant for sports video, not stationary filming.
I hope that helps,
Eric