First stop; Library or book store. Get Video's/DVD's visual aids on beginning golf (grip, stance, swing, equipment), read beginning books on same. Watch the Golf Channel. Golf looks easy, but requires a lot of continual practice of mechanics and mental confidence.
Not lesson oriented, too much cash roll out....try group lessons. Many courses, driving ranges have group clinics that are fairly inexpensive to get the basics. Rental clubs are generally available.
Obviously, the more money you put into rentals the less to buy; So, if you are thinking this is a sport/skill/activity you'd like to continue, think about getting a set of clubs. You get what you pay for, so research equipment and buy quality clubs...Demo them out at Golf Superstores (Golfsmith, Golf Mart, Roger Dunn). Once you have demoed them and know what you're looking for; start the hunt, new or used (eBay.com). TGW.com, Rockbottom.com and right now on sale at Costco Stores or Costco.com (free shipping) there are some great beginner clubs for medium to high handicappers. TaylorMade R7DX Full set, minus a putter and bag. $599.99. Again, you get what you pay for, so if you walk into Wallmart and leave with a full set/bag for 299.99 of some unknown brand 6mos down the road, I guarantee you, you will get itchy for a "real" set of clubs as you see your game progressing.
If you just want cheap and something to use immediately...Play it Again Sam, Swap Meet, local papers/garage sales, Craigs List, eBay.
When you begin playing, start with "Executive" courses or Par 3's. I know it's much more fun to just grip it and rip it, but learning the controlled swing or short game is essential to lowering your scores. Don't believe for a minute that you have a game....just because you can drive a ball 300yds. Focus on the controlled swing and feel.
Putting Green....practice, practice, practice. Establish a routine of practicing your putting, look for good drills to help out, learn how to read greens (If you were to throw a bucket of water on the green, which way would the water run/break?), learn how to feel/speed/lag putting. Make your 3-6ft putts no brainers, make your 3ft putts tap in's. Putting practice is free....do it often. Drive for show, Putt for dough.
Remember first step....Learning and drilling in the basics, finding your pre-shot routine, addressing the ball, swing mechanics. Take advantage of at least one round of group lessons and do rent some DVD's from your local library.