Car aside, we budget about $1,450 for the race fee (+$200 each if you have 5 or 6 drivers), about $500 for a set of tires (mounted and balanced) - more after a few races when really good tires will make a difference, $250-$300 in gas for the race (more if you run a Crown Vic or other large, thirsty car), towing gas (depends on how far you have to drive), plus food and lodging. That assumes you already have individual equipment. Generally speaking, initially plan on $3,000-3,500 per race unless you live really close to a particular track.
Since you mentioned being completely new to this, check the Lemons gear page for prices on driver's gear if you don't have helmets, suits, and such. That is the "easy button" solution, and their prices are actually pretty reasonable.
If you don't have a car, plan to spend about $4,500-$5,000 to buy a car and get it ready; a little over half that if you can do your own cage (meaning you have a really talented welder as part of your team). Check other threads on this forum for more info on car build costs.
Tips on saving money:
Buy a used car. We just sold one for way less than we had in it. If you are just looking for fun, another team's old (slower) car is a ball of fun to drive on a track and way cheaper than building one. Plus the headache of getting a car ready for a race is far, far less than the headache of building a "new" car and getting it ready for a race.
You can save money by dedicated trolling of eBay for closeouts and lightly used suits, helmets, shoes, gloves, etc. Stores dump their last year's versions, unwanted colors, or odd leftover lots, and a fair number of drivers race a time or two and then move on to a new hobby. Make sure it is "SFI" or "FIA". SFI-1 requires Nomex undies, SFI-3 doesn't. We picked up a couple of closeout suits to have on hand for newbies or unpaid crew who didn't have their own. I think the suits were about $50 each; of course they were closeouts because nobody else wanted the XL canary-yellow one-piece suits, but we didn't care. Check often and be prepared to check regularly for a while to find deals. If you are in a hurry, check the Lemons store.