The Subject of AZW's or Anti-Zombie Weapons, has been long debated, and by people much smarter than me, I'll just give you my opinions, and a little bit of an over view of the things that I would want with me in a Zombie outbreak. If you're still in the preparation stages, strongly consider getting some professional training. But whatever you do, know how to use whatever weapons you have!
Knives Are always handy, but realistically, for a knife to be useful against a Zombie, one of two things has to happen. Either you must be way too close to that sucker, or you've got to throw the knife. Either way, you're screwed, so I would classify a knife more as a tool than a weapon. Don't get me wrong, if it comes down to it, use that knife, but when things are that nitty gritty, the choice of knife is "whatever's handy".
Handguns Are relatively compact, and tend to be less cumbersome than long guns, though they suffer with a shorter range, and lower accuracy. My recommendation would be to have at least 2 handguns for yourself, and try to equip the other members of your party with at least one per.
Rifles Tend to be more accurate, have higher velocity, and are more powerful than handguns, while being somewhat less easily portable. I would recommend having rifles of several variations available to your party. Personally, I would want a .22 caliber scoped rifle (preferably suppressed) for small game hunting, and picking off Zombies at ranges of up to about 100 yards. And a higher powered scoped rifle for taking bigger game, and for shooting the undead at greater range. These are generalities, check the gear section of the forum for more specific recommendations.
Shotguns Are a great close quarters weapon, and I'd imagine a shotgun with 00 buckshot would do quite a number on a Zombie out to about 40 yards or so, but the place the shotgun really would shine, would be in horde situations. Level that sucker at about shoulder height, and watch the Zombies drop like flies.
Other Weapons There's nothing wrong with using other weapons on Zombies, especially if you don't have any guns available to you, but I would highly recomend getting guns as quickly as possible. Unless of course, you want to be dinner.