If you have to reach out and touch a Zombie, it's best to touch a Zombie from a distance. Unfortunately, guns make noise and attract more Zombies. Bows and crossbows can be unwieldy in close quarters and hard to reload quickly. For close quarters fighting against Zombies, bats, hatchets, axes, machetes, and knives are the typical weapons used by survivors. However, if you had a choice wouldn't it be better to kill a Zombie from a little more of a distance? When it comes to killing Zombies, you might want to get medieval.
In ancient and medieval times, soldiers used spears, pikes, lances, and other pole arms. These were intended to allow soldiers to jab enemies from behind a wall of shields or to reach up and engage mounted horsemen. They typically had spear points that could pierce through armor. Some weapons such as a halberd had a spike and an axe built in. Both of these features would be outstanding against Zombies.
When it comes to Zombie killing, a spike or spearhead could penetrate a skull or a spinal cord. A pole axe might be able to remove a head. Since Zombies don't feel pain and aren't intimidated by martial displays, they may well impale themselves on a spear or pike.
Unfortunately, when the Zombie apocalypse starts, most people won't have a spear or a halberd laying around. However, if you are really getting ready, you could be the first in your neighborhood to own a pole arm. I think the most practical modern pole arm would be the Cold Steel Boar Spear. But, there are plenty of halberds available on Amazon. In a pinch and with some access to metal working tools a shovel could also be converted into a fearsome anti-zombie weapon.