i worked for few years with screw press, making custom coins from pre-made brass or copper discs for souvenir shop, closely to the size of the 1ozt gold ones. it was a historic manual machine from like 40s or 50s, building your muscles quickly as some days you should make some 200 rounds
tiring but interesting work.
i never needed to cast my own "empty" coins, but working with copper was relatively efortless, compared to brass. but after heat treatment, brass was much easier to press (as expected, but oxidation patterns should be sanded off and roughly polished before). i am assuming that pure gold or silver would be soft enough to mint at home with some basic press, but i am concerned about diameter.
maybe some sacrifice of diameter could be beneficial for you, because pressing area will reduce significantly (sqare order). you could than use much less pressing power to obtain good results.
i personally would not use hand held hammer, espetially on high end matrices (brittle, extremely wear resistant steel). maybe some sort of falling weight would assure the proper direction of force
when attempted to hit another time after the die moved from position after the first whack, "focus" the dies by means of turning them gently, and you will hear a "click" - mint on the coin properly fit the die, and then you can whack again without making "double pressed" coin
wishing luck with this beautiful hobby
illustrative picture of the machine, one that i operated was somewhat bigger, but man... it weights somwhere about 2 tonnes
delivery of the dissasembled thing and properly getting it to the room, and assembly of the thing onto two cubic meters of concrete (so it wouldn´t dance around)... that was hell of a work
4 man, 2 days