The ignition switch has several positions, and it's failing to properly transition from the "start" position to the "run" position. When you hold the key, you're manually forcing power to the necessary circuits to keep the engine running. When you release, the faulty switch isn't making the proper connections for the continued operation.
Other less likely possibilities, but still worth checking (after the ignition switch) include:
* Neutral Safety Switch: This prevents the car from starting unless it's in park or neutral. A faulty switch could be intermittently preventing the engine from running after the initial start.
* Wiring Issues: Corroded or damaged wires in the ignition system could be causing intermittent power loss. This is less likely to be the primary cause if it starts easily when the key is held.
In short: The ignition switch is the most probable culprit and should be replaced. A mechanic can easily diagnose this issue definitively and perform the repair. Attempting to repair the switch yourself can be tricky, but it's possible for experienced DIYers. However, for safety and to avoid further damage, professional assistance is recommended.