Chances are that you have a vacuum leak somewhere that is leaning out the engine.
By pressing down on the pedal, essentially you are fooling the computer into "thinking" that
you are driving at a higher speed (requires more fuel) so, it increases the amount of fuel going in and it starts. That will also cause a high or poor idle.
If you can't find a leak by just a visual inspection, try putting one end of a short length of hose to your ear and moving the other end along any vacuum lines, fittings and manifold/block connection while it's running. It looks silly when you do it, but it works.
And yes... There are other things that can cause your problem, but simple leaks are the best place to start testing.