Think of it like this: the throttle body is a gate controlling airflow into the engine. A bigger gate (larger throttle body) *could* allow more air to flow *if* the rest of the intake system (air filter, intake tubing, etc.) and the engine itself (cylinder heads, exhaust, etc.) can handle the increased airflow. If the rest of the system is restricted, a larger throttle body won't make a noticeable difference, or might even slightly reduce performance due to turbulence and poor air distribution.
In short: a larger throttle body is often part of a larger performance modification package, not a standalone horsepower adder. It's rarely a worthwhile upgrade unless you've already made significant other modifications to the engine and intake system.