You'll need to consult your owner's manual. The manual will have a diagram showing the fuse box layout and identify which fuse(s) are responsible for the power windows. This is the most accurate way to find the correct fuse(s).
Without the manual, I can't give you a precise location within the underhood fuse box. The fuse box itself is usually located near the battery.