To troubleshoot your power windows, you'll need to consult your owner's manual. It will show you the location of the fuse box (usually under the hood and/or inside the car, often near the steering column) and list the fuses relevant to the power windows and related circuits (like the power window switch). There's likely not a single "Power Windows" fuse. You might find individual fuses related to the window motor circuits themselves or to the general power supply to the system.
A faulty fuse is less likely the problem than a failed window switch, wiring issue, or a problem with the BCM itself. If checking the fuses doesn't solve it, you'll probably need to consult a repair manual or a qualified mechanic.