Even if you did come to an agreement not exchange information, if you are determined to be at fault for the accident, you could still face penalties such as fines, imprisonment, suspension or revocation of your driver's license, and civil lawsuits for damages.
For these reasons, it is always best to cooperate with law enforcement and exchange information with the other driver after an accident, regardless of who is at fault.