To increase the security of your secret key you may decide to store your key pair in a different location than the default directory chosen by GnuPG, which for Windows is C:\Documents and Settings\your_username\Application Data\GnuPG in the local machine.
The easier solution is to keep the GnuPG files in an external USB drive, or an encrypted volume in the local hard disk. A more complex solution involves the use of a smart card.