At the interview
If you're using an omnidirectional mic, simply place the mic between you and the interviewee. If, however, you're using a mic that faces just one direction (like the Samson Meteor Mic), you'll need to ensure that you've turned the mic enough that it picks up both of your voices. For this reason, it is often easiest to sit at a 90 degree angle from your interviewee, rather than directly across from them with the mic in-between.
Try to keep the mic away from sources of white noise (vents, florescent lights, your computer's fan), but do not fuss about this too much: a consistent stream of noise can usually be removed in Audacity without sacrificing voice quality (see next page). It is much more important to avoid erratic bursts of noise (clock chimes, airplanes, other voices) which cannot be removed easily.
Ensure that you've switched your computer's "input" to your external microphone. This can be done in your computer's "settings" menu, or (usually) in the settings of your recording software.
Run a sound test before you begin
Ask the interviewee to state their name and any relevant information about their job as they'd like it reported on the podcast. Simultaneously, use this time to make sure that the mic is functioning properly.
Before you begin the interview, pause 4-5 seconds. This is important. You will need a few seconds of recorded "room noise" in order to remove it from the whole recording in post-production.
Begin by welcoming your interviewee to the podcast
During the recording
While your interviewee is answering questions, try to avoid making too many affirmative noises like "hmm" and "yeah". This feels natural, but it can sound distracting to listeners.
Do not type or click on your computer, as the mic will pick this up.
If you or your interviewee make a mistake, try to back up to a natural starting point so that you can delete the mistake in post-production. Although it's a normal thing to do in everyday speech, try not to correct yourself by quickly jumping back to your error (as in, "Well, thanks you... I mean THANK you for stopping by"). This makes it almost impossible to make a natural edit later.
Finish by thanking your guest
Save your audio file immediately after you finish.
It's also important to back-up your interview in the cloud, via a service like Dropbox, iCloud, or Google Drive.
The file format does not matter, as Audacity can handle almost any (including FLAC, WAV, MP3, MP2, AIFF, and PCM). You may also record in WMA, MP4, MOV, or M4A, all of which can be converted using an Audacity FFmpeg library add-on. (Of course, you can use Audacity as your recording software, thereby eliminating any question regarding compatibility.)