When you see binary and source code offered, they're talking about the form the program comes in.
The computer does its magic by following instructions in binary codes. These codes are specific to each kind of CPU. When you see a "binary" offered for download, they are talking about a program in a form ready to be digested by a computer.
Programmers used to enter these codes directly into the computer, but it was very cumbersome. They wrote programs to translate short words into the binary notation needed by the computer. These short words were called 'assembly language,' and the translating program was called an 'assembler'.
Eventually, programmers designed symbolic languages which could be given to a program that would do the translation to the computer. Initially, these programs (called compilers) were very inefficient, but over the course of time began to meet and even exceed the efficiency of hand-writing the assembly programs. Since it is so much easier for programmers to use these higher levels of abstraction, these days almost no programming is done in assembler (except for the construction of compilers).
A program that is still in the form of one of these symbolic languages is known as the 'source code', while the binary representation is known as the 'object code'. If you have the source code of a program, it is much easier to both read and alter than in its binary form.
So to convert source code to a binary file that will make the computer do things, you must compile it to 'object code'. Then, usually there is a linking phase where each bit of object code that goes to make the program is put together. Installation is usually just a matter of putting things in the right place so that the computer system knows where it is and what to do with it.
Hope this helps.
-