PHP is an interpreted language. This means that each time a PHP generated page is requested, the server must read in the various files needed and "compile" them into something the machine can understand (opcode). A typical Drupal page requires more than a dozen of these bits of code be compiled.
Opcode cache mechanisms preserve this generated code in cache so that it need only be generated a single time to server hundreds or millions of subsequent requests.
Enabling opcode cache will reduce the time it takes to generate a page by up to 90%.
PHP is known for its blazing speed. Why would you want to speed up your PHP applications even more? Well, first and foremost is the coolness factor. Next, you'll increase the capacity of your current server(s) many times over, thereby postponing the inevitable need to add new hardware as your site's popularity explodes. Lastly, high bandwidth, low latency visitors to your site who are currently seeing page load times in the 1-2 second range will be shocked to find your vamped up site serving up pages almost instantaneously. After enabling opcode cache on my own server, I saw page loads drop from about 1.5 seconds to as low as 300ms. Now that's good fun the whole family can enjoy.