Chances are, your first couple of questions are:
Using the links above you can find more detailed information on those questions.
There are several bits of information that can be changed on the Site Information page: Configuration » System » Site Information
Included in that are:
The main pieces on this page you're going to use are the Site Name, E-mail address, and Default front page.
For those who have used older versions of Drupal, you'll notice that Mission and Footer are no longer on this page. You'll control those via blocks now instead of doing it on the site configuration page.
This is what shows up in the left hand corner of your web browser. It is the text to the right of the |. The text to the left of the | is the name of the page you're on.
Page Title | Site Name
So if you're on the Site Information page at the Drupal 101 site, the text in the left-hand corner will say
Site Information | Drupal 101
You can control what this says by changing the Site Name and the title of each page, event, etc.
Whenever an e-mail goes out from the site, such as that with log-in information for a site user, the e-mail address attached to that message can be set on this page.
Be sure it's an e-mail address that actually exists, because you may occasionally get responses or e-mails to that address.
Depending on the settings for your theme, this may show up somewhere in the banner area of your site, often times below or very near to the logo and site name.
More information on the front page options can be found here.