Where to put the connection string

Machine.config is used for machine specific items - thus the name. It never get moved from machine to machine.  So that is where you put your connection strings.  Doing this you will never have the horror of realizing your QA team has been hammering against your production data because someone forgot to account for the web.config when it was moved DOWN from prod to back out a bad release to QA (or whatever).  And all your web.configs will be the same and easy to promote when they do change.

You can also add a value to the machine.config so that you can tell which environment you are in at runtime.

So in my machine.config I added:
  <appSettings>
    <add key="ADEDevelopmentEnvironment" value="Yes"/>
  </appSettings>

Then in my code I have:
If Not String.IsNullOrEmpty(ConfigurationManager.AppSettings("ADEDevelopmentEnvironment")) Then
            Me.Label1.Text = "You are STILL in Dev?!!!"
Else
            Me.Label1.Text = "You are on production"
End If