You can edit the content of the Alert Button by going to Content > Sections > Alert Button.

The Description field controls the text listed on the button itself, and there is a field where you can put the name of the icon you want to use. A full list of your site's icons can be found here: http://ekklesia360.screenstepslive.com/s/11328/m/78713/l/905809-icon-font

The rest of the content can be controlled via the section content area.

To turn off the alert button, you can change the name of the section to "Alert Button - Off". To turn the alert back on, it just needs to be called "Alert Button"