DNS (domain name system) does not allow multiple A records in a fail over fashion. There are also no priorities with A records. If you create multiple A records, the dns will resolve your domain in a round-robin fashion to all IPs. What this means is that if you have say 2 A records for your domain, and host your domain at two locations, some clients will see site at one location and others on the second one, at the same time, depending on what A record (resolution) they received.
To setup a fail over requires much more than that. It involves clustering (expensive and reliable) OR automatic DNS update through some script (cheaper solution and slow).
Example of clustering is that you have one device (e.g. a server) that handles the routing of traffic to which ever server is online. If one goes off, it starts redirecting the traffic to other. http://www.linux-ha.org is one such project.
Second solution can be that if main server goes off, the DNS hosted at 3rd location updates the A record (which should have a low refresh time e.g. a few minutes). Depending on how fast the dns propagates, the traffic will start flowing to backup/second location. This solution requires custom scripting or 3rd party dns provider.