The quartet - who broke the 27-year-old record set by the former East Germany of 41.37sec - beat home arch-rivals Jamaica (41.41sec) while Ukraine took bronze (42.04sec).
It was the United States' first Olympic gold in the event since the 1996 Atlanta Games.