2, United States Patent 6a a cy my wo a @ 6s) (2) (58) Guzman et al. MANAGING STATE FOR UPDATES TO LOAD BALANCERS OF AN AUTO SCALING Group Applicant: Amazon Technologies, Ine, Seattle, WA (US) Inventors: Mareel Robert Guzman, Buraby (CA); Norman Jordan, Vancouver (CAy: Shawn Jones, Vancouver (CA): Ahmed Usman Khalid, Post Coquitlam ca Assignee: Amazon Technologies Ine, Seattle, WAS) Notice: Subject to any disclaimer, the term ofthis pateat is extended of adjusted under 35 USC. 154(b) by 0 days. Appl. Nos 14/701,522 Filed: Apr 30, 2015 Prior Publication Data US 201610323188 A1 Nov. 3.2016 In C1, HOw, 127803 (201301) Hos. 2908 (2006.01) Hos. 1224 (2006.01) coc Hos. 47/125 (2013.01), HOAL 41/0816 (2013.01); odd 41/0893 (2013.01); Hoa. 67/101 (201301) Field of Classification Search OSL 47/125; HOSL 12/5691; GO6F 15/15: Gi6E 9748558; GOSF 11/2028 708/224, 226; 370/410 US0100. 40B2 1386: (10) Patent No.: (4s) Date of Patent: US 10,038,640 B2 Jul. 31, 2018 66) References Cited US. PATENT DOCUMENTS. 4403286 A$ 911989 Fry canst 9505 “wis SaS7247 A 7998 Norin ta 6720281 BL 42004 Okano eta 7430464 B2* 92008 Base oat 29/12000 sm2i7 7SM4SO8 BLY 82009 Kommula vo HOAL23/12066 708 7881208 BL* 22011 Nosella OAL. 291 (Continved) (OTHER PUBLICATIONS US. Appl No. 14701,515, ad Ape 30, 2016. (Continved) Primary Esaminer —Tesfay Yohannes (14) Attorney, Agent, or Firm — Davis Wright Tremaine LLP on ABSTRACT A computing resource service provider may provide com- puting instances onganized into logical groups, such as flo-scaling groups, Computing instances assigned t0 an futo-scaling group may be associated with one or more load balaneers configived to ditect walfic 0 the computing customers of the computing resource service provider may add or remove load balancer from the auto-sealing groups, The computing resource ser- vive provider may track a state of the load balancer in & atabase in order to enable efficient addition and removal of Toad balancers to auto-scale groups. 18 Claims, 10 Drawing SheetsUS 10,038,640 B2 Page 2 66) 3302401 412810 Bous202 8756298 Pecan] 5705300 2on3bo40402 202016989 aoosonsaess 20090133600 0080148850 aonnorT9973 sov.0036003 aorn0nr69s6 doroo107113 dorvogTe3os dorwo126010 20110276698 dorvorteos sorDio0esr 71 aorvorsaias doLyoo31082 aowy01nsst do1son97802 sn40067914| dovaousstt References Cited USS. PATENT DOCUMENTS. BI BI BI Be BI 32013 4209 H2013 ‘e204 32016 Tao 43002 2003 ‘72004 72006 007 2010 22010 4010 01 S20 sv201 120ut "y2012 02012 1an3 52013 12083 ‘32014 3204 aly Hoa 67/2833 sons “Tompkins Hayden Ma 1041 67/1008 09/203 Bhan a Sanka et Eevyebegnol et a Cane oa Bare W041 67/1002 sao Chaufour ot Nevetal Brodie a ‘Sh et Nitkin Innes eta. tistal Riva ea Maldaner Nakayama etal Patt ea. Ue Labi Salton ea. tab ea Nish Metiath ta 20140089500 AL 92014. Sankar ta soia1012%6 Al 42014 Rhandkar eta soo AL 2014 Muller et D1eot6s AL 62014 Inline a dorgoso4ai4 AL 102014 Yongaasa ta aoisbowooe AL ‘32018. Sickle ea doieo26sia AL 1/2016 Cucina a aorunsosee AL_ 22016. Schoenen etl SniaoDDIOND AL* 32016 Sanlake st v6 "700/295 20160127456 AL $2016. Rashad ot al aO1OUESSO AL 92016 Cox eta SOMGUILST AL 112016 Gasman ota 2160323188 AL 112016 Gueman & a aniens07 2016 Guzman eal. (OTHER PUBLICATIONS US. Appl No, 14701518, fled Ape 30, 2015 Ineraional Search Repott an Weiden Opinion dat Jun. 23 2016, Ineznational Patent Application No, PCTUS2016.030279, Sled ap. 29,2016. Sato, “implementing Blve-Green Deployments with AWS", pubs ihe hy Danilo Sto on Avg. 27, 2013, at ipswwe thoughtworks.comnsighte og inplemeating bergen deploy sment-aw, 13 page, Swine, “EC? Instance Belonging to Mulijle ELBs,” published by Shlomo Swidler on Jal. 23, 2009, a tp: shone com 7009107 e2-nsancebeloaging-t0-multipesbs html, 6 pages * cited by examinerUS 10,038,640 B2 Sheet 1 of 10 Sul. 31, 2018 U.S. Patent WZ doi ajeog-ojny aS | | jl en jh kee I | m3 : || \ | | a man uBissy I een ho ke | | KK epirorg 8010108 FOF eosnosey Sunndwod b Old (spiewoysng, 2US 10,038,640 B2 Sheet 2 of 10 Sul. 31, 2018 U.S. Patent We aseqeieg seouzjeg peoy 1S0H 1e91SAYg 4 seKe7 vor: id Bee ssid |. ¥ | punosbyoeg " eoineg Supueleg pet a oe somes 3 wR aeog-o1ny we dnosp ejeog-oiny JOPIADIg BOIS We sounosey BugndwiogUS 10,038,640 B2 Sheet 3 of 10 Sul. 31, 2018 U.S. Patent | TE (shoourisut “ €"Old | Fre ebouey eouesay | | | ze erawo somes OEE aomes, Gupuereg p27 (| FEE Somes sone | Y |_| Cb eBenerr sone) = —— TRE (eueousea pe | | femmes coueuewew! | houeoeig | |_ Be anteco ones OEE soinseg ZTE somes | ze dnosg ajeog-ony tre weg Teens WE (sjeouesy | * (mmm) Pie sbouey Sousa | By seindwwog souwog, DIE aseqeieg soouejeg peor, | BFE e1eq -souejen peor [FOE sepinoig eames eounosey Bunndwiog BEE isonboy 00€ Weisser | Tre sbeuey sous | ZF seindwoo senagUS 10,038,640 B2 Sheet 4 of 10 Sul. 31, 2018 U.S. Patent v ‘Sid 49indwog 180g eer KK 90r GS Tole UORETIEMA aFeoneg |! /— | SS (s)eworsng | eouersuy eoueisuy Q couessu gq weFeones = A] | _oeveames Burouejeg peo7 1 ‘8R0g-o1Ny | | | | | | | ‘sour ISU | 1 | | | | | | | FOF sepiaorg cones oorU.S. Patent Jul. 31, 2018 Sheet 5 of 10 US 10,038,640 B2 Receive Request to Initiale Asynchronous Work Flow Execute Background Process Determine Load Balancers in Transient State Determine Computer System Instances Associated with Load Balancers in Transient State 502 508 Determine Order of Operations and Batch Size for Computer System Instances Associated with Load Balancers in Transient State 510 Execute Operations Associated with Yes Determined Batches 512 Generate Audit Log Information an ‘Kaditional Request Received?, 516 No Update Database with Load Balancer(s) Information 518 FIG. 5U.S. Patent Jul. 31, 2018 Sheet 6 of 10 US 10,038,640 B2 600 Receive Desoribel oadBalancers Application Programming Interface Call 602 Requestor Authenticated’ No- Retum Error Message 610 604 Yes Query Database for List of Load Balancers Return List of Load Balancers FIG. 6 606 608U.S. Patent Jul. 31, 2018 Sheet 7 of 10 US 10,038,640 B2 Receive AddLoadBalancers Application Programming Interface Call 702 Requesio? Authenticated! 712 Yes Update Database with Load Balancer(s) G ite Audit Log Informatic FIG. 7U.S. Patent Jul. 31, 2018 Sheet 8 of 10 US 10,038,640 B2 Receive RemoveLoadBalancers Application Programming Interface Call 802 Requestor Authenticated! No i Authorized?, 804 Yes Update Database with Load Balancer(s) Information Queue Asynchronous Work Flow Generate Audit Log Information FIG. 8 806 808 810U.S. Patent Jul. 31, 2018 Sheet 9 of 10 US 10,038,640 B2 Load Balancing Service 935, Update Load Balancers 202 {Launch Background Proces: ee L Loop j}-—Describe Load Balancers—m}} 926 | 906 {List of Loas Balancers Foon -——Deserive instances—P4} 908 8 {fester nea I Determine Batch Operations 12 denned [top ~~ ——AssienDessstning atanees———e} ore | 816 {enone ——— | een eee ete Describe Instance Health} 918 t | | 1 | | | 920 { |<——Acknowiedgrent Load Balancer Update Complete aA [Coretta Boner Upte————} 022 ___Update Background Process Complete ’ ’ yFIG.9U.S. Patent Jul. 31, 2018 Sheet 10 of 10 US 10,038,640 B2 ooo Application Server Production ' ' ' ' User ' information | | ! ' 4 a 19010 1012 1014 1016US 10,038,640 B2 1 MANAGING STATE FOR UPDATES TO LOAD BALANCERS OF AN AUTO SCALING GROUP. CROSS REFERENCE TO RELATED 'APPLICATIONS, ‘This application incorporates by reference forall purposes the fll dislosure of co-pending U.S. patent application Ser. No. 147701,513, filed concurrently herewith, entitled “MANAGING LOAD BALANCERS ASSOCIATED WITH AUTO-SCALING GROUPS" and co-pending US. patent application Ser. No, 14/701.518, filed concurrently herewith, eatilled “BACKGROUND PROCESSES IN UPDATE LOAD BALANCERS OF AN AUTO SCALING GROUP" BACKGROUND ‘The use of hosted computing service and storage has proliferated in recent years. The resources for network ‘computing and sionige are ollen provided by computing resotiree providers who leverage large-sale networks of ‘computers, servers and storage drives to enable clients, > including content providers, customers and the like, t0 host and execute a variety of applications and web services. The usage of network computing allows content providers and ‘customers, among others, to efficiently and adaptively sat- isfy their computing needs. However, with the growing use ‘of vital resources, customers are encountering situations in ‘which the virtual resources cannot accommodate their cede during eeriin situations, such as unanticipated trafic spikes ‘or need for immediate responses to stisty increased loads, In response to this, web resource service providers are ‘introducing automated scaling. In many cases, customers transmit requests fo ereate instances such as, for example, Viral aschine instances to execute on hardware devices The instance can be automatically sealed enabling the ser- view provider to accommodate customer news diring the situations deseribed above. Once a customer has set up ‘automated sealing activities it may be dfiult for eustomers to modify instances andr load balancers easily. BRIEF DESCRIPTION OF THE DRAWINGS Various embodiments in accordance with the present disclosure will be described with rference to the drawings, in whiebs FIG. Lillosteates an environment in which customers may provide requests toa auto-scaling service to interact with Instanees included in an auto-scaling group and load. bal- ‘ancers associated with the auto-seaing group in aecondance ‘with an embodiment PIG, 2 illistates an environment for instantating instances and provisioning the instances to @ set of imple- mentation resources for an auto-scaling group and assoc ting the instances to load balancer in accordance with an ‘embodiment, FIG. 3 illstates an environment in which instances for ‘an auto-scaling group may be associated with the @ Toad balancer operated by a load balancing service in accordance ‘with an embodiment FIG. 4 illustrates an environment in which an instance service and auto-scaling service may manage load balancers {or an aulo-scaling group in accordance with an embodi- ments o 2 FIG, Sis a illustrative process for assigning instances of ‘an auto-scaling group ta load balancer using a background process in acconlance with an embodiment; FIG. 6s anillustrative process for retuming. Toad balancer information to a customer ofthe computing resource service provider based at least in part on an application program interlace callin accordance with an embodiment FIG. Tis anilusteative process for adding a load balancer to an aulo-scaling group based at least in part on aa application program interface call in accordance with an cembodiment FIG. # is an illustrative process for removing a load balancer to an auto-sealing group based atleast in part on an application program interlace esll in accordance with an embodiment FIG. 9 is an ilostrative messaging diagram for updating instances and load balancers of an auto-scaling group using a background process in accordance with an embodiment; and FIG, 10 illustrates an environment in which various embodiments can be implemented. DETAILED DESCRIPTION In the following description, various embodiments will be Lescribed. For purposes of explanation, specific configur- ‘ions and details are set frth in order provide a thorough ‘understanding ofthe embodiments, However it will also be Apparent to one skied in the art thatthe embodiments may be practiced without te specific details. Furthermore, well known features may he omitted or simplified inorder not to ‘obscure the embodiment being deseribed “Techniques described and suggested hercin relate to cus tomer management of Virtual compute instances oF simply instances assigned fo an autoscaling group and load bale ‘ancers associated withthe auto-scaing group. A customer of ‘computing resource service provider that hosts computer systems managed hy the customers may, programmatically through one or more application programming interface calls, ereate an auto-sealing group to meet resource needs ‘during certain situations, sich as unanticipated trai spikes ‘or need for immediate responses to satisfy inreased loads The auto-scaling group may manage the resources for one or ‘more computer instances (eg, virtual computer system instances) created by the customer. The load balancers may be offered to customers of @ computing resource service provider to facilitate request processing by the one or more Jnstanees athe customer's auto-saling group, For example, the customer may opente a website using computing resources ofthe auto-scaling group, and the load balancers may be configured to direct tralie among the computing resourees of the auto-sealing group, such asthe one or more computer instances. Additionally, the website may receive requests from multiple other customers over a network. The ‘computing resource service provider may then configure a Toad balancer to direct the requests to the auto-saling group ‘or particular instances of the auto-saling group executing the website in such a way that the load generated by processing the request is distributed among the computer Instances executing the website, ‘An auto-scaling service may automatically and dynam cally manage the auto-sealing group to manage computing resources oF any esourees that might e subject to demand ‘Tuctuation. For example, the auto-sealing service may use various factors to determine whether to automatically and dynamically adjust andor allocste resources in response to event, such as load on the group exceeding a thresholdUS 10,038,640 B2 3 Funhermore, the eustomer may also need to adjust andor allocate resources in response 1 an event andlor manage aa instance assigned to the auo-scaling group, The customer may interact with the avto-saling service by transmitting requests 1o the computing resource service provider. The rexjuest may include # command, application programming imerface (API cal, remote procedure eall (RPC) or other instruction configured t cane # service of the computing resources service provider to perform various Tunetions. The computing resource service provider may expose APL calls to the customer in order to enable the customer 10 ‘modify load bulancers associated with an auto-seale group ‘while the auto-scale group is in operation, These API eas, ‘may eliminate a customer need to temminate andor delete an auio-scale group or foad balancer in order to modify the ‘association between the load balancer and auto-scale group, For example, customer may submit an API request fo the sulo-scaling service to add a load balancer 10 the auto- scaling group. In various embodiments, adding a load bal- fancer to the auto-saling group causes the computing ‘nstanees included in the auto-scaling group to be assigned to the load balancer sues thatthe load balancer disiesbutes network traffic to the computing instances assigned to the Joad balancer. The auto-scaling service may include a data- base configured 10 maintain status information andor ether ‘information about the load balancers attached to the autor scale group. ‘The database may be configured to track state information forthe load balancers associated with the ato- scale group to avoid maintaining a one-to-one mapping of Toad balancers to instances “The state information may include a variety of different states, such as an adding state, an added state, an in-service state, and a removing state. The adding sate may indicate that background process is assigning all he instances inthe roup 10 the load Balancers, upon completion the back- round process updates the status to be added. The added State may indicate that all the instances inthe auto-sealing group have beet assigned to a load balancer previously in the adding state, The in-service state may indicate tht the Joad balancer is directing trae to at least one healthy instance of the anto-scaling group as described in greater detail below. The removing state may indicate that the instance ofthe auto-scaling group is curently being deas- signed o the particular load Balancer For any new instances ‘added (9 the aulo-scaling group, eer as @ result of cus ‘omer interaction or configuration of the auto-sealing group, the background process may detemine if a Toad balancer assigned fo the auto-scaling group isin the adding, added, oF inservice state IF the load balancer isin one of those sates then the background process may repister or assign the new instances to the load balancer. FIG. 1 shows an illustrative example of an environment 4100 in which load bolancers may be assigned to a parcular auio-scaling group 102 in order to process regiests 118 ‘directed at computer instances of the auto-scaling group ‘operated by a customer 106. A computing resource service provider 104 may provide computing resources to customers 106, The customers 106 may utilize physical hosts. 142, described in greater detail below, operated by the computing resource serve provider 104 to perform a variety of fune- tions, For example, the customers 106 may operate a website ‘of log server sing the computing resources of the compat ing resource service provider 104, Funbermore, customers 106 may ereate auto-sealing groups managed by the com puting resource service provider 104. The auto-scaling ‘groups 202, described in greater detail below, may comprise 4 set of computer instances operated by the customer 106, 0 o 4 The auto-scaling group may be configured to add or remove computer instances and other computing resources trom the auto-sealing group based at least in part on setting provided by the customers 106 and/or computing resource service provider 104 Tn some embodiments, the customers 106 may include ‘organizations which operate computing resources which may be accessible to other customers of the computing resource service provider or other entities in general. For ample, the customer may operate a website which is accessible 1 users over a network, such as the Internet. The {erm “organization,” unless otherwise clear from context. is intended to be read in the broad sense to imply a set of principals organized in some manner. The customers 106 ‘may use computing resourees of the computing resource service provider 104 in oer to operate a load balancer 120 And auto-scaling group 102 configured o isteibute requests 18 to computing resources, suc 3s computer instances. of | the customers 106. As ilustated in FIG. 1, the computing resources of the customers 106 may include and/or be ‘implemented by physical hosts 142 operated by the com- puting resource service provider 104. However, in various embodiments, the eompting resources utilized by the eos: ‘omers 196 include computing resourees of the customer oF other entities in accordance withthe present disclosure. For ‘example, the eustomer may operate a load balancer 120 using computing resources of the computing resouree ser- vieer provider 104 to diret tralie to ane oF more servers ‘operated by the customers 106 “The roquests 118 may'be received by the load balancer 120 or by one oF more other systems of the computing resource service provider 104, such as a request listener not ilkstated in FIG. 1 for simplicity, and rected to the load yy be a computer system or virtual computer system configured to distbute the request 118 fo one or more computer instances, sup: ported by the physical hosts 142, in onder to optimize resource utilization andlor avoid oveeloading a particular computer instance, Por example, the load balancer 120 may include physical hardware connected to a server rack oF otherwise included in datacenter. In another example, the Jad balancer 120 may include one of more virtual machines supported by a particular physical host 142 operated by the computing resource serviee provider 104 as deseribed in freater detail below Tn some embodiments, the aumber of requests 118 may be jareater than the number of requests that the load balancer 120 may process given the amount of computing resources currently allocated to the load balancer 120. As a result, the customer 106 may determine to add an additional load balancer 122 to the autoscaling group. The customer 106 ‘may transmit a request, such as an API call, configured t0 ‘cause the eomputing resource service provider 104 or com- ponent thereat (eg, an auto-scale service described in greater detail below) to associate the addtional load bal- fancer 122 with the auto-sle group 102. As described Doerein associating the load balancer 120 withthe auto-seale soup’ 102 includes assigning. the computer instance Included in the auto-seale proup 202 (othe load balancer 120 such that the load balancer directs requests and/or other network traffic 10 computer instances of the auto-seale ‘group, dseribed in greater detail below, The customers 106 may be able to update (e.g. ad or remove) the sot of load balancers associated withthe auto- scale group 102 regardless of whether the computing resource service provider 104 of component thereof is ‘currently updating the set of load Balancers associated withUS 10,038,640 B2 5 the auto-scale proup 102, For example, particular euston ‘may accidentally transmit request to add andor remove the load balancer 120 from the auto-seale group 102 and thea at some point in time Inter the customer may, while the ‘computing resource service provider 104 is processing the socidental add andlor remove request, transmit a second request to reverse the accidental request. As a result of the second request, the computing resource service provider 104 may terminate processing the accidental request and begin processing the second request. Aditionally, the computing resource service provider 104 may generate audit informs tion corresponding to operations performed in adding and removing load balaacers to auto-scale groups. The audit ‘information may enable the customers 106 to determine the result of previous add or remove operations an the progress or state of existing foad balancers, As deseribed above, the computing resource service pro= vider 104 or component thereof, such as the auto-scale service described in greater detail below, may execute @ background process configured to assign and de-assign ‘computer instances of the auto-scale group 102 wo the load balancer 120 and additonal load balancer 122, Furthermore, the background process may be configured such that the ‘customers 106 moxily a capacity ofthe auto-scale group 102, (ex, cause computer instances tobe launched or terminated within the auto-scale group 102) regardless of the current state of the background process. For example, the back- ground process may be assigning # st of eomputer instances ‘of the auto-scale group 102 10 the Toad balancer 120; the ‘customer may then case a new set of computer instances to be added to the auto-scale group 102 and, as a result, the background process may add the new set of computer instances tothe set of computer instances to be assigned (© the load balancer 120. “The background process may query load balancer ‘database, deseribed in grenter detail below in connection ‘with FIG, 2, maintained by the auto-seale service in onder to determine if any load balancers associated with the auto- scale group 102 aro in a transient state (et, adding stato, fdded state, andlor removing state). If there is atleast one load balancer in a teansieat state, the background process may determine a set of computer instances and correspond- ng operations to perform based at leat in part on the atleast ‘one load Balancers state, Por example, ifthe load balancer jin the adding state, the haekground process may determine a batch of computer instances to assign tothe load balancer. “The background process may assign a computer instance to ‘Toad balancer by transmitting an API call oa Toad balancer service indicating the load balancer 120 and the computer instance to be assigned tothe load balancer 120. In various ‘embodiments, the background process may transmit 3 request directly wo the load balancer 120 requesting that load balancer 120 manage trafic directed tothe “computer In some embodiments, adding the sditional foad bal- ‘ancer 122 may include instantiating one or more load balaneers and assigning the instantiate Joad balancers wo the ‘auto-scale group 102, Furthermore, assigning the alditonal Joad balancer 122 may include updating one or more other ‘computer systems, For example, the request Tistener described shove may be updated with information corre sponding tothe addtional load balancer 122 in onder to route requests. In another example, a Domain Name System (DNS) entry or other naming system service entry may be ‘dated to include a Internet Protocol (IP) address associ- ‘ated with the additonal resources 122 0 o The euston autos the group by submitting API calls to an interface, not shown in FIG. 1 for simplicity, of the computing resource service provider 104, For example, the customers 106 may assign ‘addtional computer instances to the auto-scale group 102 ‘through a management console or other application contig- ‘ured t enable the customer to manage computing resources provided by the computing resource service provider 104 The management console may be exposed tothe customers 106 as a webpage wherein the executable code of the ‘webpage is configured to generate API calls tothe comput- ing resource service provider 104. The customers 106 may Provide information corresponding to the aulo-scale eroup 102, the compte instance of the auto-scale group 102, and Jad balancers assigned t the auto-scale group 102 through the management console. For example the customer may provide te computing resource service provider 104 with an APLeall, the API ell may indicate a frst umber of load balaneers to add to the auto-scale group and a second number of load balancers to remove from the auto-seale ‘group 102. The information may include IP addresses ofthe ‘computer instances associated with the auto-seale group 102, identifiers ofthe anto-seale group, computing resources allocated ( the computer instances andor auto-sale proup 102, computational capacity and network capacity of the computer instances andor auto-seale group 102, routing information, identifiers ofthe load balancer, and any other information configured to enable the auto-scale service 10 fad andr remove load balancers from the auto-sale group. PIG, 2 illustrates an environment for instantiating instances 222 and provisioning the instances 222 to a set of implementation resources for an auto-sale group 202 and ‘associating the instances to a load balancer in accordance ‘with an embodiment, As illustrated in FIG. 2, « computing resource service provider 204 operates a multitude of phys cal hosts 242, The physical hosts 242 may be any device or ‘equipment configured to exceute instnictions for peronming data computation, manipulation, or storage tasks, such 28 3 ‘computer of a serve. The physical hosts 242 may include any Computer system oF virtual computer instance desribed above. A virtualization layer 244 operated by the computing resources service provider 204 enables the physical hosts 242 to be used to provide computational resources upon ‘whieh one or more instances 222 may operate. Desribed in greater detail below, the viwualization layer 244 may be any Gevice, software, or firmware used for providing a virwal ‘computing platform forthe instances 222. The virwal com- puting platform may include various virtual computer com poneals, such as one or more vetual CPUs, virtual meme fies, virual disks, and the like “The auloscale group 202 may contain one oF instances 222 and may provision new instances 224 based at least in parton a variety of atibutes, The varity of anrbutes may ‘Include a desired capacity as indicated by «customer, overall Toad on the auto-scale group 202, load on a particular instance of the auto-seale group 202, a request from a customer, a schedule, and any other atibute suitable for determining whether to add or remove instanees from the aulo-scale group 202, The instances 222 may be virwal ‘computer systems configured to provide computing resourees to the auto-cale group 202 and customers of the ‘computing resource service provider 204 by at least per {orming Various operations. In some embodiments, the auto- scale group 202 and instances operate asa single computing entity performing various operations for the custome.US 10,038,640 B2 7 A load balancing service 236, described in greater detail below in connection with FIG. 3, may provide load balanc- ‘ers and other trafic dsieibution services to the auto-seale ‘group 202, Forexample, a DNS entry of s load balancer may nude the TP address for various load instances 222 of the suto-scale group 202 and requests directed to the auto-scale sr0up 202 may be distributed tothe instance 222 assigned 10 the load balancer, During a scaling event requiring the allocation of computing resources tO the auto-scale group 202, a computer instance image may be used to instantiate ‘new instance 224, When dealocating computing resources to the aut-scale group 202 operation of one or more of the instances 222 may be terminated or suspended. An auto- scale service, described in greater detail below in connection ‘with FIG. 3, may determine whether instances 222 are to be ‘aked or removed from the auto-scale group 202. Te autorsale service 230 may also inchudea background process 232, as described above, and the background pro- ‘cess my assign or desssign instances 222 fo one or more Toad balancers. The background process 232 may include Jogie or other computer executable code that, when exceed by one oF more processors of the physical host 242, cases the physical host 242 to perform Varios operations core= sponding to adding and removing load halancers From the auto-scale group 202. In various embodiments, the auto scale service 230 or Joad balancing service 236, whe: ‘deassigning instances 222 from the one or more load bal- ‘ancers, will Wait until all network connections 10 the instances 222 to be deassigned are terminated or complete. ‘Once there are no more connection tothe instances 222 10 be deassigned, the load balancing scrvice may remove the ‘instances 222 Trom the set of instances to which the one oF ‘more load halancers direct traffic. This process may be refered to as connection draining, eg. waiting unt all ‘current connections to particular instances have terminated ‘hile preventing new connections tothe instance fom being ‘established, Funhermore, when allocating or dalloeating computing resources tothe att-scale group 202, a background process 232 of an atoseale service 230 may transmit information ‘corresponding tothe allocation or dealloeation of computing resources t0 the load balancing service 236 to enable the load balancing service to correctly disect tral to the auto-scale group 202. For example, ifthe new instance 224 Js added to the group, the background process may transmit ‘an API call including an identifier of the new instance 224 ‘configured to eause the lod balancing sevice to asiga the new instance 224 to one or more load halancers associated with the auto-Scale group 202. Furthermore, the background process may’ update a Joad balancer database 210 with Information corresponding to the instances 222, the new instance 224, and one or more load halancers assigned i the ‘auto-seale eroup. “The load balancer database 210 may contain a set of rows ‘and columns, where each row corresponds toa Toad balancer associated with the auio-seale group 202 and each column ‘contains information corresponding tothe auto-seale group 202, the instances 222, of other information suitable for ‘adding or removing load balancer from an auto-scale group 202, The load balancer database 210 may include informs tion suitable to tacking the state of a particular load bal- ancer. Additionally, the information included in the load balancer database 210 may be used by the background process 232 to determine operations to perform in order to ‘add or remove load balancer fom the ato-seale group 202, For example, the background process 232 may quety the datahase to determine one of more load balancers 0 2880 0 o 8 ciate with the new instance 224, Furthermore, a8 described herein, the load balancer database 210 may inclade any database or any other data store configured to store infor ‘mation corresponding to the one or more load balancer associated with the avto-seale group 202 including state {information associated with the one or more load balancers. The load balancer database 210 may’ be any data store andlor storage device described herein. Furthermore data regarding the operation of the auto-scale group 202, the instances 222, the background process 232, and the new instance 224 may be collected and/or monitored by the faulo-scale service 230 and stored in the load balancer database 210 oF other dat store inorder to generate adit log information and mets information, Data regarding oper- tion of the auto-seale group 202, the instances 222, the background process 232, and the new instance 224 includes any information regarding the actions perfomed by the aulo-scale group 202, the instances 222, the background process 232, and the new instance 224, such as the metrics data (eg. memory usage of the vital host, fle systems, information regarding vital CPUs, or any other informa. tion regarding the auto-scale group 220 or component thereol) ar actions preformed. The data may. in some embodiments, be converted into a format suitable for use ‘with the audit log or audit service. In various embodiments, the data is stored persistently using an on-demand storage service of the computing resource service provider 204 in suela way that he customer may use the data to determine the operation ofthe background process 232 and the state Jd halanoers associated tothe ato-scale group 202 In some embodiments, when a particular customer sub- its an API call configured to update (e-.. add or remove) ‘one of more load balancers associated with the Auto-scale saroup 202, the anto-scale service 280 may perfont several ‘opertions ia onder to process the API call. First the auto- scale service 230 may update a recordin the Toad balancer database associated with the one of more foad balancers indicated in the API call forthe auto-seale group 202. This may ensure that any future instances, such as the new instance 224, launched in the autoscale group 202 will be assigned to theone or more load halancers. The update to the load balancer database 210 may occur synchronously with receiving the request, The auto-sale service 203 may thea queue an asynchronous process (ez, the background pro- cess 282) that may determine a set of existing instances 222 And update the load balancers associated with the set of existing instances 222 to the one or more Joa Balancers indicated by the customer in the API cal, The background process 282 may he non-deterministic such thatthe same ‘input (eg, adding or removing the same set of fo balane- crs) may not cause the background process 232 to operate ia the same manner. For example, the order of operations may be different, dhe batch size of instances 222 1 assign to the Jad balancer may be different, and any other operation of the backpround process described herein may be different "The background process 232 may also ereate a record in 4 particular customer's sealing. history (which canbe rettioved using. the DeseribeScalingActivities API). The record andlor API may be used to track the process of ‘updates othe one or more load balancers associated with the auto-scale group 202 and for audit purposes as needed by the ccustomer. The record and any corresponding stadt informa- ‘ion may include a human-readable representation of the ‘portions performed by the auto-scale service 230, for ‘example, “updating load balancers for auto-scale group A in response to eustomer-II0S81's request to: add (loadbal- ancerl, loadbalancer2, loadbalances3).” The record. mayUS 10,038,640 B2 9 ‘alka include start time, an end time, and completion stats ‘once the auto-scale service has completed processing the request. As described above, the operations of the back- ‘ground. process 232 will not lock or otherwise prevent Updates to the ato-seale group 202, nor may the background process 232 prevent other updates to the one or more load halancers astociated with the auto-scale group 202 from being processed concurrenlly. For example, if particula ‘customer updates the one or more load balancers associat ‘with the auto-scale group 202 while another update is being processed, the auto-seale service 230 may terminate the previous update, restart the process, and generate another reco in the customer's auto-seale history. "The environment such as that lusiated in PIG. 1 may be usefl fora provider such a a computing resource provider, ‘wherein the computing resource system responds to requests from customers to manage load balancers assigned to to- scale groups. As discussed above, the computing resource system provides a mechanism to allow customers to add of remove load balanvers from a sel of instances associated ‘with an auto-seale group. The environment in stich a case ‘may include addlional components andlor other arange- ments, such as those illistrated in the networked environ- ‘ment 300 of FIG. 3. In this example, the networked env ronment 300 includes a computing fesource service provider 304 in dats communication with a cleat device 306 and ‘server computers 342 over a network 306. In one embodi- ment, the server computers 342 may’ be oe oF more com- puter hardware devices that are used to implement instances 520, For example, the server computers 342 may’ include hardware for implementing types of computing resoaces, such a8 storage devices, virtualized storage devices, net ‘working devices, and the like as described above in con nection with FIG, 2. Additionally, the implemented comput- ing resources may be programmatically and remotely managed by a customer of the distibuted computing resouree provider. “The server computers 42 inelndo a plurality of computer system devices that are each capable of executing one oF ‘more instances 320 created by the distributed computing resource service provider M04, In one embodiment, cach of the server computers 342 includes a processor, a data store, an inpuloutput bus, and/or any other composent known ia the art for executing. instances 320. Additionally, the instnees 320 may be vital machine instances. As Known Jn the art, a virual machine instance is an instance of & software implementation on a machine (Le. a computer) that executes programs like a physical machine, For ‘example, each of the server computers 342 may be config ‘ured fo execute an instance manager 318 capable of imple- menting the instances 320. For example, the instance man- ayer 318 may bea hypervisor, viralization layer, or another Iype of progrum configured to enable the execution of rukiple instances 320 on a single server computer 342. As «discussed above, each of the instances 320 may be config tured to exceute all ora portion ofan application, Addition- ally, the network 306 may be similar to the network as ‘described above. The networked environment 300 may span, ‘one or more data centers, where each data center may be peographically dstinet from each other. Additionally, the networked environment 300 shown in FIG, 3 may be one of several embodiments employed by the computing resource serve provider Tn one embodiment, the computing resource service pro- vider 304 includes a load balancer database 310, an instance service 312, a placement service 326, an auto-scale service 330, maintenance service 382, a metrics service 334, load 10 balancing service 336, andlor other componeats, The load balancer database 310 may include load balancer data 342. For example, as described above, the load balancer database 30 includes one or more records of load balancers 340 associated with the auto-seale group 302. Fach one of the records ofthe load balancer dats 346 corresponds to a load balancer 340 ofthe networked environment 300. The instance service 312 instantiates instances 320 based at least in part on a set of preferences provided by the customer. In one embodiment, the instance service 312 receives, from the customer on the client device 306, 2 request 336 o ereale one oF more instances 332 and eption- ally assign the created instances $20 t0 an suto-cale group 302. Adtionally, the request 336 received from the cus- tomer on the client device 306 may sso indicate atime to start execution ofthe requested instances 320, In response t0 receiving the oques, the instance service 312 instantates instances 320, In various embodiments, the auto-scale ser viee 330 receives the request and trasmits @ command t0 the instance service 312 wo instantiate the instances 320 such that the instances are associated with tho auto-scale group, or example, by associating auto-seale group 302 metadata ‘with the instances 320. In one embodiment, the instance service 312 may place instances in standby or detach instances from the auto-scale group in response to a request Irom theclient device 306 and or to-scale service 380, For example, the snto-scale service 330 may transmit a request to the instance service 312 to remove the auto-scale group 302 metadata associated with the instances 320 being ‘etached from the auto-seale group 302 according 10 the request 336. Furthermore, the autoscale service 330 may eassign a detached instance from the load balancer 340, ‘The customer may interact with the computing resource service provider 304 (via appropriately configured and authenticated API calls) to provision, operte, and manage instances 320 associated with the auo-seale group 302 that js instantiated on server computers 342 and operated by the ‘computing resource service provider 304, Additionally, the ‘customer may create one or more auto-sale groups 302, and the auto-scale groups 302 may be a logical collection of instances” $20. Furthermore, the instances 320 may be assigned f0 the auosscale group 302 or may be members of the aulo-seale group 302. The auto-wale service 330 may allow customers to interact with and manage Various auto- scale groups 302, For example, the customer may, dough the auto-scale service 330, set a maximm oF minimnm capacity for an auto-scale group 302. The auto-seale proup 302 may then manage the instances 320 assigned to the futo-scale group in order (© maintain the setings provided by the customer: In various embodiments, the customer may create and manage auto-scale groups 302 tarough a mane agentent console, as described above, provided by the com puting resource service provider 304 "The management console may’ Be exposed tothe eustom- cent as a Webpage; by interacting with the webpage (€2 through « bowser application) the customer may cause APL calls to he generated. The generated API calls may cause the computing resource service provider 304 oF component thereof 10 perform various operations indicated by the eestomer. Onee the customer has ereated the avto-scale aroup 302, the customer may assign one or more load balaicers tothe auto-scale group 302 by submitting requests 336. The requests 336, in his ease, may be processod By the ‘auto-scale service 330 or other component ofthe computing resource service provider 304. The instances 320 of the auto-scale group 302 may be used for various purposes, such as to operate a servers supporting @ website, to operateUS 10,038,640 B2 un business applications, or generally, to serve as computing power forthe customer. Other applications for the instances 320 may be to support database applications, electronic ‘commerce applications, business applications andlor other applications. Additionally, load balancers 340 may distribute trafic 1 various instances 320 of the auto-scale group 302 1w enable operation ofthe instances forthe various purposes described above and prevent tbe instances 320 of the auto scale group 302 from being overloaded. Although the instance service 312i shown in FIG. 3, any other computer system or computer system service may be utilized by the ‘computing resource service provider 302, such as a com- puler system or computer system service that does not ‘employ virualizaton or instantiation and instead provisions ‘computing resources on dedicated of shared. computers servers andor other physical devices. ‘The placement service 326 provisions the instances 32060 ‘one or more of the server computers 342, In one embodi ment, the placement service 326 determines the server 2 ‘computers 342 to provision the new instances 320 based at least in parton the indicated auto-seale group 302 ofthe new ‘instances 320, For example, the placement service 326 may Identify one or more server computers 342 with the appro- priate capacity’ to execute the instances 420. To tis end, the Phicement service 326 determines the capacity” of ec! server computer 342 from the resource data 310 storedin the ‘data store and aecontingly provision the instances 230, as ‘vill be described, The auto-scale service 330 automatically scales the capacity of a collection of previously requested instances 320 up or down based at least in part on circum ‘ances defined hy the customer that requested the instances 320. For example, the auto-eale sevice 330 may decrease themimber of instances 320 allocated othe eustomer during demand lulls and increase the number of instances 320 ‘allocated to the customer during demand peaks. Tn one embodiment, the auto-scale service 330 sheds a subset of the requested instances 320 during peri of low usage andor idle time. For example, the auto-scale service 330° may determine that the amount of instances 320 rexuested by the customer i redundant andor excessive ln response, the autoscale service 330 may tenminate a certain ‘numberof instances 320 allocates to the customer such that the remaining number of instances 320 allocated to the ‘eusiomer is aot redundint andlor excessive, In another ‘embosdiment, the sto-seale service 830 may shed the subset ‘of the requested instances 330 iF the usage rate does not ‘excood a predetermined threshold, Similaely, the auto-eale service 330 increases the amount of instances 320 during @ period of high usage. In one embodiment, the auio-scale service 330 may increase the amount of instances 320 ifthe Usage rate excoeds a predetermined threshold “The request 326, eccived fom the cient device 306, may inchide API calls to update or modify load balancers 340 associated with the auto-scale group 302 as described above. ‘The API ealls included inthe request may be received at an Interface of the computing resource service provider 304 Additionally the API calls andor requests may be authen= ticated by the computing resource service provider 304, The linerface may direct the API calls to the appropriate service for processing the roques, For example, at API eall o add 1 load balancer tothe ato-seale group 302 may be directed to the auto-scale service 330 capable of processing. the roquest 326. The auto-seale service 330 may then synehro- ously with receiving the request 326, store information ‘corresponding to the load balancers 340 included in the request 326 to the load balancer database 319. 0 o 12 The background process, described in greater det above in connection with FIG. 2, may periodically or aperiodically query the load balancer database 310 10 determine if any oad belancers 340 are in a transient state and ercate a wworkilow that exocutes in the background configured 10 ‘update at least a portion of the instances 322 of the auto- scale group 302 by at least acing andor removing the kad bblancers 340 tothe instances 322. In various embodiments the auto-scale service 330 may be configured such that only ‘one background process may be executed per auto-scale soup 302. For example, if the customer, via the elient evice 306, attempts to update the load balancers associated with the auto-seale group while another update is process- ing. then the existing background process may be signaled thatthe background process should recalculate the oper tions required to update the load balancers 340 and restart the process ‘The background process may generate dynamically-con- figurable batch sizes for both the load balancers 340 and instances 322 to perform operations on, For example, the background process may determine a fall set of load bal- ncers 340 associated with the auto-eale group 302, and the fill sct of instances 322 inchided in the auto-seaie group [302."The background process may then perform the oper. ‘ions required to update the Joad balancers 340 in batches (ex, adding and/or remove load balancers 340 to/from the ‘instances 322 in the auto-scale group 302). The background process my prioritize the batch based on a varity of tiflerent factors. For example, the background process may perform all add operations priot to performing any remove ‘pemtions. In another example, the background process may ‘complete the operations fora single load halaicer prior to performing an operation on any other load balancer ind- fated in the roguest 326, The background process. may ‘communicate with one of more other services in onder to ‘complete the update Toad balancer operation included in the request 326, For example, the background process. may generate an API request configured to cause the load bal- fncing service 336 to exceute the operation inclided in @ particular bateh (eg, adding andr removing. lod balane- fs 340 to/from the instances 322 in the auto-scale group 302), The maintenance service 332 schedules maintenance, software updates, andir firmvvare updates for the server ‘computers 342 In one embodiment, dae maintenance service 332 Schodules the maintenance and software updates at an ‘appropriate time based at least in pact on the available capacity of the server computers 342. For example, the ‘maintenance service 332 may sthedile the maintenance and software Updates at a time when the respective server computer 342 has a projected availability. In one embodi- ‘meat, the maintenance service 332 may patch and restart the server computers 342 when the muinlenance service 332 ddtermines thatthe server computer 342 is not hosting any instances 320. Additonaly, the maintenance service 332 ‘may patch virtual machines associated with the instance 342 if necessary prioe to instntating new images that are associated with the respective virual machines. For ‘example, the maintenance service 332 may schedule a patel fof the machine image based at least in part on the health status of the instances 320. In one embodiment, no add tional instances may be provisioned on the sever computer ‘342 untl the scheduled maintenance is competed "The maintenance service 332 may also periodically or periodically check the health tats of the instances 320, including instances assigned to the aut-scale group 302 andlor load balancers 340, The health check may includeUS 10,038,640 B2 13 determining the load, wilization, and operation of various ‘components of the instances 320 such as the central pro- ‘cessing Unit, memory, networking interice, operating SYs- tem, application, and other components of the instances 320. Invarious embodiments, when the maintenance service 332 determines that an instance 320 is unhealthy based atleast in parton the health check, the maintenance service 332 oF collier component of the service provider 304, such as the avio-scale service 230, may initiate a workllow to remove the unhealthy instances from the auto-scale group 302. Additionally i the maintenance service 332, detennines that ‘2 previously unhealthy instance 320 hus retumed to a healthy’ stats the maintenance service 332 oF other compo- rent of the service provider 304, such as the auto-scale service 230, may move the instances 320 into service oF attach the instances 320 tothe auto-scale group 302, Fur thermore, ian instance 322 assigned to a loa balancer 340 returns a healthy status the auto-scale group 302 may update the status ofthe foad balancer 340 wo in-service, as deseribed ‘above, In various embodiments, the health check may be performed by a second background process distinet from the background process described above in connection with FIG. 2 The metrics service 334 may be responsible for collecting resource data corresponding 10 the instances 320. The resource data obtained by the metrics service 334 may Indicate the uiiization of various components of the instances 320 sueh as the central processing unit, memory, networking interface, operating system, applications, and ‘other components of the instances 320. This information may be used fora variety of different purposes, for example, determining whether to allocate or deallocate resources 10 the aito-scale group 302. Additionally. the information may be used by the maintenance service 332 to determine the health ofan instance 320 andlor s server computer 342. The metrics service 334 may obtain and agaregate uilizton information for all of the instances 320 assigned to the ‘auto-scale group 302, Furthermore, when load balancers 340 fare adcled or removed fiom the auto-seale group 420, the metres service 334 may receive a command to add or emove the lod balancers 340 from the st of load balancers M0 for which the mettics service 382 collects andior fagaregates utilization information. ‘Aload balancer service 336 may be offered to customers ‘of a computing resource service provider 304 in onder to ‘aeiltate request processing by instances 320 of the cus- tomer. In various embodiments, the instances 320 may’ be assigned to the auto-scale group 302 and the load-balancer service 336 may distribute traffic to the instances 322 assigned to the autorscale group 302. For example, the customer may operate a website using instances’ 320 assigned w the auto-sale group 302 using the resources of ‘computing resource service provider 304, Additionally, the ‘website may receive requests from multiple ther evstomers ‘over the network 206. The computing resource service provider 304 may configure a load balancer of the load balancer service 336 to direct the requests to the instances 320 of the auvo-seale group 302 executing the website, in such a way that the load generated by processing the requests is distributed among the instances 320 of the auto-scale group 302 executing the website, The load bal- fncer service 336 may be a computer system or virtual ‘computer system configured to distribute the request to the instances 320 assigned to the lood balancer in order to ‘optimize resource utilization andlor avoid overloading 8 particular server computer 342. For example, the load bal- ‘ancer may include physical hardware connected to a server 0 o 14 rack of otherwise included ia data center, In another ‘example, the load balancer may include one or more virtual ‘machines supported by the server computer 342, FIG. 4ifkistates an instance service 412 in aecordance with at least one embodiment. The instance service 412, ‘which may be implemented by piysical hardware, is used by service provider 404 to provide computation resources for customers 406. The physical hardware may include «server ‘computer 442, The server computer 442 may be any device ‘or equipment configured to execute insiructions for perform- ing data computation, manipulation, or storage tasks, suel.as fa compiler or a server. A server computer 442 may be equipped with any needed processing capability ineluding fone or more processors, sueh as a central processing unit (CPU), a graphics processing unit (GPU) or a digital signal processor (DSP), memory, including static and dyna ‘memory, buses and input and output ports that are compliant ‘with any handshaking, communications, or data transfer protocol. The physical hardware may’ also include storage ‘devices, such as slomge disks and tapes, neworking eguip- ‘ment, and the like. ‘A virwalization layer 444 may include 9 are metal ypervisor or hosted hypervisor. The virwalization layer 444 exceuting on the service computer 442 enables the physical haniware to be used to provide computational resolves upon which one oF more computer instances 420 may operate, For example, the virwalization layer may ‘enable a viral machine 420 to access piysical hardware on the server computer 442 through viral device drivers on the viral machine 420, The virmalization layer 444 may include a hypervisor or virtualization software andior bard- \ware. The Virwalization layer may also include an instance fof an operiting system dedicated to administering the com puter instances 420 running on the server computer 442. Each virtualization layer 444 may include its own network= ing software stack, repousible for communication with other virtualization layers 44 and, atleast in some embodi- ‘ments, also responsible for implementing network connec- tivity between the computer instances 420 nimning on the server computer 442 and other computer instances. 420 rinsing on other server computers 442. Furthermore, the server computer 442 may host tiple vewalization layers 444 of the same or dilferent types on the same server computer 442. The viewalizaon layer 444 may” be any device, software, or imwvare used for providing a viral ‘computing platiorm for the computer instances 420, The Virwal computing platform may’ inchide various viral computer components, such as one or more virwal CPUs, Viral memory, and the ike, The computer instances 420 may be provided to the customers 406 of the service provider 404 and the customers 406 may aun an operating System oF an application on the computer instances 420. Furlier, the service provider 404 misy use one or more of is ‘vin eomipater instances 420 for executing its eppications. Atleast portion ofthe computer instances 420 may exeeute ikemel-level components for one ot more other computer instances 420. For example, a particular computer instance ‘may execute a parent partition configured 10 manage one oF ‘mon’ child partitions executed by other computer instances, ‘where the particular computer instance and the oiler com- puter instances are supported by the same virtualization layer 444 ‘Commands and other information may be included in an API call from the virtual machine management service 412, for the auto-seale service 430 to the vinualization layer 444 ‘The virtual machine management service 412 may enable the customers 406 t© manage and operate the computerUS 10,038,640 B2 15, instances 420, For example, the customer 406 may teansmit 1 request to the virtual machine management service 412 to terminate all computer instances 420 operated by’ the eus- tomers 406, The roguest may be an API call including information corresponding to the customers 406 and com Pler instances 420. The virtual machine management ser Vice 412 may determine the corresponding virwalization layer 44 for the computer instances 420 includ in the sad transmit a terminate command t the virtusliza- tion layer 444. The virtual machine management service 412 may be implemented in at least some embodiments, ‘enabling & variety of clieat applications to run on vietual ‘compuier servers or compute instances 420 instantiated on behalf of the enstomers 406, The computer instances 420 may each comprise a virtual machine, with is ovsn operating system comprising a networking software stack, and mul- tiple such instances may be hosted on a given server ‘computer 442 ata service provider network data center, Additionally a Toad balancing service 436, described in treater detail above in connection With FIG, 3, mney tansmit ‘commands and other information inchided in an API ell to the instance service 412, The instance service may deter mine the appropriate virwalization layer and case the instance 420 to operate in accordance with the API call. For ‘example, the load balancer service 436 may transmit an APL ‘all to a particular instance 420 implementing a load bale sncer, and the API call may be configured to cause the load balancer to manage and direct trallie (0 a set of instances of an auto-seale grup as described above, In various embod ments, the load balancing service 436 and auto-seale service ‘may transmit commands drcely to the virtlizaion layer 44 TIG, § shows an illustrative process S00 which may be used (0 assign instances of an auto-sale group to & Toad balancer using « background process in accordance with aa ‘embodiment. The process 500 may be perlormed by any Suitable system such as the auto-scale service deseribed ‘above in connection with FIGS, 2 and 3 or any combination ‘of systems of components thereof such as the background process described above, Retming to FIG. 5, in an embodi- ‘ment, the process 500 inches receiving a request to initiate an asynchronous work flow 502. An asynchronous work flow may bea set of operations or tasks to be performed by ‘computer systems of the service provider, such as. the ‘aul-scale service, where the sequence or order of operations ‘does not roquire the previous operation to be completed before starting the next operation. For example, as described above, the auto-scale service may receive a request o add a Toad Balancer to an auto-scale group which requires an asynchronous work flow inorder to process the request. The aulo-scale service may then execute the background process 504. In numerous variations 10 the process $00, if the background process in cumenily executing the auio-scule service may cause the background process to be restarted oF may cause the background process to restart at a particular point in the process 500, ‘The background process may then detemnine a set of load balaneers in a transient state 506. For example, the hack- round process may query the load balancer database for Joad balancers inthe adding state, added state, or removing state as described above. The background process may then determine a set of computer system instances associated ‘with the load balancees in the tansien state, This may inchude all instances of the auto-scale group or only instances that have yet to be assigned or deassigned to particular load balancer of the set of load balancers in & transient sate. The background process may then detenmine 0 o 16 ‘order of operations and batch size forthe instances and Toad balancers in the transient state to be updated, “The determination may be made based on a vary of factors inchuding a number of instances, a number of load balaneers, an amount of Toad on load balancing sevice, oF other factors suitable for determine a hatch size for updating instances and load balancers. Furthermore, the background process may query the load balancing service to determine An appropriate batch size, based at Jeat in parton te load fon the load balancing service andor a number or type of ‘operations to be performed on the load balancers (ez. ‘where instances are to be assigned or deassigned to the load balancer). In various embostimeats, if the Background pro- cess receives an error fromthe load balancing service, Tor ‘example if the load balancing service is unable to process the request, the background process may reduce the batch size andlor delay transmitting the API call associated with the batch for an interval of time. ‘he background process may then execute the operations associated with the determined hatches §12. Por example, the background process may transmit an API call othe oad balancing service configured to assign 10 instances of the anto-scale group to a particular Ioed halancer indicated in the API call, The aulo-sale service and/or background process may then generate audit log information S14. The Avdit log information, as described above, may enable the eestomer 10 obtain information corresponding to the load balaneers associated with the auto-seale group and the ‘operation performed in adding and/or removing the load balaneers (eg, result of executing the determined batch ‘opertions). The auto-seale service may generate atleast a portion of the audit log information after enqueuing the synchronous work flow and may generate additional suit Jog information after the asynchronous work flow is com- pleted, For example, the auto-scale service generated audit Jog information, including an operation performed in aceor- dance with the request anda start time. ARer completion of the asynchronous work flow, the auto-seale service may _genemte aut log information including atime the operation ‘was completed and whether the operation was successfl Tf additional requests are received S16 the bockground process may restart the process and determine loa balancers Jn transient stat. [F no addtional requests are roseived 516, the background process may continue executing oper ‘ions associated with the determined batches ual all open ‘ions are completed, Once all operations are completed the background process may update the load balancer database ‘with information associated with the Toad balancer in the transient state $18. For example, as described above, once all instances have been added to 2 particular load balancer, the background process may update the state information contained in the load balancer database to be added. FIG. 6 is a block diggram illustrating an example of a process 601 for enabling a customer to obtain information corresponding to one or more oad balancers associated with the customers auto-seale group in accordance with various embodiments. The process 600 may be performed by any stitale system such as a server in data center, multiple computing devices ina distributed system of & computing resource service provider, of any service operated by the ‘computing resource serve provider such as the auto-seale service described above in connection with FIGS. 2 and 3 ‘The process 600 includes a series of operations wherein 9 _Desetibel oadRalancers application programming interface call is received from a requestor, the requestor is authent cated, then, if authenticated, the load balancer database is queried for the requested information and provided to theUS 10,038,640 B2 17 requestor. In some embodiments, the information obtained fom the load balaicer database may be formatted or oth- ‘envise modified for the requestor In 602, a computing resource service provider oecives an application programming interface call from a customer oF ‘her entity to deseribe the lad balancers associated with an auto-scale group. As described above, the autoseale group for other component of the computing resource service provider may maintain a oad balancer database. The load balancer database may contain information corresponding 10 Toad balancers associated with an auto-seale group, for ‘example, identification information of the load balancers, & particular auto-scale group the load balancers are associated With, a state ofthe load halancers, a start time or end time the load halancer was assigned a particular state, and any ‘ther information suitable Tor describing the load balancers ‘oa customer. Furthermore, the request may indicate @ subset of the set of load bulancers associated with the auto-scale group for which the customer request information ‘corresponds. For example, the customer may indicate a 2 Partictlar load balancer to obtain information for or the ‘customer may indicate one or more atibutes of the oad bulancers to obtain information for such as all loed halanc- ‘ers added oF remowed from the auto-seale group in the last 24 hour interval In 604, the computing resource servive provider deter- imines whether the requestor has suficient privileges to have the request full. For example, the requestor may have provided an identity and proof of possession of eredentials ‘8 parameters with the Deseribel.adBalancers application Programming interface call For example, the requestor may supply, as proof of possession of credentials eoresponding to the ideality, information sufficient ta prove access to the ‘redentials,stch asa password, a eryptographic bash digest fof the password, eryplographic digital signature generated by a signing encryption key, or otber secret key verifiable by the computing resource service provider for authorizing the ‘identity of the requestor. The computing resource service provider may provide that identity and proof to an suhen- ication service which may then verify’ the identity and proot ‘of crcentials of the roquestor. Once the identity of the requestor is verified, the computing resource service pro- vider or authentication service may determine whether 2 Scourty policy andor ole associated withthe identity grants suliient privileges to allow the request to be Flies Ifthe computing resource service provider was unabe to determine the identity (eg, not found in a security data base), proof of credentials was insuficent to prove the Hey (ea, wre mec, th My of fe stor covld not otherwise be confirmed, the system performing the process 600 may nol proceed farther inthe process 600 ‘The system performing the process 600 may respond Io a fuilure in authentication with an errr messae to the requestor andor register the authentication faire in ‘security log 610, Othervise, if the requestor is authent ‘cated ancl determined to have suficient privileges to have the DeseribeLoadBalancers request fulfilled, the system per forming the process 600 may proceed to 606 ‘Once the computing resource service provider determines the requestor has sulicient privileges to have the request {ufiled, the computing resource service provider or com- ponent thereo, such asthe auto-seale service, may query the Toad balancer database forthe requested information 606. ‘The load balancer database may be maintained in a data orage service or datahase service of the computing resource service provider. In response to the query, the ‘aut-scale service may obtain the information indicated i 0 o 18 te request. For example, the auto-sale service may obta alist of load balancers associated with the auto-scale oup and a status of the load balances included in the list. The uto-sale service may then provide the list of load batancers to the requester in response to the request 608, FIG. 7 is-a block diagram illustrating an example of 3 process 700 for adding a load balancer to an auto-seale stoop in acconlance with various embodiments. The process 4700 may be performed by any suitable system such as @ server in a data center, ulliple computing devices in a Aisibuted system ofa computing resource service provider, fr any service of the computing resource service provider such as the anto-scale service desribed above in connection With FIGS. 2 and 3. The process 700 includes a series of ‘operations wherein @ AddLoadBolaneer application pro- gramming interface call is received from a requestor. the requestor is authenticated, then, if authenticated, a load balancer database is updated with lad balance information and the lod balaneer in added tothe auto-seale group. 1702, a computing resource service provider eoeives an application programming inerfoe call from a customer oF other entity t0 add one or more Toad bulancers 0 the suto-sale group. Adding the one oF more lad balancers to the auto-scale group may cause the one or more load balancerst© manage and direct network trafic to a set of instances included inthe autoscale group. Furthermore, axdling the one or more load balancers to the auto-seale oop may include operations performed by various other services of the computing resource service provider. For example, a customer may exuse the load balancers to be created by submiting a request to a lood balancing service described in greater detail above. Additionally. the auton scale service or component thereof, such as a backround process, may’ add the one oF more load balancers to the suto-scale group by submitting batches of API calls to the Joad balancing service, For example, the background pro- cess may determine a batch of 10 instances to assign 10 8 paniclar load balancer, generate an API call configured to Cause the load balancing service to assign the 10 instances to the particular lod balancer, and provide the API cll t0 the loa balancing service In 704, the computing resource service provider deter- snines whether the requestor has suliceatpivileges to have te request full. For example, the requestor may have provided an identity and proof of possession of eredntials 4s parameters with the Adal oadBalancer application pro- gramming interface call. Por example, the requestor may supply as proaf of possession of credentials eoeresponding {0 the identity, information sufieient 1 prove aecesst the credentials, suchas a passwort a erypropraphie hashidigst ofthe password, eeyplozraphie digit sianature generated by a signing encryption ke, or other secret key veniiable by the computing esource service provider for authorizing the ‘deniy of the requestor. The computing resource service provider may provide that identity and proof to an authen- fication service which may then verify the identity andl proof of credentials of the requestor. Once the identity of the requestor is verified, the computing resource service pro- Vider oF authentication service may determine whether a security policy andr role associate with the identity grants sulicient privileges (0 allow the request © be filled the computing resource service provider was unable © termine the identity (e, not found in a security daa- base), proof of credentials was insufficient to prove the ‘entity (eg. rong password), or denity of the requestor could otherwise not be confirmed, the system performing the process 700 may not proceed ftherin the process 700, TheUS 10,038,640 B2 19 system performing the process 700 may respond a failure in authentication with an emor message to the requestor ‘andor register the authentieation failure in a security log 712. Otherwise, ifthe requestor is authonticated and deter- mined to have suificient privileges to have the Addl.oad- Balancer request flied, the system performing the process 7700 may proceed to 706, 1706, the system performing the process 700 may update the load balancer database with information coreesponding to the load balancers indicated inthe request Por example, the request may include an identifier of the load balancer t© be added, a customer associated withthe load balancer, @ ‘computing resource service provider or service thereof ass0~ ‘ciated withthe load balancer, or other information suitable for enabling the anto-scale service o add the load balancer to the autoscale group. For example, updating the load balancer datahase may include generating. a row in the ‘database containing the identifier ofthe load balancer and 3 status ofthe load balancer The status may be determined by the auto-scale service. For example, ithe load balancer has not been previously added to the auto-seale group, the auto-scale service may assign a status of adding to the row ‘corresponding t0 the load balancer in the load balancer database, The ato-scale service or other system performing the process 700 may then queue an asynchronous work flow 708. The asynchronous work Now may be a process, such as process $00 described above in conncetion with FIG. 5, ‘configured to assign instances of the ato-seale group tthe Toad balancer indicated in the request. The asynchronous ‘work flow may be completed by the background process described hove. The auto-sale service may then generate fudit log information 710. The anit log information, as described above, may enable the customer to obtain infor- ‘ation corresponding to the load balancers associated with the auto-scale group and the operation performed in adding andr removing the load belaneers. The atto-seale service may generate atleast a portion of the audit log information aller enqueuing the asynchranows work flow and may gen- ‘erate ational audit log information afte the asynchronous ‘wot How is completed. For example, te asto-seale service generated aut log information, including. an operation Performed in accordance with the request and a start time, alter completion of the asynchronous work flow, the auto scale service may generate aut log information including atime the operation was completed and whether the opera tion was success FIG. 8 is a block diogram illustrating an example of a process 800 for removing 4 load balancer to an auto-scale ‘roup in accordance with various embodiments. The process 800 may be performed by any suitable system such as a server in a data center, multiple computing devices in & tating an entry in the load balancer database for information associated with the load balancer if que= ‘ying indicates thatthe load halancer isnot associated ‘ith the auto-seale group: nd lating the slate associated with the load batancers based atleast in parton the modification as a result of detecting completion ofthe update operation. 2. The computer-implemented method of claim 1, ‘wherein the state of the Ioad balancers includes a least one ‘of the following states: adding state, added state, removing. state, and in-service sate 3. The computer-implemented method of claim 1, ‘wherein the comptterimplemented method further com= prises modifying the slate of the load balancers in the load balancer database toa removing sate based at least impart ‘on a customer request fo remove the load balancer from the suto-seale group, 4, The computer-implemented method of claim 1, ‘wherein the comptterimplemented method further com- prises modifying the slate of the load balancers in the load balancer database to an adding state ase atleast in part on customer request toad the Joa balancer fo the auto-seale sroup, '3.A system, comprising: fone oF more processon: and ‘memory that stores computer-exeeutable instructions that, if exceuted, cause the one or more processor fo: detect an operation associated with one oF more load ‘alancers assigned toa group of computer instances associated with an auto-scale group; query a data store for information associated withthe auoseale group indicating the one o more load Talancers, the information including an assignment ‘of a foad balancer of the one or more oad balancers to the auto-scale group: generate an entry in the data store if querying indicates that the load balancer is aot associated with the auto-seale group sore an update tothe data store that indicates state of the load balancer and a type of the operation; snd query the data store forthe state of the Toad balancer in ‘order to determine performance of the operation 6. The systom of claim §, wherein the operation further ‘comprises adding load balancer to the ato-scale group and the sate of the load balancer indicates that atleast one instance ofthe auto-scale group is being assigned tothe load balancer 0 o 30 7. The system of claim 6, wherein the memory further includes computer-executable instructions tha, if executed, cause the one or more processors to determine thatthe at Jeast one instanceof the auto-scale group has boen assigned to the load balancer and the state of the one or more load blancers indicates thatthe at least one instance has bec assigned to the load balancer. 8. The system of claim 7, wherein the memory further includes computer-executable instructions that, if executed, cause the one or more processors to determine thatthe at Teast one instance ofthe auo-wale group has passed a heath check and the state of the one oF miore Toad balancers indicates thatthe load balancer of the load balancers is 9. The system of claim 8, wherein the memory further includes computer-executable instructions that, if executed, cause the one oF more processors to receive a request 10 remove atleast one Toad balancer of the one or more Toad Dalancers from the auto-sale group and the state of the at least one load balancer indicates that at least one instance of the auto-scale group is being deassigned from the at least ‘one Toad balancer 10. The system of claim 9, wherein the memory furher includes computor-exccutable instructions tht, if executed, cause the oe oF more processors to ddtemnine that the at least one instance has been deas- sianed from the load balancer; and ‘ag an entry from the data store associated withthe load ‘lancer for deletion, 11. The system of claim , wherein the entry inthe data store includes a row in the data store associated with an identifier of the load balancer. 12, A non-iransitory computer-readable storage medium having stored thereon executable instractions tha, a8 reslt of being exceuted by one or more processors of a computer system, cause the computer system to at Teast receive a request 10 update at east one load balancer associated wih a group of computer instances of an auo-seale group: ‘modify data store to include an identifier ofthe atleast ‘one load balancer and a state of the at least one load balancer, whore the state of the at least one load balancer is determined based at least in part on the request and the data store includes information indi- cating an assignment of the atleast one loa balancer to the auto-seale gronp and wherein the mosiieation includes generating an entry in the data store if a response to the request indicates that the at least one load balancer is not associated with the auro-seale ‘group; and perform one or more operations to full the request based ‘at least in part on the stale of the at least one load balancer indiated inthe data store 13. The non-ransitory computerreadable storage ‘medium of claim 12, wherein the instretions that cause the computer system to modify the data store further inelude instetions that modify the data store t0 include audit information corresponding to the request, 14. The non-tansitory computer-readable stonge medium of claim 12, wherein the insirvtions that cause the ‘computer system to modify the data store further include instructions that determine a previously reeorded state ofthe at least one load balancer by atleast querying the data store prior to modifying the data storeUS 10,038,640 B2 31 15, The non-ransitory computer-readable storage medium of elaim 12, wherein the instructions that cause the ‘computer system to receive the request furher cause the ‘compiler system to receive a roquest to remove the at least one load balancer from the auto-seale group; and ‘modily the stale of the at Teast one load balancer in the data sore to removing state. 16, The non-transitory computerreadable storage medium of claim 12, wherein the instructions further cause the computer system to: receive, from a background process, a second request for Toad balancers in one oF more transient sates; ‘bain, from the data store, information indicating load ‘palancers in a transient state of the one or more transient states based at least in part onthe state of the at least one load halaneer included in the data sore; and provide, in response to the second request, the informa tioa indicating Toad balancers in the transient state of the one or more transient sites 17. The non-ransitory computerreadable storage medium of claim 12, wherein the instructions further cause the computer system to modify the state of the atleast one Jad balancer based atleast in parton an update tothe state ‘ofthe atleast one load balancer receive from a background process 18, “The non-ransitory computerreadable storage medium of elim 12, wherein the instructions further cause the computer system to detect completion of the update to the atleast one load balancer and modify the state of the at least one load balancer as a rest 32