Edge Computing sa Distributed Data Problem ation are the only prerequistes for bung tomorrow's edge apoiations hope. Eels Note: Th fst post from an industry expert 2 ofthe Edge blog welcomes diverse opinions Industry practioner, analysts and researchers, highlighting thought leadersip inal areas of edge computing and acjacent technologies I you'd ike to propose an article, please see our Submission Guidelines, very thorny problem ing nothing mor nan a pha reau ple local calculations via serverles function. This lopcaleq onard Shelby in Christopher Nolan's excell rmavie Memento. Li ese edge apalcations would be incapable of emembering anything of ignfiance, (eg. the centralized cloud) foranyting more than the ne large globally-dstributed scale ng bs 1 of centraaing the coordination of state chang f you can take your data state and make gant ples oft inane data center, i's easier to do useful things wit bout f you have litle bts oft spread everyahere, your have the ahore er of keeping everthing consistent and cootnated across al the lacatons inorder ta achieve idempotent computing Eig compute s easy whanit's stateless or when sta stv parttionabe Take, for example an OT ds Similar, stateful computing suchas fevice maintains ts own stat ile phone app whic only mar asy when everything is centralize Je, when you want top ful computing at any ofthe many ea au manage and coordinate state a ‘edge locations or nodes and synchvon warantees? Without consistency guarantees, applications, devices and users version nl able applications, data corruption and data loss, idempotent computing prncpltiwohed = Q. generalize to the edge For the lst 20 years, the world has been industraizng the clent-server paradigm in giant, centralized hyperscale datacenters, And within these clouds, efforts are being made to super-size the database to run globally and across Inter and intercontinental distances, By relaxing data consistency and quality guarantees, t'shoped thatthe current generation of distributed databases cistibuted within a datacenter willsomehaw overcame the laws of Physics governing space and te to enable edge computing by becoming geo distributed mult master databases, Distributed databases that scale out within a datacenter do not cleanly generalze to scaling ut aross geography and break down under the weight oftheir design assumptions. rational distributed databases depend onthe following design assumptions ‘© AReliable data cents © Lowlatency igh avalabiey * Consistent latency & ter behavior 1 Very fw (or no} network spits ‘© Accurate tinekeeping using physial docks and network ime protocol {NTP} area networe ‘© NTPIs good enough for use cases where data ordering is handled across servers within the same rack o datacenter NTP sippage is< Isl ‘© Consensus mechanisms ae goed enough due tothe low latencies and high avalabty ofthe datacenter dass aN, The design assumptions fora geo dstbuted database are almost entirely opposite 1 Unretabe wide area networks 1 High and variable latency especially at inter-ity and intercntinental distances ‘© Dynamic network behavor with topology changes ang sporadic pation. Lossy time keeping ‘© Asymmetric autes cause inter-ciy and intercontinental clock coordination challenges resulting in stppage of ndceds of rilkseconds acoss a set of geo distributed time servers 1+ Atomic lacs may be used to overcome ths problem but are prohibtively expensive and complex to operate, ' Consensus too expensive and too slow with large numberof particpants to coordinate with over the internet ‘© Consensus is bitin that quorum must be centralized and highly availabe Fnetwerte pts (particulary asymmetric splits occur, managing quorum and geting relable consensus becomes very challenging ‘+ stant participants slow everyone cow asit takes more ime for ther to sené and receive messages ‘Adding more participants fe, edge cations) adds more overhead and slows consensus Gown as more participants need to vote Coordination stfu because partcpants ina large geographically-dstibuted syste need to agree that events happen n some temporal order Mecharisms like quorums are used in conventional distributed systems to Implement such coocdinatin. in geo-dstrbuted systems, the mechanisms of coordination become te constraining factor in how many partiipants or actors can participate in and perform complex behavior in a network of coordinating nodes. For geo-distributed databases to support edge computing acoocdnatin fee approach s Fequced that minmizes or even eliminates the need for caerdnation among participating actors For edge computing to become a realty, we need geo-distrbuted databases that can scale across hundreds of| locations worldwide yet actin concert to provide a single coherent mult-master database. This in turn requires us to ‘design systems that work on the internet with ts unpredictable network topology, use some form of tme-keeping that is nat lossy, and avid centralized forms of consensus yet starve at some shared version of truth in realtime For stateful edge computing tobe viable at scale withthe ability to handle real worl workloads, edge locations need to work together in a way tat = coorsnation-tree and able to make forward progress indepencenty even when network partitions may eccu.\ Edge native databases will unlock the promise andtiwohed = Q. Additonal these edge native databases willnot need appcabon designers and architects ta re-archtec or Fedesign thei cloud applications to scale and service rbons of users with hyperlocaly a he edge, The edge native databases will provide mul-region and mult-data-center orchestration of data and code without equing developers to have any special knowledge ot how to design archtect or buld these databases. Edge natve databases are coming When they artve, the true power and promise of edge computing wil be realized [Andonce that happens, willecore true that low latency and imagination wile the only prerequisites for buling, the applications of tomorow. Chetan Venkatesh and OurgaGokina are the Founders of Macremeta Corporation a Pale Alto CA based company that has bil the fist edge native geo-clstnbuted database and data services platform, Macrometa isin stealth fr ‘now Opinions expressedin this arte do nat necessarily reflect the opinions of any persons or entities other than the authors. baeFanatn aren Tuner at