Juniper Networks

Juniper was founded in February 1996 a5er observing an unusual confluence of three exponen>al trends: traffic on
the Internet was doubling every six to nine months, indica>ng strong demand for the Internet Protocol (IP) as the
basis for a worldwide network; meanwhile, IP routers were doubling in bandwidth only every two years, falling
well behind demand; finally, the cost of raw long distance bandwidth, which had been a major impediment in
building faster networks, was star>ng to drop exponen>ally thanks to improvements in op>cal fibers. So, the
demand for IP bandwidth was clearly there, the supply of raw fiber bandwidth was also there—what was missing
were routers of sufficiently high performance to turn the raw bandwidth on fibers into the exponen>ally growing
IP bandwidth needed by applica>ons.

At its founding Juniper was unique amongst vendors to bet exclusively on IP. Established network vendors were
inves>ng in ATM and Frame Relay for carriers, with a side bet on IP which was considered too slow, and on mul>-
protocol routers for enterprise. We bet on IP not only because the Internet was growing explosively, but also for
fundamental reasons: IP was designed from incep>on for computer-to-computer communica>on, and this we felt
was a much beVer basis for future networks than protocols designed for human-to-human communica>on. It was
also apparent at the >me that an IP router based network could be far more scalable and reliable than alterna2ves
because there was no central command and control structure—routers discovered network topology dynamically
and were therefore able to respond to failure induced topology changes using distributed rou>ng algorithms.

Juniper focused all its energy on fundamentally re-inven>ng router design, with the goal of making drama>c
improvements to performance and reliability. The ul>mate goal was, of course, to improve the performance and
reliability of the Internet itself. At the >me, routers were built by programming microprocessors to run both the
control plane (rou>ng protocols) and the data plane (packet forwarding) with some hardware accelera>on. This
architecture suffered from two major flaws: first, the forwarding design meant that router performance could not
keep pace with Internet traffic growth because of Moore’s Law limita>ons which provided a doubling of
performance on average only every two years, while Internet traffic was doubling on average every six to nine
months! The second, equally fundamental flaw, was that the microprocessor responsible for forwarding traffic was
also used to compute the rou>ng tables used by the forwarding logic. Limita>ons in rou>ng protocol
implementa>ons at the >me imposed an undue load on the microprocessor a5er topology changes. For example,
a5er a link failure the microprocessor would spend most of its cycles recompu>ng its rou>ng table and did not
have enough cycles le5 to also forward traffic reliably. This led to neighboring routers falsely declaring that their
links to the boVlenecked router had also failed because they were no longer receiving traffic, causing them in turn
to recompute their tables and as a result also fail to forward their traffic. PreVy soon, the en>re network would
come to a grinding halt. These flaws were causing increasingly frequent Internet outages, leading some industry
luminaries to declare that the Internet was doomed to fail.

Juniper solved this problem first by cleanly separa>ng forwarding and control: we implemented the en2re
forwarding path in silicon, while keeping the control plane running on a microprocessor. In other words, there was
no excep>on path to the microprocessor for forwarding, as there was in older accelerator assisted designs. We
coupled this separa>on by a clean-sheet implementa>on of rou>ng protocols to ensure that link failures did not
overload the control plane to the point where one router could affect its neighbors, causing network-wide
disrup>on. These simple-sounding changes to router architecture were the keys to scaling the Internet to grow to
become the global pla[orm it is today. This new architecture was first implemented in Juniper’s M40 router and
then in all of its subsequent router products. The silicon implementa>on of forwarding was about 20x faster than
so5ware forwarding but also flexible enough to support IP and MPLS, and even ATM as an interface. Indeed, when
UUNet and MCI decided to deploy M40s in their networks, each M40 replaced close to a dozen Cisco 7xxx routers!
Given the drama>c improvements the M40 and its successors made to the Internet’s performance as well as its
reliability, this architecture soon became the standard way to build routers.
This new way to build routers also had an influence on Ethernet switches. In 2000, Juniper’s second product, the
M160, was not only the fastest router on the market but also faster than all Ethernet switches! Before Juniper,
routers had the reputa>on of being slow and unreliable, but this changed drama>cally with the introduc>on of the
M40 and M160. Once the industry realized that IP routers could be designed to be faster than simple Ethernet
switches, design elements of Juniper routers also started to influence Ethernet switches. In fact, today, Ethernet
switches are designed effec>vely like IP routers, albeit at lower scale and func>onality in certain areas like packet
manipula>on, buffering, and forwarding table scale.

Pugng the en>re forwarding path for IP in silicon was not simple—it required fundamental inven>on at the
intersec>on of algorithms and silicon to solve a host of difficult problems: how to do fast route lookup with large
tables while making it easy to change routes on the fly; how to switch packets from input to output with minimal
conten>on, independent of packet size and traffic paVerns; how to solve the packet buffering and memory
management problems given the high packet arrival and departure rates; how do deal with network conges>on;
how to handle the mul>ple encapsula>ons that were in vogue at the >me; how to implement high performance
mul>cast; how to deliver granular quality of service; and of course, how to do this at full line rate concurrently on
all interfaces. The M40’s design featured a highly efficient single write/single read shared memory architecture
which was close to op>mal in terms of power consump>on per bit transported. M40’s basic design was scalable
enough that in the two years following the introduc>on of M40, Juniper quadrupled performance with the M160,
and then quadrupled it again two years later with the introduc>on of the T640, which replaced the shared memory
design with a shared switch design. The T-Series routers were the first to use the CLOS network topology to build a
fully non-blocking cell-switch with a distributed architecture that incorporated end-to-end conges>on control. To
my knowledge, this design was also the first to do cell spraying to maximize bandwidth u>liza>on and minimize
jiVer by exploi>ng all available paths—a technique that is now coming into vogue two decades later in the form of
packet spraying in high performance Ethernet networks for AI.

These fundamental advances in hardware design were accompanied by a co-designed opera>ng system called
JUNOS, which encompassed both a new embedded OS to manage real->me aspects of the hardware as well as a
standard BSD Unix environment for running rou>ng protocols and management func>ons. As men>oned earlier,
Juniper’s rou>ng protocols were wriVen from scratch in a record 18 months to be compa>ble with exis>ng
implementa>ons, while solving fundamental scaling issues that bedeviled older implementa>ons. JUNOS also
featured an innova>ve management system that enabled interac>ve and batch use as well as full rollback
capability to recover from human error and so5ware bugs. These advances were recognized by rapid adop>on of
Juniper’s products, and in 1999 Juniper became the fastest growing technology company in history.
Vinod Khosla, the venture capitalist most responsible for Juniper’s crea>on, maintains that “Juniper saved the
Internet”. Whether this is true or not will be for historians to judge. What is undeniable, though, is that Juniper has
had a profound impact on the networking industry through its many innova>ons over its 28 years as an
independent company.

The M40 was conceived by a small and excep>onally talented team of people. The ini>al design of both the
hardware and the so5ware was done by a mere dozen engineers. One year a5er its founding, there were only 25
engineers at the company. When the M40 launched in September 1998, there were only 42. These engineers
invented the hardware and so5ware architecture, built the 5 silicon chips that implemented the hardware,
designed, and built the first system, wrote the complex so5ware that makes up a router, par>cularly the rou>ng
protocols that are at the core of a router’s func>onality, made sure that this so5ware interworked with exis>ng
routers at scale, and got the machine ready for volume produc>on in a mere two years and seven months! For
these engineers, the period from 1996 to 1998 will forever remain a transcendent part of their careers—as it will
for me personally. It was a privilege to lead such an incredibly talented and passionate team and to have had the
opportunity to solve such an important problem at a cri>cal juncture in the development of the Internet. To these
people, especially Dennis Ferguson and Bjorn Liencres, my co-founders, and to so many others who came later,
and to those outside the company, par>cularly the investors and customers who helped Juniper grow and become
the industry force that it was, I owe a special gra>tude, as does the network industry.