tY Ptoc 1t!xt!/ 2 8 8 8 9 9 9 10 10 10 10 10 12 12 12 12 12 12 12 12 12 13 13 13 13 13 13 13 13 13 13 14 14 14 14 15 15 15 16 17 18 19 20 22 22 22 23 TabIe of Content Table of Content Credits About this book Acknowledgements ntroduction Why the sudden, exponential popularity? What does this book cover? What does this book not cover? Prerequisites Exercises Source code Where will this book lead you? Chapter Overview Why? Starting up Understanding Basics AP Quick Tour Utilities Buffers Event Emitter Timers Low-level File System HTTP Streams TCP Server UNX Sockets Datagrams (UDP) Child Processes Streaming HTTP Chunked Responses TLS / SSL HTTPS Making Modules Debugging Automated Unit Testing Callback Flow Why? Why the event loop? Solution 1: Create more call stacks Solution 2: Use event callbacks Why Javascript? How Learned to Stop Fearing and Love Javascript Function Declaration Styles Functions are first-class objects JSLint Javascript versions Handling callbacks References Table of Content Table of Content 2/147 25 25 26 26 27 27 28 28 29 29 29 30 30 31 32 32 32 32 32 32 34 34 34 34 34 34 34 35 35 35 35 35 35 35 36 36 37 39 39 40 40 40 40 40 42 42 42 43 43 44 44 44 45 45 45 Starting up nstall Node NPM - Node Package Manager NPM commands npm ls [filter] npm install package[@filters] npm rm package_name[@version] [package_name[@version] ...] npm view [@] [[.]...] Understanding Understanding the Node event loop An event-queue processing loop Callbacks that will generate events Don't block! Understanding Modules How Node resolves a module path Core modules Modules with complete or relative path As a file As a directory As an installed module AP quick tour Processes process child_process File system fs path Networking net dgram http tls (ssl) https dns Utilities console util Buffers Slice a buffer Copy a buffer Buffer Exercises Exercise 1 Exercise 2 Exercise 3 Event Emitter .addListener .once .removeAllListeners Creating an Event Emitter Event Emitter Exercises Exercise 1 Exercise 2 Timers setTimeout clearTimeout Table of Content Table of Content 3/147 46 46 46 47 48 49 49 50 51 51 52 53 55 55 55 55 55 55 56 57 57 58 58 58 58 59 59 59 60 60 60 60 61 62 62 62 63 63 64 64 64 64 65 65 65 65 66 66 66 67 67 67 68 68 70 70 70 72 setnterval clearnterval process.nextTick Escaping the event loop A note on tail recursion Low-level file-system fs.stat and fs.fstat Open a file Read from a file Write into a file Close Your files Advanced Tip: careful when appending concurrently File-system Exercises Exercise 1 - get the size of a file Exercise 2 - read a chunk from a file Exercise 3 - read two chunks from a file Exercise 4 - Overwrite a file Exercise 5 - append to a file Exercise 6 - change the content of a file HTTP HTTP Server The http.ServerRequest object req.url req.method req.headers The http.ServerResponse object Write a header Change or set a header Remove a header Write a piece of the response body HTTP Client http.get() http.request() HTTP Exercises Exercise 1 Exercise 2 Exercise 3 Exercise 4 Streams, pump and pipe ReadStream Wait for data Know when it ends Pause it Resume it WriteStream Write Wait for it to drain Some stream examples Filesystem streams Network streams The Slow Client Problem What can we do? Pump ... and his cousin "pipe" Making your own ReadStream WriteStream TCP Table of Content Table of Content 4/147 72 73 73 73 74 74 74 74 75 76 76 76 76 78 78 78 78 79 80 82 82 83 84 85 85 86 86 86 87 87 88 88 89 89 90 90 91 91 92 92 92 92 93 93 94 94 94 94 94 95 95 Write a string or a buffer end ...and all the other methods dle sockets Keep-alive Delay or no delay server.close() Listening TCP client Error handling TCP Exercises Exercise 1 Exercise 2 UNX sockets Server Client Passing file descriptors around Read or write into that file Listen to the server socket Datagrams (UDP) Datagram server Datagram client Datagram Multicast Receiving multicast messages Sending multicast messages What can be the datagram maximum size? UDP Exercises Exercise 1 Child processes Executing commands Spawning processes Killing processes Child Processes Exercises Exercise 1 Streaming HTTP chunked responses A streaming example Streaming Exercises Exercise 1 TLS / SSL Public / private keys Private key Public key TLS Client TLS Server Verification TLS Exercises Exercise 1 Exercise 2 Exercise 3 Exercise 4 Exercise 5 Table of Content Table of Content 5/147 96 96 96 98 98 98 99 100 100 101 102 103 103 103 104 107 108 108 109 109 109 110 110 110 110 110 110 110 110 110 110 110 111 111 111 111 111 111 111 113 114 116 118 120 120 120 120 121 121 122 122 123 123 123 124 124 125 125 125 HTTPS HTTPS Server HTTPS Client Making modules CommonJS modules One file module An aggregating module A pseudo-class A pseudo-class that inherits node_modules and npm bundle Bundling Debugging console.log Node built-in debugger Node nspector Live edit Automated Unit Testing A test runner Assertion testing module should.js Assert truthfulness: or untruthfulness: === true === false emptiness equality equal (strict equality) assert numeric range (inclusive) with within test numeric value is above given value: test numeric value is below given value: matching rec+gular expressions test length substring inclusion assert typeof property existence array containment own object keys responds to, asserting that a given property is a function: Putting it all together Callback flow The boomerang effect Step Parallel execution Appendix - Exercise Results Chapter: Buffers Exercise 1 One solution: Exercise 2 One solution: Exercise 3 One Solution: Chapter: Event Emitter Exercise 1 One solution: Exercise 2 One solution: Chapter: Low-level File System Exercise 1 - get the size of a file One solution Table of Content Table of Content 6/147 126 126 127 127 128 128 129 129 130 130 131 131 131 132 132 133 133 134 134 135 135 135 137 137 137 138 138 138 139 139 139 140 140 141 141 141 144 144 145 145 146 146 147 147 Exercise 2 - read a chunk from a file One solution Exercise 3 - read two chunks from a file One solution Exercise 4 - Overwrite a file One solution: Exercise 5 - append to a file One Solution: Exercise 6 - change the content of a file One solution: Chapter: HTTP Exercise 1 One solution: Exercise 2 One solution: Exercise 3 One solution: Exercise 4 One solution: Chapter: Child processes Exercise 1 One solution: Chapter: Streaming HTTP Chunked responses Exercise 1 One solution: Chapter: UDP Exercise 1 One solution: Chapter: TCP Exercise 1 One solution: Exercise 2 One solution: Chapter: SSL / TLS Exercise 1 One solution: Exercise 2 One solution: Exercise 3 One solution: Exercise 4 One solution: Exercise 5 One solution: Table of Content Table of Content 7/147 Cred|ts This secono revision ol this book coulo not have been oone without the the help ol Timothy Kevin Oxley, who co-revieweo eno eoiteo most ol this book. About th|s book This book was built using Nooe.js, Express, the Jaoe templating engine, wkhtmltopol, bash scripts, git ano github. Version: 1.1 Acknow|edgements I'm gratelul to my oear wile Susana lor putting up with my mooo swings about this book, to my Iather lor teaching me how to program Basic on a ZX Spectrum when I was 10 years olo, to my Farents ano my Granomother Maria oo Carmo lor ollering me my nrst IBM FC clone a little later ano to my lrieno Feoro Menoes who gave me the ioea lor making this book. Hands-on Node Credits 8/147 Introduct|on At the European JSConl 2009, a young programmer by the name ol Ryan Dahl, introouceo a project he hao been working on. This project was a platlorm that combining Google's V8 Javascript engine ano an event loop. The project then took a oillerent oirection lrom other server-sioe Javascript platlorms: all IO primitives were event-oriven, ano there was no way arouno it. Leveraging the power ano simplicity ol Javascript, it turneo the oilncult task ol writing asynchronous applications into an easy one. Since receiving a stanoing ovation at the eno ol his talk, Dahl's project has been met with unpreceoenteo growth, popularity ano aooption. The project was nameo Nooe.js, now known to oevelopers simply as 'Nooe'. Nooe provioes purely eventeo, non-blocking inlrastructure lor builoing highly concurrent soltware. Nooe allows you to easily construct last ano scalable network services. Why the sudden, exponent|a| popu|ar|ty? Server-sioe Javascript has been arouno lor some time, what makes this platlorm so appealing? In previous server-sioe Javascript implementations, javascript was the raison o'etre, ano the approach locusseo on translating common practices lrom other platlorms like Ruby, FERL ano Fython, into Javascript. Nooe takes a leap lrom this ano says: "Let's use the successlul event-oriven programming mooel ol the web ano use it to make an easy way to builo scalable servers. Ano let's make it the only way people can oo anything on this platlorm.". It can be argueo that Javascript itsell contributeo to much ol Nooe's success, but that woulo not explain why other the server-sioe projects proceeoing Nooe have not yet come close in popularity. The ubiquity ol Javascript surely has playeo a role, but, as Ryan Dahl points out, unlike other SSJS attempts, a unilying clientserver language was not the primary goal lor Nooe. In the perspective ol this author, there are three lactors contributing to Nooe's success: 1. Nooe is Easy - Nooe makes event-oriven IO programming, the best way to oo IO programming, much easier to unoerstano ano achieve than ever belore. 2. Nooe is Lean - Nooe ooes not try to solve all problems. It lays the lounoation ano supports the basic internet protocols using clean, lunctional AFIs. 3. Nooe ooes not Compromise - Nooe ooes not try to be compatible with pre-existing soltware, it takes a lresh look at what many believe is the right oirection. What does th|s book cover? Hands-on Node ntroduction 9/147 We will analyze what makes Nooe a oillerent proposal to other server-sioe solutions, why you shoulo use it, ano how to get starteo. We will start with an overview but quickly oive into some cooe mooule- by-mooule. By the eno ol this book you shoulo be able to builo ano test your own Nooe mooules, service prooucersconsumers ano leel comlortable using Nooe's conventions ano AFI. What does th|s book not cover? This book ooes not attempt to cover the complete Nooe AFI. Insteao, we will cover what the author thinks is requireo to builo most applications he woulo builo on Nooe. This book ooes not cover any Nooe lrameworks, Nooe is a great tool lor builoing lrameworks ano many are available, such as cluster management, inter-process communication, web lrameworks, network tralnc collection tools, game engines ano many others. Belore you oive into any ol those you shoulo be lamiliar with Nooe's inlrastructure ano what it provioes to these builoing blocks. Prerequ|s|tes This book ooes not assume you have any prior knowleoge ol Nooe, but the cooe examples are written in Javascript, so lamiliarity with the Javascript language will help. Exerc|ses This book has exercises in some chapters. At the eno ol this book you can nno the exercise solutions, but I aovise you to try oo them yoursell. Consult this book or use the comprehensive AFI oocumentation on the olncial http:nooejs.org website. Source code You can nno some ol the source cooe ano exercises useo in this book on GitHub: https:github.compgtehanoson_nooejs_source_cooe or you can oownloao it oirectly: https:github.compgtehanoson_nooejs_source_cooezipballmaster Where w||| th|s book |ead you? By the eno ol it, you shoulo unoerstano the Nooe AFI ano be able to pursue the exploration ol other things built on top ol it, being aoaptors, lrameworks ano mooules. Let's get starteo Hands-on Node ntroduction 10/147 Hands-on Node ntroduction 11/147 Chapter Overv|ew While this book ooes not neeo to be reao lrom cover to cover, it will help since some common concepts presenteo earlier in the book will probably not be repeateo. Why? Why ooes Nooe use event-oriven programming ano Javascript together? Why is Javascript a great language? Start|ng up How to install Nooe ano get Nooe mooules using NFM. Understand|ng Bas|cs How Nooe loaos mooules, how the Event Loop lunctions ano what to look out lor in oroer not to block it. API Qu|ck Tour Quick overview ol Nooe's core mooules. Ut|||t|es Uselul Nooe utilities. Buffers Learn to create, mooily ano access buller oata, an essential part ol the Nooe lunoamentals. Event Em|tter How the Event Emitter pattern is useo throughout Nooe ano how to use it lor nexibility in your cooe. T|mers Hands-on Node Chapter Overview 12/147 Nooe's timer AFI, reminiscent ol browsers. Low-|eve| F||e System How to use Nooe to open, reao ano write nles. HTTP Nooe's rich HTTF server ano client implementation. Streams The richness ol this great abstraction in Nooe. TCP Server Quickly setup a bare TCF server. UNIX Sockets How to use UNIX sockets ano use them to pass nle oescriptors arouno. Datagrams (UDP| The power ol oatagrams in Nooe Ch||d Processes Launching, watching, piping ano killing other processes. Stream|ng HTTP Chunked Responses HTTF in Nooe is streamable lrom the get go. TLS / SSL How to provioe ano consume secure streams. HTTPS Hands-on Node Chapter Overview 13/147 How to builo a secure HTTFS server or client. Mak|ng Modu|es How to make your app more mooular. Debugg|ng How to oebug your Nooe app. Automated Un|t Test|ng How to unit test your mooules. Ca||back F|ow How to manage intricate callback now in a sane way. Hands-on Node Chapter Overview 14/147 Why? Why the event |oop? The Event Loop is a soltware pattern that lacilitates non-blocking IO ,network, nle or inter-process communication,. Traoitional blocking programming ooes IO in the same lashion as regular lunction calls, processing may not continue until the operation is complete. Here is some pseuoo-cooe that oemonstrates blocking IO: l. var ost = db.query{`SELECT * FROM osts where id = l`|, 2. // rocessing from this line onward cannot execute until the line above comletes 3. doSomethingWithPost{ost|, 4. doSomethingElse{|, What is happening here? While the oatabase query is being executeo, the whole processthreao ioles, waiting lor the response. This is calleo blocking. The response to this query may take many thousanos ol CFU cycles, renoering the entire process unusable ouring this time. The process coulo have been servicing other client requests insteao ol just waiting. This ooes not allow you to parallelize IO such as perlorming another oatabase query or communicating with a remote web service, without involving concurrent programming trickery. The call stack becomes lrozen, waiting lor the oatabase server to reply. This leaves you with two possible solutions to keep the process busy while it's waiting: create more call stacks or use event callbacks. So|ut|on 1: Create more ca|| stacks In oroer lor your process to hanole more concurrent IO, you have to have more concurrent call stacks. Ior this, you can use threaos or some kino ol cooperative multi-threaoing scheme like co- routines, nbers, continuations, etc. The multi-threaoeo concurrency mooel can be very oilncult to conngure, unoerstano ano oebug, mainly because ol the complexity ol synchronization when accessing shareo inlormation, you never know when the threao you are running is going to be preempteo, which can leao to strange ano inconsistent bugs creeping up when the threao's interleaving is not as expecteo. On the other hano, cooperative multi-threaoing is a "trick" where you have more than one stack, ano each "threao" ol execution explicitly oe-scheoules itsell to give time to another "threao". This can relax the synchronization requirements but can become complex ano error-prone, since the threao scheouling is lelt at the hanos ol the programmers. Hands-on Node Why? 15/147 So|ut|on 2: Use event ca||backs An event callback is a lunction that gets invokeo when something signincant happens ,e.g. the result ol a oatabase query is available., To use event callbacks on the previous example, you coulo change it like so: l. callback = function{ost| { 2. doSomethingWithPost{ost|, // this will only execute when the db.query function returns. 3. }, 4. db.query{`SELECT * FROM osts where id = l`, callback|, 5. doSomethingElse{|, // this will execute indeendent of the returned status of the db.query call. Here you are oenning a lunction to be invokeo when the ob operation is complete, then passing this lunction as an callback argument to the ob query operation. The ob operation becomes responsible lor executing the callback when it completes. You can use an inline anonymous lunction to express this in a more compact lashion: l. db.query{`SELECT * FROM osts where id = l`, 2. function{ost| { 3. doSomethingWithPost{ost|, // this will only execute when the db.query function returns. 4. } 5. |, 6. doSomethingElse{|, // this will execute indeendent of the returned status of the db.query call. While ob.query,, is executing, the process is lree to continue running ooSomethingElse,,, ano even service new client requests. Ior quite some time, the C systems-programming "hacker" community has known that event-oriven programming is the best way to scale a server to hanole many concurrent connections. It has been known to be more elncient regaroing memory: less context to store, ano time: less context-switching. This knowleoge has been innltrating other platlorms ano communities: some ol the most well-known event loop implementations are Ruby's Event Machine, Ferl's AnyEvent ano Fython's Twisteo, ano there plenty ol others. Tip: Ior more inlo about event-oriven server implementations, see http:en.wikipeoia.orgwikiReactor_pattern. Implementing an application using one ol these lrameworks requires lramework-specinc knowleoge ano lramework-specinc libraries. Ior example: when using Event Machine, you shoulo avoio using synchronous libraries ,i.e. most libraries,. To gain the benent ol not blocking, you are limiteo to using only asynchronous libraries, built specincally lor Event Machine. Your server will not be able to scale Hands-on Node Why? 16/147 optimally il the event loop is constantly blocking which prevents timely processing ol IO events. Il you choose to go oown the non-blocking rabbit hole, you have to go all the way oown, never compromising, ano hope you come out on the other sioe in one piece. Nooe has been oeviseo as a non-blocking IO server platlorm lrom oay one, so generally you shoulo expect everything built on top ol it is non-blocking. Since Javascript itsell is very minimal ano ooes not impose any way ol ooing IO ,it ooes not have a stanoaro IO library,, Nooe has a clean slate to builo upon. Why Javascr|pt? Ryan Dahl began this project builoing a C platlorm, but maintaining the context between callbacks was too complicateo ano coulo to poorly structureo cooe, so he then turneo to Lua. Lua alreaoy has several blocking IO libraries ano this mix ol blocking ano non-blocking coulo conluse average oevelopers ano prevent many ol them lrom builoing scalable applications, so Lua wasn't ioeal either. Dahl then thought to Javascript. Javascript has closures ano nrst-class lunctions, making it inoeeo a powerlul match with eventeo IO programming. Closures are lunctions that inherit the variables lrom their enclosing environment. When a lunction callback executes it will magically remember the context in which it was oeclareo, along with all the variables available in that context ano any "parent" contexts. This powerlul leature is at the heart ol Nooe's success among programming communities. In the web browser, il you want to listen lor an event, a button click lor instance, you may oo something like: l. var clickCount = 0, 2. document.getElementById{`mybutton`|.onclick = function{| { 3. 4. clickCount ++, 5. 6. alert{`Clicked ` + clickCount + ` times.`|, 7. 8. }, 9. or, using jQuery: l. var clickCount = 0, 2. ${`button#mybutton`|.click{function{| { 3. clickedCount ++, 4. alert{`Clicked ` + clickCount + ` times.`|, 5. }|, In both examples we assign or pass a lunction as an argument, which may be executeo later. The click hanoling lunction has access every variable in scope at the point where the lunction is oeclareo, i.e. practically speaking, the click hanoler has access to the clickCount variable, oeclareo in the parent closure. Hands-on Node Why? 17/147 Here we are using a global variable, "clickCount", where we store the number ol times the user has clickeo a button. We can also avoio having a global variable accessible to the rest ol the system, by wrapping it insioe another closure, making the clicked variable only accessible within the closure we createo: l. {function{| { 2. var clickCount = 0, 3. ${`button#mybutton`|.click{function{| { 4. clickCount ++, 5. alert{`Clicked ` + clickCount + ` times.`|, 6. }|, 7. }|{|, In line 7 we are invoking a lunction immeoiately alter oenning it. Il this is strange to you, oon't worry We will cover this pattern later. How I Learned to Stop Fear|ng and Love Javascr|pt Javascript has gooo ano bao parts. It was createo in 199 by Netscape's Brenoan Eich, in a rush to ship the latest version ol the Netscape web browser. Due to this rush some gooo, even wonoerlul, parts got into Javascript, but also some bao parts. This book will not cover the oistinction between Javascript gooo ano bao parts. ,Ior all we know, we will only provioe examples using the gooo parts., Ior more on this topic you shoulo reao Douglas Crockloro book nameo "Javascript, The Gooo Farts", eoiteo by O'Reilly. In spite ol its orawbacks, Javascript quickly - ano somewhat unpreoictably - became the oe-lacto language lor web browsers. Back then, Javascript was useo to primarily to inspect ano manipulate HTML oocuments, allowing the creation the nrst oynamic, client-sioe web applications. In late 1998, the Worlo Wioe Web Consortium ,W3C,, stanoaroizeo the Document Object Mooel ,DOM,, an AFI oeviseo to inspect ano manipulate HTML oocuments on the client sioe. In response to Javascript's quirks ano the initial hatreo towaros the DOM AFI, Javascript quickly gaineo a bao reputation, also oue to some incompatibilities between browser venoors ,ano sometimes even between prooucts lrom the same venoor,. Despite milo to lull-blown hate in some oeveloper communities, Javascript became wioely aoopteo. Ior better ol lor worse, tooay Javascript is the most wioely oeployeo programming language on Earth. Il you learn the gooo leatures ol the language - such as prototypical inheritance, lunction closures, etc. - ano learn to avoio or circumvent the bao parts, Javascript can be a very pleasant language to work in. Hands-on Node Why? 18/147 Funct|on Dec|arat|on Sty|es A lunction can be oeclareo in many ways in Javascript. The simplest is oeclaring it anonymously: l. function{| { 2. console.log{`hello`|, 3. } Here we oeclare a lunction, but it's not ol much use, because oo not invoke it. What's more, we have no way to invoke it as it has no name. We can invoke an anonymous lunction in-place: l. {function{| { 2. console.log{`hello`|, 3. }|{|, Here we are executing the lunction immeoiately alter oeclaring it. Notice we wrap the entire lunction oeclaration in parenthesis. We can also name lunctions like this: l. function myFunction {| { 2. console.log{`hello`|, 3. } Here we are oeclaring a nameo lunction with the name: "myIunction". myIunction will be available insioe the scope it's oeclareo myFunction{|, ano also within inner scopes: l. function myFunction {| { 2. console.log{`hello`|, 3. } 4. 5. {function{| { 6. myFunction{|, 7. }|{|, A result ol Javascript treating lunctions as nrst-class objects means we can assign a lunction to a variable: l. var myFunc = function{| { 2. console.log{`hello`|, 3. } 4. This lunction is now available as the value ol the myFunc variable. We can assign that lunction to another variable: Hands-on Node Why? 19/147 var myFunc2 = myFunc, Ano invoke them just like any other lunction: l. myFunc{|, 2. myFunc2{|, We can mix both techniques, having a nameo lunction storeo in a variable: l. var myFunc2 = function myFunc{| { 2. console.log{`hello`|, 3. } 4. myFunc2{|, Note though, we cannot access myIunc lrom outsioe the scope ol myIunc itsell We can then use a variable or a lunction name to pass variables into lunctions like this: l. var myFunc = function{| { 2. console.log{`hello`|, 3. } 4. 5. console.log{myFunc|, or simply oeclare it inline il we oon't neeo it lor anything else: l. console.log{function{| { 2. console.log{`hello`|, 3. }|, Funct|ons are rst-c|ass objects In lact, there are no secono-class objects in Javascript. Javascript is the ultimate object-orienteo language, where everything is inoeeo, an object. As that, a lunction is an object where you can set properties, pass it arouno insioe arguments ano return them. Always a pleasure. Example: Hands-on Node Why? 20/147 l. var schedule = function{timeout, callbackfunction| { 2. return { 3. start: function{| { 4. setTimeout{callbackfunction, timeout| 5. } 6. }, 7. }, 8. 9. {function{| { l0. var timeout = l000, // l second ll. var count = 0, l2. schedule{timeout, function doStuff{| { l3. console.log{++ count|, l4. schedule{timeout, doStuff|, l5. }|.start{timeout|, l6. }|{|, l7. l8. // "timeout" and "count" variables l9. // do not exist on this scoe. In this little example we create a lunction ano store it in a lunction calleo "scheoule" ,starting on line 1,. This lunction just returns an object that has one property calleo "start" ,line 3,. This value ol the "start" property is a lunction that, when calleo, sets a timeout ,line !, to call a lunction that is passeo in as the "timeout" argument. This timeout will scheoule callback lunction to be calleo within the number ol seconos oenneo in the timeout variable passes. On line 9 we oeclare a lunction that will immeoiately be executeo on line 1o. This is a normal way to create new scopes in Javascript. Insioe this scope we create 2 variables: "timeout" ,line 10, ano "count" ,line 11,. Note that these variables will not be accessible to the outer scope. Then, on line 12, we invoke the schedule lunction, passing in the timeout value as nrst argument ano a lunction calleo doStuff as secono argument. When the timeout occurs, this lunction will increment the variable count ano log it, ano also call the scheoule all over again. So in this small example we have: lunctions passeo as argument, lunctions to create scope, lunctions to serve as asynchronous callbacks ano returning lunctions. We also here present the notions ol encapsulation ,by hioing local variables lorm the outsioe scope, ano recursion ,the lunction is calling itsell at the eno,. In Javascript you can even set ano access attributes in a lunction, something like this: l. var myFunction = function{| { 2. // do something crazy 3. }, 4. myFunction.someProerty = `abc`, 5. console.log{myFunction.someProerty|, 6. // #=> "abc" Javascript is inoeeo a powerlul language, ano il you oon't alreaoy oo, you shoulo learn it ano embrace Hands-on Node Why? 21/147 it's gooo parts. JSL|nt It's not to be covereo here, but Javascript inoeeo has some bao parts, ano they shoulo be avoioeo at all costs. One tool that's proven invaluable to the author is JSLint, by Douglas Crockloro. JSLint analyzes your Javascript nle ano outputs a series ol errors ano warnings, incluoing some known misuses ol Javascript, like using globally-scopeo variables ,like when you lorget the "var" keyworo,, ano lreezing values insioe iteration that have callbacks that use them, ano many others that are uselul. JSLint can be installeo using $ nm install slint Il you oon't have NFM installeo see section about NFM. ano can be run lrom the commano line like this: $ slint myfile.s To the author ol this book, JSlint has proven itsell to be an invaluable tool to guarantee he ooesn't lall into some common Javascript traps. Javascr|pt vers|ons Javascript is a stanoaro with it's own name - ECMAScript - ano it has gone through various iterations. Currently Nooe natively supports everything the V8 Javascript engine supports ECMA 3ro eoition ano parts ol the new ECMA th eoition. These parts ol ECMA are nicely oocumenteo on the lollowing github wiki page: https:github.comjoyentnooewikiECMA--Mozilla-Ieatures-Implementeo-in-V8 Hand||ng ca||backs In Nooe you can implement your own lunctions that perlorm asynchronous IO. To oo so, you can accept a callback lunction. You will invoke this lunction when the IO is oone. Hands-on Node Why? 22/147 l. var myAsyncFunction = function{someArgumentl, someArgument2, callback| { 2. // simulate some I/O was done 3. setTimeout{function{| { 4. // l second later, we are done with the I/O, call the callback 5. callback{|, 6. }, l000| 7. } On line 3 we are invoking a setTimeout to simulate the oelay ano asynchronism ol an IO call. This setTimeout lunction will call the nrst argument - a lunction we oeclare inline - alter 1000 milliseconos ,the secono argument, have gone by. This inline lunction will then call the callback that was passeo as the thiro argument to myAsyncFunction, notilying the caller the operation has enoeo. Using this convention ano others ,like the Event Emitter pattern which we will cover later, you can embrace Javascript as the ultimate language lor event-oriven applications. Tip: To lollow the Nooe convention, this lunction shoulo receive the error ,or null il there was no error, as nrst argument, ano then some "real" arguments il you wish to oo so. l. fs.oen{`/ath/to/file`, function{err, fd| { 2. if {err| { /* handle error */, return, } 3. console.log{`oened file and got file descritor ` + fd|, 4. }| Here we are using a Nooe AFI lunction fs.oen that receives 2 arguments: the path to the nle ano a lunction, which will be invokeo with an error or null on the nrst argument ano a nle oescriptor on the secono. References Event Machine: http:rubyeventmachine.com Twisteo: http:twisteomatrix.comtrac AnyEvent: http:soltware.schmorp.oepkgAnyEvent.html Javascript, the Gooo Farts - Douglas Crockloro - O'Reilly - http:www.amazon.comexecobioosASIN09o177!2wrrrlowioeweb JSLint http:www.jslint.com Hands-on Node Why? 23/147 Hands-on Node Why? 24/147 Start|ng up Insta|| Node The typical way ol installing Nooe on your oevelopment machine is by lollowing the steps on the nooejs.org website. Nooe shoulo install out ol the box on Linux, Macintosh, ano Solaris. With some ellort you shoulo be able to get it running on other Unix platlorms ano Winoows ,either via Cygwin or MinGW,. Nooe has several oepenoencies, but lortunately most ol them are oistributeo along with it. Il you are builoing lrom source you shoulo only neeo 2 things: python - version 2.! or higher. The builo tools oistributeo with Nooe run on python. libssl-oev - Il you plan to use SSLTLS encryption in your networking, you'll neeo this. Libssl is the library useo in the openssl tool. On Linux ano Unix systems it can usually be installeo with your lavorite package manager. The lib comes pre- installeo on OS X. Fick the latest stable version ano oownloao it: $ wget htt://nodes.org/dist/node~v0.4.7.tar.gz Expano it: $ tar xvfz node~v0.4.7.tar.gz Builo it: l. $ cd node~v0.4.7 2. $ ./configure 3. $ make 4. $ make install Tip: il you are having permission problems on this last step you shoulo run the install step as a super user like by: $ sudo make install Alter you are oone, you shoulo be able to run the nooe executable on the commano line: l. $ node ~v 2. v0.4.7 Hands-on Node Starting up 25/147 The node executable can be executeo in two main lashions: CLI ,commano-line interlace, or nle. To launch the CLI, just type $ node ano you will get a Javascript commano line prompt, which you can use to evaluate Javascript. It's great lor kicking the tires ano trying out some stull quickly. You can also launch Nooe on a nle, which will make Nooe parse ano evaluate the Javascript on that nle, ano when it enos ooing that, it enters the event loop. Once insioe the event loop, nooe will exit il it has nothing to oo, or will wait ano listen lor events. You can launch Nooe on a nle like this: $ node myfile.s or, il you wish, you can also make your nle oirectly executable by changing the permissions like this: $ chmod o+x myfile.s ano insert the lollowing as the nrst line ol the nle: #!/usr/bin/env node You can then execute the nle oireclty: $ ./myfile.s NPM - Node Package Manager NFM has become the stanoaro lor managing Nooe packages throughout time, ano tight collaboration between Isaac Schlueter - the original author ol NFM - ano Ryan Dahl - the author ano maintainer on Nooe - has lurther tighteneo this relationship to the point where, starting at version 0.!.0, Nooe supports the ackage.son nle lormat lormat to inoicate oepenoencies ano package starting nle. To install it you type: $ curl htt://nms.org/install.sh sh Il that lails, try this on a temporary oirectory: l. $ git clone htt://github.com/isaacs/nm.git 2. $ cd nm 3. $ sudo make install NPM commands Hands-on Node Starting up 26/147 NFM can be useo on the commano line. The basic commanos are: npm |s [|ter] Use this to see the list ol all packages ano their versions ,npm ls with no nlter,, or nlter by a tag ,npm nlter tag,. Examples: List all installeo packages: $ nm ls installed List all stable packages: $ nm ls stable You can also combine nlters: $ nm ls installed stable You can also use npm ls to search by name: $ nm ls fug ,this will return all packages that have "lug" insioe its name or tags, You can also query it by version, prenxeo with the "" character: $ nm ls @l.0 npm |nsta|| package[@|ters] With this commano you can install a package ano all the packages it oepenos on. To install the latest version ol a package oo: $ nm install ackage_name Example: $ nm install exress To install a specinc version ol a package oo: $nm install ackage_name@version Example: $ nm install exress@2.0.0beta To install the latest within a version range you can specily, lor instance: $ nm install exress@">=0.l.0 <0.2.0" You can also combine many nlters to select a specinc version, combining version range ano or tags like this: Hands-on Node Starting up 27/147 $ nm install sax@">=0.l.0 <0.2.0" bench suervisor npm rm package_name[@vers|on] [package_name[@vers|on] ...] Use this commano to uninstall packages. Il versions are omitteo, then all the louno versions are removeo. Example: $ nm rm sax npm v|ew To view all ol a package inlo. Delaults to the latest version il version is omitteo. View the latest inlo on the "connect" package: $ nm view connect View inlormation about a specinc version: $ nm view connect@l.0.3 Iurther on we will look more into NFM ano how it can help us bunole ano "lreeze" application oepenoencies. Hands-on Node Starting up 28/147 Understand|ng Understand|ng the Node event |oop Nooe makes eventeo IO programming simple ano accessible, putting speeo ano scalability on the nngertips ol the common programmer. But the event loop comes with a price. Even though you are not aware ol it ,ano Nooe makes a gooo job at this,, you shoulo unoerstano how it works. Every gooo programmer shoulo know the intricacies ol the platlorms he she is builoing lor, its oo's ano oon'ts, ano in Nooe it shoulo be no oillerent. An event-queue process|ng |oop You shoulo think ol the event loop as a loop that processes an event queue. Interesting events happen, ano when they oo, they go in a queue, waiting lor their turn. Then, there is an event loop popping out these events, one by one, ano invoking the associateo callback lunctions, one at a time. The event loop pops one event out ol the queue ano invokes the associateo callback. When the callback returns the event loop pops the next event ano invokes the associateo callback lunction. When the event queue is empty, the event loop waits lor new events il there are some penoing calls or servers listening, or just quits ol there are none. So, let's jump into our nrst Nooe example. Write a nle nameo hello.s with the lollowing content: Source code in chaters/understanding/l_hello.s l. setTimeout{function{| { 2. console.log{`World!`|, 3. }, 2000|, 4. console.log{`Hello`|, Run it using the nooe commano line tool: $ node hello.s You shoulo see the woro "Hello" written out, ano then, 2 seconos later, "Worlo". Shoulon't "Worlo" have been written nrst since it appears nrst on the cooe? No, ano to answer that properly we must analyze what happens when executing this small program. On line 1 we oeclare an anonymous lunction that prints out "Worlo". This lunction, which is not yet executeo, is passeo in as the nrst argument to a setTimeout call, which scheoules this lunction to run in 2000 milliseconos. Then, on line !, we output "Hello" into the console. Two seconos later the anonymous lunction we passeo in as an argument to the setTimeout call is invokeo, printing "Worlo". Hands-on Node Understanding 29/147 So, the nrst argument to the setTimeout call is a lunction we call a "callback". It's a lunction which will be calleo later, when the event we set out to listen to ,in this case, a time-out ol 2 seconos, occurs. We can also pass callback lunctions to be calleo on events like when a new TCF connection is establisheo, some nle oata is reao or some other type ol IO event. Alter our callback is invokeo, printing "Worlo", Nooe unoerstanos that there is nothing more to oo ano exits. Ca||backs that w||| generate events Let's complicate this a bit lurther. Let's keep Nooe busy ano keep on scheouling callbacks like this: Source code in chaters/understanding/2_reeat.s l. {function schedule{| { 2. setTimeout{function{| { 3. console.log{`Hello World!`|, 4. schedule{|, 5. }, l000|, 6. }|{|, Here we are wrapping the whole thing insioe a lunction nameo "scheoule", ano we are invoking it immeoiately alter oeclaring it on line o. This lunction will scheoule a callback to execute in 1 secono. This callback, when invokeo, will print "Hello Worlo" ano then run schedule again. On every callback we are registering a new one to be invokeo one secono later, never letting Nooe nnish. This little script will just keep printing "Hello Worlo". Don't b|ockI Nooe primary concern ano the main use case lor an event loop is to create highly scalable servers. Since an event loop runs in a single threao, it only processes the next event when the callback nnishes. Il you coulo see the call stack ol a busy Nooe application you woulo see it going up ano oown really last, invoking callbacks ano picking up the next event in line. But lor this to work well you have to clear the event loop as last as you can. There are two main categories ol things that can block the event loop: synchronous IO ano big loops. Nooe AFI is not all asynchronous. Some parts ol it are synchronous like, lor instance, some nle operations. Don't worry, they are very well markeo: they always terminate in "Sync" - like ls.reaoIileSync - , ano they shoulo not be useo, or useo only when initializing. On a working server you shoulo never use a blocking IO lunction insioe a callback, since you're blocking the event loop ano preventing other callbacks - probably belonging to other client connections - lrom being serveo. Hands-on Node Understanding 30/147 One lunction that is synchronous ano ooes not eno in "Sync" is the "require" lunction, which shoulo only be useo when initializing an app or a mooule. Tip: Don't put a require statement insioe a callback, since it is synchronous ano thus will slow oown your event loop. The secono category ol blocking scenarios is when you are perlorming loops that take a lot ol time, like iterating over thousanos ol objects or ooing complex time-taking operations in memory. There are several techniques that can be useo to work arouno that, which we'll cover later. Here is a case where we present some simple cooe that blocks the event loop: l. var oen = false, 2. 3. setTimeout{function{| { 4. oen = true, 5. }, l000| 6. 7. while{!oen| { 8. // wait 9. } l0. ll. console.log{`oened!`|, Here we are setting a timeout, on line 3, that invokes a lunction that will set the oen variable to true. This lunction is set to be triggereo in one secono. On line 7 we are waiting lor the variable to become true. We coulo be leao to believe that, in one secono the timeout will happen ano set oen to true, ano that the while loop will stop ano that we will get "openeo" ,line 11, printeo. But this never happens. Nooe will never execute the timeout callback because the event loop is stuck on this while loop starteo on line 7, never giving it a chance to process the timeout event Understand|ng Modu|es Client-sioe Javascript has a bao reputation also because ol the common namespace shareo by all scripts, which can leao to connicts ano security leaks. Nooe implements the CommonJS mooules stanoaro, where each mooule is separateo lrom the other mooules, having a separate namespace to play with, ano exporting only the oesireo properties. To incluoe an existing mooule you can use the require lunction like this: var module = require{`module_name`|, This will letch a mooule that was installeo by npm. Il you want to author mooules ,as you shoulo Hands-on Node Understanding 31/147 when ooing an application,, you can also use the relative notation like this: var module = require{"./ath/to/module_name"|, This will letch the mooule relatively to the current nle we are executing. We will cover creating mooules on a later section. In this lormat you can use an absolute path ,starting with "", or a relative one ,starting with ".",, Mooules are loaoeo only once per process, that is, when you have several require calls to the same mooule, Nooe caches the require call il it resolves to the same nle. Which leaos us to the next chapter. How Node reso|ves a modu|e path So, how ooes nooe resolve a call to "require,mooule_path,"? Here is the recipe: Core modu|es There are a list ol core mooules, which Nooe incluoes in the oistribution binary. Il you require one ol those mooules, Nooe just returns that mooule ano the require,, enos. Modu|es w|th comp|ete or re|at|ve path Il the mooule path begins with "." or "", Nooe tries to loao the mooule as a nle. Il it ooes not succeeo, it tries to loao the mooule as a oirectory. As a |e When loaoing as a nle, il the nle exists, Nooe just loaos it as Javascript text. Il not, it tries ooing the same by appenoing ".js" to the given path. Il not, it tries appenoing ".nooe" ano loao it as a binary aoo-on. As a d|rectory Il appenoing "package.json" is a nle, try loaoing the package oennition ano look lor a "main" nelo. Try to loao it as a nle. Il unsuccesslul, try to loao it by appenoing "inoex" to it. As an |nsta||ed modu|e Il the mooule path ooes not begin with "." or "" or il loaoing it with complete or relative paths ooes not work, Nooe tries to loao the mooule as a mooule that was previously installeo. Ior that it aoos "nooe_mooules"to the current oirectory ano tries to loao the mooule lrom there. Il it ooes not Hands-on Node Understanding 32/147 succeeo it tries aooing "nooe_mooules" to the parent oirectory ano loao the mooule lrom there. Il it ooes not succeeo it moves again to the parent oirectory ano so on, until either the mooule is louno or the root ol the tree is louno. This means that you can put your bunole your Nooe mooules into your app oirectory, ano Nooe will nno those. Later we will see how using this leature together with NFM we can bunole ano "lreeze" your application oepenoencies. Also you can, lor instance, have a nooe_mooules oirectory on the home loloer ol each user, ano so on. Nooe tries to loao mooules lrom these oirectories, starting nrst with the one that is closest up the path. Hands-on Node Understanding 33/147 API qu|ck tour Nooe provioes a platlorm AFI that covers mainly ! aspects: processes nlesystem networking utilities This book is not meant to be a comprehensive coverage ol the whole Nooe AFI. Ior that you shoulo consult the Nooe online oocumentation on http:nooejs.org. Processes Nooe allows you to analyze your process ,environment variables, etc., ano manage external processes. The involveo mooules are: process Inquire the current process to know the FID, environment variables, platlorm, memory usage, etc. ch||d_process Spawn ano kill new processes, execute commanos ano pipe their outputs. F||e system Nooe also provioes a low-level AFI to manipulate nles, which is inspireo by the FOSIX stanoaro, ano is compriseo by the lollowing mooules: fs Iile manipulation: create, remove, loao, write ano reao nles. Create reao ano write streams ,covereo later,. path Hands-on Node AP quick tour 34/147 Normalize ano join nle paths. Check il a nle exists or is a oirectory. Network|ng net Create a TCF server or client. dgram Receive ano seno UDF packets. http Create an HTTF server or Client. t|s (ss|| The tls mooule uses OpenSSL to provioe Transport Layer Security anoor Secure Socket Layer: encrypteo stream communication. https Implementing http over TLSSSL. dns Asynchronous DNS resolution. Hands-on Node AP quick tour 35/147 Ut|||t|es conso|e Nooe provioes a global "console" object to which you can output strings using: console.log{"Hello"|, This simply outputs the string into the process stdout alter lormatting it. You can pass in, insteao ol a string, an object like this: l. var a = {l: true, 2: false}, 2. console.log{a|, // => { `l`: true, `2`: false } In this case console.log outputs the object using util.inspect ,covereo later,, You can also use string interpolation like this: l. var a = {l: true, 2: false}, 2. console.log{`This is a number: %d, and this is a string: %s, and this is an obect oututted as JSON: %`, 42, `Hello`, a|, 3. Which outputs: This is a number: 42, and this is a string: Hello, and this is an obect oututted as JSON: {"l":true,"2":false} console also allows you to write into the the stoerr using: console.warn{"Warning!"|, ano to print a stack trace: console.trace{|, Hands-on Node Utilities 36/147 l. Trace: 2. at obect Context:l:9 3. at Interface.<anonymous> {rel.s:l7l:22| 4. at Interface.emit {events.s:64:l7| 5. at Interface._onLine {readline.s:l53:l0| 6. at Interface._line {readline.s:408:8| 7. at Interface._ttyWrite {readline.s:585:l4| 8. at ReadStream.<anonymous> {readline.s:73:l2| 9. at ReadStream.emit {events.s:8l:20| l0. at ReadStream._emitKey {tty_osix.s:307:l0| ll. at ReadStream.onData {tty_osix.s:70:l2| l2. ut|| Nooe has an util mooule which which bunoles some lunctions like: l. var util = require{`util`|, 2. util.log{`Hello`|, which outputs a the current timestamp ano the given string like this: l4 Mar l6:38:3l ~ Hello The insect lunction is a nice utility which can aio in quick oebugging by inspecting ano printing an object properties like this: l. var util = require{`util`|, 2. var a = {l: true, 2: false}, 3. console.log{util.insect{a||, 4. // => { `l`: true, `2`: false } util.insect accepts more arguments, which are: util.insect{obect, showHidden, deth = 2, showColors|, the secono argument, showHidden shoulo be turneo on il you wich insect to show you non- enumerable properties, which are properties that belong to the object prototype chain, not the object itsell. deth, the thiro argument, is the oelault oepth on the object graph it shoulo show. This is uselul lor inspecting large objects. To recurse inoennitely, pass a null value. Tip: util.insect keeps track ol the visiteo objects, so circular oepenoencies are no problem, ano will appear as "|Circular|" on the outputteo string. The util mooule has some other niceties, such as inheritance setup ano stream pumping, but they belong on more appropriate chapters. Hands-on Node Utilities 37/147 Hands-on Node Utilities 38/147 Buffers Natively, Javascript is not very gooo at hanoling binary oata. So Nooe aoos a native buller implementation with a Javascript way ol manipulating it. It's the stanoaro way in Nooe to transport oata. Generally, you can pass bullers on every Nooe AFI requiring oata to be sent. Also, when receiving oata on a callback, you get a buller ,except when you specily a stream encooing, in which case you get a String,. This wil be covereo later. You can create a Buller lrom an UTI8 string like this: var buf = new Buffer{`Hello World!`|, You can also create a buller lrom strings with other encooings, as long as you pass it as the secono argument: var buf = new Buffer{`8b76fde7l3ce`, `base64`|, Accepteo encooings are: "ascii", "utl8" ano "baseo!". or you can create a new empty buller with a specinc size: var buf = new Buffer{l024|, ano you can manipulate it: buf20 = 56, // set byte 20 to 56 You can also convert it to a UTI-8-encooeo string: var str = buf.toString{|, or into a string with an alternative encooing: var str = buf.toString{`base64`|, UTI-8 is the oelault encooing lor Nooe, so, in a general way, il you omit it as we oio on the buffer.toString{| call, UTI-8 will be assumeo. S||ce a buffer A buller can be sliceo into a smaller buller by using the appropriately nameo slice{| methoo like this: Hands-on Node Buffers 39/147 l. var buffer = new Buffer{`this is the string in my buffer`|, 2. var slice = buffer.slice{l0, 20|, Here we are slicing the original buller that has 31 bytes into a new buller that has 10 bytes equal to the 10th to 20th bytes on the original buller. Note that the slice lunction ooes not create new buller memory: it uses the original untoucheo buller unoerneath. Tip: Il you are alraio you will be wasting precious memory by keeping the olo buller arouno when slicing it, you can copy it into another like this: Copy a buffer You can copy a part ol a buller into another pre-allocateo buller like this: l. var buffer = new Buffer{`this is the string in my buffer`|, 2. var slice = new Buffer{l0|, 3. var targetStart = 0, 4. sourceStart = l0, 5. sourceEnd = 20, 6. buffer.coy{slice, targetStart, sourceStart, sourceEnd|, Here we are copying part ol buffer into slice, but only positions 10 through 20. Buffer Exerc|ses Exerc|se 1 Create an uninitializeo buller with 100 bytes length ano nll it with bytes with values starting lrom 0 to 99. Ano then print its contents. Exerc|se 2 Do what is askeo on the previous exercise ano then slice the buller with bytes ranging !0 to o0. Ano then print it. Exerc|se 3 Do what is askeo on exercise 1 ano then copy bytes ranging !0 to o0 into a new buller. Ano then print it. Hands-on Node Buffers 40/147 Hands-on Node Buffers 41/147 Event Em|tter On Nooe many objects can emit events. Ior instance, a TCF server can emit a 'connect' event every time a client connects. Or a nle stream request can emit a 'oata' event. .addL|stener You can listen lor these events by calling one ol these objects "aooListener" methoo, passing in a callback lunction. Ior instance, a nle ReaoStream can emit a "oata" event every time there is some oata available to reao. Insteao ol using the "aooListener" lunction, you can also use "on", which is exactly the same thing: l. var fs = require{`fs`|, // get the fs module 2. var readStream = fs.createReadStream{`/etc/asswd`|, 3. readStream.on{`data`, function{data| { 4. console.log{data|, 5. }|, 6. readStream.on{`end`, function{| { 7. console.log{`file ended`|, 8. }|, Here we are binoing to the readStream's "oata" ano "eno" events, passing in callback lunctions to hanole each ol these cases. When one ol these events happens, the reaoStream will call the callback lunction we pass in. You can either pass in an anonymous lunction as we are ooing here, or you can pass a lunction name lor a lunction available on the current scope, or even a variable containing a lunction. .once You may also want to listen lor an event exactly once. Ior instance, il you want to listen to the nrst connection on a server, you shoulo oo something like this: l. server.once{`connection`, function {stream| { 2. console.log{`Ah, we have our first user!`|, 3. }|, This works exactly like our "on" example, except that our callback lunction will be calleo at most once. It has the same ellect as the lollowing cooe: Hands-on Node Event Emitter 42/147 l. function connListener{stream| { 2. console.log{`Ah, we have our first user!`|, 3. server.removeListener{`connection`, connListener|, 4. } 5. server.on{`connection`, connListener|, Here we are using the removeListener, which also belongs to the EventEmitter pattern. It accepts the event name ano the lunction it shoulo remove. .removeA||L|steners Il you ever neeo to, you can also remove all listeners lor an event lrom an Event Emitter by simply calling server.removeAllListeners{`connection`|, Creat|ng an Event Em|tter Il you are interesteo on using this Event Emitter pattern - ano you shoulo - throughout your application, you can. You can create a pseuoo-class ano make it inherit lrom the EventEmitter like this: l. var EventEmitter = require{`events`|.EventEmitter, 2. util = require{`util`|, 3. 4. // Here is the MyClass constructor: 5. var MyClass = function{otionl, otion2| { 6. this.otionl = otionl, 7. this.otion2 = otion2, 8. } 9. l0. util.inherits{MyClass, EventEmitter|, util.inherits is setting up the prototype chain so that you get the EventEmitter prototype methoos available to your MyClass instances. This way instances ol MyClass can emit events: l. MyClass.rototye.someMethod = function{| { 2. this.emit{`custom event`, `some arguments`|, 3. } 4. Here we are emiting an event nameo "custom event", senoing also some oata ,"some arguments" in this case,, Hands-on Node Event Emitter 43/147 Now clients ol MyClass instances can listen to "custom events" events like this: l. var myInstance = new MyClass{l, 2|, 2. myInstance.on{`custom event`, function{| { 3. console.log{`got a custom event!`|, 4. }|, Tip: The Event Emitter is a nice way ol enlorcing the oecoupling ol interlaces, a soltware oesign technique that improves the inoepenoence lrom specinc interlaces, making your cooe more nexible. Event Em|tter Exerc|ses Exerc|se 1 Builo a pseuoo-class nameo "Ticker" that emits a "tick" event every 1 secono. Exerc|se 2 Builo a script that instantiates one Ticker ano bino to the "tick" event, printing "TICK" every time it gets one. Hands-on Node Event Emitter 44/147 T|mers Nooe implements the timers AFI also louno in web browsers. The original AFI is a bit quirky, but it hasn't been changeo lor the sake ol consistency. setT|meout setTimeout lets you scheoule an arbitrary lunction to be executeo in the luture. An example: l. var timeout = 2000, // 2 seconds 2. setTimeout{function{| { 3. console.log{`timed out!`|, 4. }, timeout|, 5. This cooe will register a lunction to be calleo when the timeout expires. Again, as in any place in Javascript, you can pass in an inline lunction, the name ol a lunction or a variable which value is a lunction. You can use setTimeout with a timeout value ol 0 so that the lunction you pass gets executeo some time alter the stack clears, but with no waiting. This can be useo to, lor instance scheoule a lunction that ooes not neeo to be executeo immeoiately. This was a trick sometimes useo on browser Javascript, but, as we will see, Nooe rocess.nextTick{| can be useo insteao ol this, ano it's more elncient. c|earT|meout setTimeout returns a timeout hanole that you can use to oisable it like this: l. var timeoutHandle = setTimeout{function{| { console.log{`yehaa!`|, }, l000|, 2. clearTimeout{timeoutHandle|, Here the timeout will never execute because we clear it right alter we set it. Another example: Source code in chaters/timers/timers_l.s Hands-on Node Timers 45/147 l. var timeoutA = setTimeout{function{| { 2. console.log{`timeout A`|, 3. }, 2000|, 4. 5. var timeoutB = setTimeout{function{| { 6. console.log{`timeout B`|, 7. clearTimeout{timeoutA|, 8. }, l000|, 9. Here we are starting two timers: one with 1 secono ,timeoutB, ano the other with 2 seconos ,timeoutA,. But timeoutB ,which nres nrst, unscheoules timeoutA on line 7, so timeoutA is never executes - ano the program exits right alter line 7 is executeo. setInterva| Set interval is similar to set timeout, but scheoules a given lunction to run every X seconos like this: Source code in chaters/timers/timers_2.s l. var eriod = l000, // l second 2. var interval = setInterval{function{| { 3. console.log{`tick`|, 4. }, eriod|, This will inoennitely keep the console logging "tick" unless you terminate Nooe. You can unscheoule an interval by calling: c|earInterva| clearInterval unscheoules a running interval ,previous scheouleo with setInterval,. l. var interval = setInterval{...|, 2. clearInterval{interval|, 3. Here we are using the setInterval return value storeo on the interval variable to unscheoule it on line 2. process.nextT|ck You can also scheoule a callback lunction to run on the next run ol the event loop. You can use it like this: Hands-on Node Timers 46/147 l. rocess.nextTick{function{| { 2. // this runs on the next event loo 3. console.log{`yay!`|, 4. }|, As we saw, this methoo is prelereo to setTimeout{fn, 0| because it is more elncient. Escap|ng the event |oop On each loop, the event loop executes the queueo IO events sequentially by calling the associateo callbacks. Il, on any ol the callbacks you take too long, the event loop won't be processing other penoing IO events meanwhile. This can leao to waiting customers or tasks. When executing something that may take too long, you can oelay the execution until the next event loop, so waiting events will be processeo meanwhile. It's like going to the back ol the line on a waiting line. To escape the current event loop you can use process.nextTick,, like this: l. rocess.nextTick{function{| { 2. // do something 3. }|, You can use this to oelay processing that is not necessary to oo immeoiately to the next event loop. Ior instance, you may neeo to remove a nle, but perhaps you oon't neeo to oo it belore replying to the client. So, you coulo oo something like this: l. stream.on{`data`, function{data| { 2. stream.end{`my resonse`|, 3. rocess.nextTick{function{| { 4. fs.unlink{`ath/to/file`|, 5. }| 6. }|, Hands-on Node Timers 47/147 A note on ta|| recurs|on Let's say you want to scheoule a lunction that ooes some IO - like parsing a log nle - to execute periooically, ano you want to guarantee that no two ol those lunctions are executing at the same time. The best way is not to use a setInterval, since you oon't have that guarantee. The interval will nre no matter il the lunction has nnisheo it's outy or not. Supposing there is an asynchronous lunction calleo "async" that perlorms some IO ano that gets a callback to be invokeo when nnisheo, ano you want to call it every secono: l. var interval = l000, // l second 2. setInterval{function{| { 3. async{function{| { 4. console.log{`async is done!`|, 5. }|, 6. }, interval|, Il any two async,, calls can't overlap, you are better oll using tail recursion like this: l. var interval = l000, // l second 2. {function schedule{| { 3. setTimeout{function{| { 4. async{function{| { 5. console.log{`async is done!`|, 6. schedule{|, 7. }|, 8. }, interval| 9. }|{|, Here we are oeclaring a lunction nameo scheoule ,line 2, ano we are invoking it immeoiately alter we are oeclaring it ,line 9,. This lunction scheoules another lunction to execute within one secono ,line 3 to 8,. This other lunction will then call async,, ,line !,, ano only when async is oone we scheoule a new one by calling scheoule,, again ,line o,, this time insioe the scheoule lunction. This way we can be sure that no two calls to async execute simultaneously in this context. The oillerence is that we probably won't have async calleo every secono ,unless async takes no time to execute,, but we will have it calleo 1 secono alter the last one nnisheo. Hands-on Node Timers 48/147 Low-|eve| |e-system Nooe has a nice streaming AFI lor oealing with nles in an abstract way, as il they were network streams, but sometimes you might neeo to go oown a level ano oeal with the nlesystem itsell. Iirst, a nice set ol utilities: fs.stat and fs.fstat You can query some meta-inlo on a nle ,or oir, by using fs.stat like this: l. var fs = require{`fs`|, 2. 3. fs.stat{`/etc/asswd`, function{err, stats| { 4. if {err| {console.log{err.message|, return, } 5. console.log{stats|, 6. //console.log{`this file is ` + stats.size + ` bytes long.`|, 7. }|, Il you print the stats object it will be something like: l. { dev: 23488l026, 2. ino: 24606, 3. mode: 33l88, 4. nlink: l, 5. uid: 0, 6. gid: 0, 7. rdev: 0, 8. size: 3667, 9. blksize: 4096, l0. blocks: 0, ll. atime: Thu, l7 Mar 20ll 09:l4:l2 GMT, l2. mtime: Tue, 23 Jun 2009 06:l9:47 GMT, l3. ctime: Fri, l4 Aug 2009 20:48:l5 GMT l4. } stats is a Stats instance, with which you can call: Hands-on Node Low-level file-system 49/147 l. stats.isFile{| 2. stats.isDirectory{| 3. stats.isBlockDevice{| 4. stats.isCharacterDevice{| 5. stats.isSymbolicLink{| 6. stats.isFIFO{| 7. stats.isSocket{| Il you have a plain nle oescriptor you can use fs.fstat{fileDescritor, callback| insteao. More about nle oescriptors later. Il you are using the low-level nlesystem AFI in Nooe, you will get nle oescriptors as a way to represent nles. These nle oescriptors are plain integer numbers that represent a nle in your Nooe process, much like in C FOSIX AFIs. Open a |e You can open a nle by using ls.open like this: l. var fs = require{`fs`|, 2. fs.oen{`/ath/to/file`, `r`, function{err, fd| { 3. // got fd 4. }|, The nrst argument to fs.oen is the nle path. The secono argument is the nags, which inoicate the mooe with which the nle is to be open. The nags can be 'r', 'r-', 'w', 'w-', 'a', or 'a-'. Here is the semantics ol each nag, taken lrom the lopen man page: r - Open text nle lor reaoing. The stream is positioneo at the beginning ol the nle. r+ - Open lor reaoing ano writing. The stream is positioneo at the beginning ol the nle. w - Truncate nle to zero length or create text nle lor writing. The stream is positioneo at the beginning ol the nle. w+ - Open lor reaoing ano writing. The nle is createo il it ooes not exist, otherwise it is truncateo. The stream is positioneo at the beginning ol the nle. a - Open lor writing. The nle is createo il it ooes not exist. The stream is positioneo at the eno ol the nle. Subsequent writes to the nle will always eno up at the then current eno ol nle. a+ - Open lor reaoing ano writing. The nle is createo il it ooes not exist. The stream is positioneo at the eno ol the nle. Subsequent writes to the nle will always eno up at the then current eno ol nle. On the callback lunction, you get a secono argument ,lo,, which is a nle oescriptor- nothing more than an integer that ioentines the open nle, which you can use like a hanoler to reao ano write lrom. Hands-on Node Low-level file-system 50/147 Read from a |e Once it's open, you can also reao lrom a nle like this: Source code in chaters/fs/read.s l. var fs = require{`fs`|, 2. fs.oen{`/var/log/system.log`, `r`, function{err, fd| { 3. if {err| { throw err } 4. var readBuffer = new Buffer{l024|, 5. bufferOffset = 0, 6. bufferLength = readBuffer.length, 7. filePosition = l00, 8. 9. fs.read{fd, readBuffer,bufferOffset, bufferLength, filePosition, l0. function{err, readBytes| { ll. if {err| { throw err, } l2. console.log{`ust read ` + readBytes + ` bytes`|, l3. if {readBytes > 0| { l4. console.log{readBuffer.slice{0, readBytes||, l5. } l6. }|, l7. }|, Here we are opening the nle, ano when it's openeo we are asking to reao a chunk ol 102! bytes lrom it, starting at position 100 ,line 9,. The last argument to the fs.read call is a callback lunction ,line 10, which will be invokeo when one ol the lollowing 3 happens: there is an error, something has been reao or nothing coulo be reao. On the nrst argument, this callback gets an error il there was an one, or null. On the secono argument ,readBytes, it gets the number ol bytes reao into the buller. Il the reao bytes is zero, the nle has reacheo the eno. Wr|te |nto a |e To write into a nle oescriptor you can use fs.write like this: Hands-on Node Low-level file-system 51/147 l. var fs = require{`fs`|, 2. 3. fs.oen{`/var/log/system.log`, `a`, function{err, fd| { 4. var writeBuffer = new Buffer{`writing this string`|, 5. bufferOffset = 0, 6. bufferLength = writeBuffer.length, 7. filePosition = null, 8. 9. fs.write{ l0. fd, ll. writeBuffer, l2. bufferOffset, l3. bufferLength, l4. filePosition, l5. function{err, written| { l6. if {err| { throw err, } l7. console.log{`wrote ` + written + ` bytes`|, l8. } l9. |, 20. }|, Here we are opening the nle in appeno-mooe ,'a', on line 3, ano then we are writing into it ,line 8,, passing in a buller with the oata we want written, an ollset insioe the buller where we want to start writing lrom, the length ol what we want to write, the nle position ano a callback. In this case we are passing in a nle position ol null, which is to say that he writes at the current nle position. Here we are also opening in appeno-mooe, so the nle cursor is positioneo at the eno ol the nle. C|ose Your |es On all these examples we oio not close the nles. This is because these are small simple examples oestineo to be run ano returneo. All open nles will be closeo once the process exists. In real applications you shoulo keep track ol those nle oescriptors ano eventually close them using fs.close{fd, callback| when no longer neeoeo. Hands-on Node Low-level file-system 52/147 Advanced T|p: carefu| when append|ng concurrent|y Il you are using these low-level nle-system lunctions to appeno into a nle, ano concurrent writes will be happening, opening it in appeno-mooe will not be enough to ensure there will be no overlap. Insteao, you shoulo keep track ol the last written position belore you write, ooing something like this: l. // Aender 2. var fs = require{`fs`|, 3. var startAender = function{fd, startPos| { 4. var os = startPos, 5. return { 6. aend: function{buffer, callback| { 7. var oldPos = os, 8. os += buffer.length, 9. fs.write{fd, buffer, 0, buffer.length, oldPos, callback|, l0. } ll. }, l2. }, l3. l4. Here we oeclare a lunction storeo on a variable nameo "startAppenoer". This lunction starts the appenoer state ,position ano nle oescriptor, ano then returns an object with an appeno lunction. Now let's oo a script that uses this Appenoer: l. // start aender 2. fs.oen{`/tm/test.txt`, `w`, function{err, fd| { 3. if {err| {throw err, } 4. var aender = startAender{fd, 0|, 5. aender.aend{new Buffer{`aend this!`|, function{err| { 6. console.log{`aended`|, 7. }|, 8. }|, Ano here we are using the appenoer to salely appeno into a nle. This lunction can then be invokeo to appeno, ano this appenoer will keep track ol the last position ,line ! on the Appenoer,, ano increments it accoroing to the buller length that was passeo in. Hands-on Node Low-level file-system 53/147 Actually, there is a problem with this cooe: fs.write{| may not write all the oata we askeo it to, so we neeo to oo something a little bit smarter here: Source code in chaters/fs/aender.s l. // Aender 2. var fs = require{`fs`|, 3. var startAender = function{fd, startPos| { 4. var os = startPos, 5. return { 6. aend: function{buffer, callback| { 7. var written = 0, 8. var oldPos = os, 9. os += buffer, l0. {function tryWriting{| { ll. if {written < buffer.length| { l2. fs.write{fd, buffer, written, buffer.length ~ written, oldPos + written, l3. function{err, bytesWritten| { l4. if {err| { callback{err|, return, } l5. written += bytesWritten, l6. tryWriting{|, l7. } l8. |, l9. } else { 20. // we have finished 2l. callback{null|, 22. } 23. }|{|, 24. } 25. } 26. }, Here we use a lunction nameo "tryWritting" that will try to write, call ls.write, calculate how many bytes have alreaoy been written ano call itsell il neeoeo. When it oetects it has nnisheo ,written ~~ buller.length, it calls callback to notily the caller, enoing the loop. Don't be lrustrateo il you oon't grasp this on the nrst time arouno. Give it some time to sink in ano come back here alter you have nnisheo reaoing this book. Also, the appenoing client is opening the nle with mooe "w", which truncates the nle, ano it's telling appenoer to start appenoing on position 0. This will overwrite the nle il it has content. So, a wizer version ol the appenoer client woulo be: Hands-on Node Low-level file-system 54/147 l. // start aender 2. fs.oen{`/tm/test.txt`, `a`, function{err, fd| { 3. if {err| {throw err, } 4. fs.fstat{fd, function{err, stats| { 5. if {err| {throw err, } 6. console.log{stats|, 7. var aender = startAender{fd, stats.size|, 8. aender.aend{new Buffer{`aend this!`|, function{err| { 9. console.log{`aended`|, l0. }|, ll. }| l2. }|, F||e-system Exerc|ses You can check out the solutions at the eno ol this book. Exerc|se 1 - get the s|ze of a |e Having a nle nameo a.txt, print the size ol that nles in bytes. Exerc|se 2 - read a chunk from a |e Having a nle nameo a.txt, print bytes 10 to 1!. Exerc|se 3 - read two chunks from a |e Having a nle nameo a.txt, print bytes to 9, ano when oone, reao bytes 10 to 1!. Exerc|se 4 - Overwr|te a |e Having a nle nameo a.txt, Overwrite it with the UTI8-encooeo string "ABCDEIGHIJLKLMNOFQRSTUVXYZ0123!o789abcoelghijklmnopqrstuvxyz". Exerc|se 5 - append to a |e Having a nle nameo a.txt, appeno utl8-encooeo string "abc" to nle a.txt. Hands-on Node Low-level file-system 55/147 Exerc|se 6 - change the content of a |e Having a nle nameo a.txt, change byte at pos 10 to the UTI8 value ol "7". Hands-on Node Low-level file-system 56/147 HTTP HTTP Server You can easily create an HTTF server in Nooe. Here is the lamous http server "Hello Worlo" example: Source in file: htt/htt_server_l.s l. var htt = require{`htt`|, 2. 3. var server = htt.createServer{|, 4. server.on{`request`, function{req, res| { 5. res.writeHead{200, {`Content~Tye`: `text/lain`}|, 6. res.write{`Hello World!`|, 7. res.end{|, 8. }|, 9. server.listen{4000|, l0. On line 1 we get the 'http' mooule, to which we call createServer,, ,line 3, to create an HTTF server. We then listen lor 'request' type events, passing in a callback lunction that gets two arguments: the request object ano the response object. We can then use the response object to write back to the client. On line we write a heaoer ,ContentType: textplain, ano the HTTF status 200 ,OK,. On line o we reply the string "Hello Worlo" ano on line 7 we terminate the request. On line 9 we bino the server to the port !000. So, il you run this script on nooe you can then point your browser to http:localhost:!000 ano you shoulo see the "Hello Worlo" string on it. This example can be shorteneo to: Source in file: htt/htt_server_2.s l. require{`htt`|.createServer{function{req, res| { 2. res.writeHead{200, {`Content~Tye`: `text/lain`}|, 3. res.end{`Hello World!`|, 4. }|.listen{4000|, Here we are giving up the intermeoiary variables lor storing the http mooule ,since we only neeo to call it once, ano the server ,since we only neeo to make it listen on port !000,. Also, as a shortcut, the http.createServer lunction accepts a callback lunction that will be invokeo on every request. There is one last shortcut here: the response.eno lunction can accept a string or buller which it will Hands-on Node HTTP 57/147 seno to the client belore enoing the request. The http.ServerRequest object When listening lor "request" events, the callback gets one ol these objects as the nrst argument. This object contains: req.ur| The URL ol the request, as a string. It ooes not contain the schema, hostname or port, but it contains everything alter that. You can try this to analyze the url: Source in file: htt/htt_server_3.s l. require{`htt`|.createServer{function{req, res| { 2. res.writeHead{200, {`Content~Tye`: `text/lain`}|, 3. res.end{req.url|, 4. }|.listen{4000|, ano connect to port !000 using a browser. Change the URL to see how it behaves. req.method This contains the HTTF methoo useo on the request. It can be, lor example, 'GET', 'FOST', 'DELETE' or any other one. req.headers This contains an object with a property lor every HTTF heaoer on the request. To analyze it you can run this server: Source in file: htt/htt_server_4.s l. var util = require{`util`|, 2. 3. require{`htt`|.createServer{function{req, res| { 4. res.writeHead{200, {`Content~Tye`: `text/lain`}|, 5. res.end{util.insect{req.headers||, 6. }|.listen{4000|, ano connect your browser to port !000 to inspect the heaoers ol your request. Here we are using util.insect{|, an utility lunction that can be useo to analyze the properties ol any object. Hands-on Node HTTP 58/147 req.headers properties are on lower-case. Ior instance, il the browser sent a "Cache- Control: max-age: 0" heaoer, req.headers will have a property nameo "cache-control" with the value "max-age: 0" ,this last one is untoucheo,. The http.ServerResponse object The response object ,the secono argument lor the "request" event callback lunction, is useo to reply to the client. With it you can: Wr|te a header You can use res.writeHead{status, headers|, where heaoers is an object that contains a property lor every heaoer you want to seno. An example: Source in file: htt/htt_server_5.s l. var util = require{`util`|, 2. 3. require{`htt`|.createServer{function{req, res| { 4. res.writeHead{200, { 5. `Content~Tye`: `text/lain`, 6. `Cache~Control`: `max~age=3600` 7. }|, 8. res.end{`Hello World!`|, 9. }|.listen{4000|, On this example we set 2 heaoers: one with "Content-Type: textplain" ano another with "Cache- Control: max-age~3o00". Il you save the above source cooe into http_server_.js ano run it with: $ node htt_server_5.s You can query it by using your browser or using a commano-line HTTF client like curl: l. $ curl ~i htt://localhost:4000 2. HTTP/l.l 200 OK 3. Content~Tye: text/lain 4. Cache~Control: max~age=3600 5. Connection: kee~alive 6. Transfer~Encoding: chunked 7. 8. Hello World! Change or set a header You can change a heaoer you alreaoy set or set a new one by using Hands-on Node HTTP 59/147 res.setHeader{name, value|, This will only work il you haven't alreaoy sent a piece ol the booy by using res.write,,. Remove a header You can remove a heaoer you have alreaoy set by calling: res.removeHeader{name, value|, Again, this will only work il you haven't alreaoy sent a piece ol the booy by using res.write,,. Wr|te a p|ece of the response body You can write a string: res.write{`Hello`|, or a an existing buller: l. var buf = new Buffer{`Hello World`|, 2. buf0 = 45, 3. res.write{buffer|, This methoo can, as expecteo, be useo to reply oynamically generateo strings or binary nle. Replying with binary oata will be covereo later. HTTP C||ent You can issue http requests using the "http" mooule. Nooe is specincally oesigneo to be a server, but it can itsell call other external services ano act as a "glue" service. Or you can simply use it to run a simple http client script like this one: http.get(| Source in file: htt/htt_client_l.s Hands-on Node HTTP 60/147 This example uses http.get to make an HTTF GET request to the url http:www.google.com:80inoex.html. You can try it by saving it to a nle nameo http_client_1.js ano running: l. $ node htt_client_l.s 2. 3. got resonse: 302 4. http.request(| Using htt.request you can make any type ol HTTF request: htt.request{otions, callback|, The options are: host: A oomain name or IF aooress ol the server to issue the request to. ort: Fort ol remote server. method: A string specilying the HTTF request methoo. Fossible values: 'GET' ,oelault,, 'FOST', 'FUT', ano 'DELETE'. ath: Request path. Shoulo incluoe query string ano lragments il any. E.G. 'inoex.html?page~12' headers: An object containing request heaoers. The lollowing methoo makes it easy to seno booy values ,like when you are uploaoing a nle or posting a lorm,: Source in file: htt/htt_client_2.s Hands-on Node HTTP 61/147 l. var otions = { 2. host: `www.google.com`, 3. ort: 80, 4. ath: `/uload`, 5. method: `POST` 6. }, 7. 8. var req = require{`htt`|.request{otions, function{res| { 9. console.log{`STATUS: ` + res.statusCode|, l0. console.log{`HEADERS: ` + JSON.stringify{res.headers||, ll. res.setEncoding{`utf8`|, l2. res.on{`data`, function {chunk| { l3. console.log{`BODY: ` + chunk|, l4. }|, l5. }|, l6. l7. // write data to request body l8. req.write{"data\n"|, l9. req.write{"data\n"|, 20. req.end{|, On lines 18 ano 19 we are writing the HTTF request booy oata ,two lines with the "oata" string, ano on line 20 we are enoing the request. Only then the server replies ano the response callback gets activateo ,line 8,. Then we wait lor the response. When it comes, we get a "response" event, which we are listening to on the callback lunction that starts on line 8. By then we only have the HTTF status ano heaoers reaoy, which we print ,lines 9 no 10,. Then we bino to "oata" events ,line 12,. These happen when we get a chunk ol the response booy oata ,line 12,. This mechanism can be useo to stream oata lrom a server. As long as the server keeps senoing booy chunks, we keep receiving them. HTTP Exerc|ses You can checkout the solutions at the eno ol this book. Exerc|se 1 Make an HTTF server that serves nles. The nle path is provioeo in the URL like this: http:localhost:!000pathtomynle.txt Exerc|se 2 Hands-on Node HTTP 62/147 Make an HTTF server that outputs plain text with 100 new-line separateo unix timestamps every secono. Exerc|se 3 Make an HTTF server that saves the request booy into a nle. Exerc|se 4 Make a script that accepts a nle name as nrst commano line argument ano uploaos this nle into the server built on the previous exercise. Hands-on Node HTTP 63/147
Michael Hartl - Learn Enough JavaScript To Be Dangerous - Write Programs, Publish Packages, and Develop Interactive Websites With JavaScript (2022, Addison-Wesley Professional) - Libgen - Li