Lets learn node the node way! 🐪 🙈 🐣
It's uncertain where this trek will take us, but we will try to strike a balance between instructional material, history, and philosophies on node, javascript, software, programming, engineering, communication, and tooling.
We may also cover a few other useful places to learn cool tools and skills like:
- html
- css
- git
- irc
- linux/unix (aka *nix)
- app deployment
- people skills
- node-learnbook
- Getting started.
- What is javascript?
- What is node?
- How to get node
- How to know node
- Callbacks visualized
- Javascript the hard parts
- ... [WIP]
npm
stuffs- Semver
- Node in the browser?
- Event Emitters
- 👻👻 Inheritance, Composition, and the Prototype Chain
- Write your tests, eat your greens
- Whats the deal with Streams?
- Node.js and io.js Anthropology
- Tools to write Node
- Spellcheck for Javascript
- All the badges
- Hex Stickers
- An adventure of your own
- JS is the worlds polyglot assembly language now
- ES6 and Beyond
- Frameworks
- Haters gonna hate
- html + css
- Community
- Mastering git
- Electron?
- Cool Cats writing Cool Modules
- People to watch
- Link Dump
- A computer 🐫 (hopefully running a flavor of unix)
- OS X will be easiest to use
- Linux and windows are a bit more difficult
- Internet and a browser 🌐
- some time and interest 🙇
We will be acquiring items and tools along the way.
Getting through all the material is going to take time. Visit the places that seem most interesting, and don't get stuck on something you find boring or overwhelming. Programming is more fun and effective as a hobby and you will learn more if you treat it that way.
Do take breaks and try to build something interesting. Even if you fail, you will learn something. Publishing your experiments feels good and creates breadcrumbs for others to learn from.
Don't go at it alone. Be open to companions along the way. Offer collaborative opportunity to others who may be seeking similar experience, and try to be willing to offer help and contributions to others when it seems appropriate and useful to both parties.
Don't live near anyone?
There is a vibrant and active community that is on-line at all hours of the day so you can remain isolated but still be connected with thousands of people. Skip ahead to #community to find possible avenues of interest, and places where you might meet other like minded people.
Before we get started, we need to be somewhat prepared to face what lies ahead. In order to communicate with the local populous you need to learn how to speak and think javascript.
If your a cat, like to have fun or learn like a cat, this will teach you the basics of javascript.
If you tend to be a bit more serious, Codecademy's Javascript is also a good place to start. Don't feel bad if you get bored and don't finish. It's picking up the syntax that counts.
We will revisit nodeschool.io in the near future, but for now the javascripting
nodeschool adventure is a good place to start learning javascript. It may be a bit steep for absolute beginners. Read through the get going
section to get up and running.
For those who know their way around npm
already:
npm install -g javascripting
Javascript is the programming language that your web browser comes with, but these days its becoming a lot more. Its pretty okay. Its not the best language and has a lot of warts, but it gets a lot right, and you don't really have a choice about using it or not (although this is changing, for better or for worse). You shouldn't skip learning javascript though.
Douglas Crockford gave a good presentation that effectively answers the question: "What is Javascript?" in his 2012 "Javascript: Your New Overlord" presentation:
You should watch it!
Also while we are on the topic, Javascript is the real world implementation of ECMAScript (abbreviated ES*, where * is the spec version), which is just a language specification e.g. a document that describes how it should work.
The current trend right now is to call the specs ES2015 (for ES6), ES2016 (for ES7) etc as a way to help promote quicker releases and access to newer language features.
More language features means more to learn for everyone. The more everyone has to learn, the more intimidating starting out can be.
ES5 is a simple yet expressive language that we have right now. Adding new features isn't guaranteed to improve the language (but there are quite a few welcomed features and data structures). Be open about what to learn, and picky about what you choose to use.
You don't need to read these right now, but here are the last couple specs:
- ES5
- ES6 Draft (ES2015?)
- ES7 (ES2016?)
- ES* Compatibility Table
- http://www.bloomberg.com/graphics/2015-paul-ford-what-is-code/
Node was created by Ryan Dahl. He has since pulled a Mark Pilgrim and HTTP 410'd himself from the INTERNET, but occasionally will post interesting undocumented code projects to his github or post to the node mailing list.
The best way to understand what node is to listen to Ryan describe it himself.
Ryan Dahl: Original Node.js presentation
Ryan Dahl: Original Node.js presentation Slide Deck (pdf)
Ryan Dahl Talk - NodeConf Theatre 2012.ogv
Ryan Dahl Talk NodeConf Theatre 2012 Slides (pdf)
- https://news.ycombinator.com/item?id=4892174
- https://news.ycombinator.com/item?id=7064470
- http://shitryandahlsays.tumblr.com/
- https://www.youtube.com/watch?v=SAc0vQCC6UQ
- https://github.com/ry/v8worker
- http://siliconangle.com/blog/2012/01/31/how-a-vacuum-cleaner-salesman-became-the-new-king-of-node-js/
- http://www.quora.com/What-is-happening-to-Joyent-and-how-does-it-affect-NodeJS
- https://gist.github.com/cookrn/4015437
Some links discussing issues related to what node tries to solve and other general readings. Caution some of these are overly advanced... maybe revisit these if the subject interests you.
- About Node.js®
- A little holiday present: 10,000 reqs/sec with Nginx!
- The C10K problem
- Wikipedia: Event Loops
- Wikipedia: Asynchronous I/O
There are lots of ways to install node. Lets visit some of the better ways of this contentious topic.
Node needs the following two things to really work effectively:
A build toolchain is a complicated set of programs that let you build software from source. Usually building software from source means building an executable binary from raw C and C++ source files and libraries. Node needs a toolchain in order to build native addons (e.g. programs written in c that node modules sometimes needs to communicate with).
Examples
This is node iteself! It provides things like the node
command/runtime, and usually comes with npm
bundled with it.
Node requires python2 to build native addons.
A package manager is optional, but having one available and set up is extremely helpful. Package managers install software for you, automatically and unattended. They go out and download the programs you want, as well as the other programs required to run them, then take all the necessary steps to put them into the right place where you can use said programs. Some people can't be bothered to use a package manager because you need to learn a little bit about how they work, but you will become a more powerful developer if you learn to use a traditional package manager.
Examples
The only prerequisite to installing node is that you have a copy of Xcode. Its free and the only place to get it is from App store (boooo). This fills the toolchain requirement on the OS X side of things.
OSX Terminal.app is pretty great for everything you need to do (although historically it used to suck). Just use OSX's terminal unless you have a reason not to.
There are two great options to install node
on OSX: Homebrew and the Official Installer.
Homebrew is a lightweight package manager for OS X. Homebrew:
- downloads and installs Unix CLI programs from source code
- keeps track of which programs you installed and at what version
- updates your programs when updates are available
- download non-npm programs and utilities
Until npm has packages for all external dependencies, having brew
installed can be really helpful.
s
Visit the Homebrew website for the latest instructions on how get
brew
installed.
Once you have homebrew installed installing node is as easy as running:
$ brew install node
To get new releases of node in the future download run:
$ brew update
$ brew upgrade node
Easy.
A quick brew 101
so that you know what you just did:
/usr/local
is a special unix folder where 'userspace' (e.g. not managed by the OS)programs can be safely installed. Since homebrew was installed by the user by hand, it is a userspace program.- Homebrew turns
/usr/local
into a git repo brew
uses "Formula" written as simple ruby scripts to download, build and install programs to/usr/local/Cellar
- The active version of a program installed by
brew
is symlinked to/usrlocal/{binlib,...}
brew update
updates the/usr/local
repo so that you have the latest 'Formula' available.- All Formula can be built from source, but most have precompiled 'Bottles' (a.k.a binaries) to save time and battery power.
- Old versions of programs can be installed by going back into the git history.
A few more links discussing the merits of homebrew:
Easy. You go to the node website, you [download the .pkg](https://nodejs.org/download, and install it. It installs to the same location that homebrew installs to: usrlocal/bin
.
Running Linux? (:+1: btw)
Don't just reach for your systems package manager!
Linux distributions almost universally ship painfully dated versions of node and npm (unless you are running something like Arch Linux), and install them in ways that make them a total pain to use for the sake of 'stability'.
This sucks.
Luckily there is a comprehensive resource on how to add software channels that have updated versions of node to common package mangers:
Installing-Node.js-via-package-manager
Basically it boils down to this:
Your linux distribution will dictate which package manager you use.
Don't sudo with npm! (even -g). Set up npm
so that it works without sudo:
Setting npm
to install to /usr/local
is the correct place for npm
to install global library binarys to. It is in all user path's by default, and is the conventional well known location to look in.
If you are stuck in userland on a shared system and you don't have permissions to modify /usr/local
, you can configure npm
to install -g
libs right into your home directory and add the resulting folder to your $PATH. Only do this if you don't have access to /usr/local
.
Windows is very important. Just like PHP -- Ryan Dahl
On windows, pretty much stick with the official installer:
There are a few package managers on windows which you are free to explore on your own time:
Windows support for node is pretty good. Windows support for most npm modules is pretty bad.
You will also need to install a free version of Visual Studio for building modules that have native addons:
as well as python2 installed:
For up to date windows caveats, see Microsoft/nodejs-guidelines
Now that we have seen a bit about the history and motivations behind node, lets actually visit the necessary materials to actually understand node.
Written by Max Ogden,
An image of Max holding a bag of fish for some reason. Probably to lure cats. Image by substack
maxogden/art-of-node gives a thorough explanation of how node works, how to write node flavored javascript, callbacks and async programming, writing and using modules and how to be apart of the node/js community and do your best. Its full of insight and clear reasoning, but when you are new to js and/or node, all of the subtleties can fly by pretty quickly. It's a short and easily digestible read, that you should probably read through a few times.
Written by a group of early prolific node.js module developers, this book is totally great and somewhat flawed, and a bit dated at this point as well. It has a great set of examples though, and it's "hello world" app is the well-known socket.io chat server.
It covers the basics of all the well known Visionmedia modules (e.g. express, mocha, connect, jade, ejs etc...). You will learn to use all of these early node tools, including core node modules, async programming, testing, templating, CLI programs, and even node clustering from this book through the various included projects and examples.
Despite the fact that many of the modules discussed have changed their APIs, this is still a great reference and useful for learning node.
Unfortunately, this great collection of knowledge is tombed away in a book (we should change that though). Get the book if you can.
What this book does well:
- Introduction to node, events async programming and flow control
- socket.io examples
- npm modules: publishing and consuming
- Module resolution and loading (a.k.a. require('foo'))
- http
- ORMs and Databases
- Middleware and Connect
- Express photo hosting and microblogging app
- Testing (mostly Mocha)
- EJS and Jade templating
- Node application deployment
- System Calls
- Raw TCP/IP
- CLI Tools
Focus on the first and last 1/3 of the book, and don't sweat the middle 1/3rd (express/connect).
Nodeschool is a free resource that offers lessons and tutorials on tons of topics, mostly relating to node and js. The key is that the lessons are written for node
and you install them with npm
(usually).
They were designed to be taught at community events around the world, but you can learn on your own at home if there isn't an event going on nearby.
Start with learnyounode which can be installed with
npm i -g learnyounode
(No, not learnyouhaskell, that lives on a different planet)
After this, brush up on your git
in git-it
:
npm i -g git-it
Git is a super important tool to know, use and love. Its like a checkpoint system in a video game but for text files on your computer. Check all of your projects into git, and throw them up on github. It's a great way to publish your work, and keeps a backup of everything you do!
Finally, wrap your head around npm
npm i -g how-to-npm
npm
is node's package manager. Its like pip
, gem
or cabal
for javascript.
TODO: Lets get a bunch of super simple examples of callbacks and work our way up to understanding what runs async and what doesn't all the way up to process.nexTick()
.
Callbacks are confusing at first, because you are writing functions that accept variables that seemingly come out of nowhere.
These variables that your callback accepts come from the internals of the function accepting the callback function!
Before we get into it, remember:
- Async functions return immediately
- If your code needs the results of an async function, that code needs to live in the scope of that async function's callback.
- Code that comes after an async function must complete before the async function calls its callback function. This the "Run to completion" guarantee.
- Functions that take a callback must be async or not async. Don't write a function that takes a callback that does not perform async work as this will break the "Run to completion" guarantee.
- You can't
return
values from an async callback like you might think.
Lets take a look:
-
First, we define
fs.readFile
. Typically you import this as a module (e.g.var fs = require('fs')
), but for this example we want get an idea of what is going on internally. The code to actually read files is omitted, but at the end of the function we see that thecallback
argument is invoked as a function (e.g.callback(err, data)
where theerr
anddata
variables would be defined inside the function somewhere when reading the file.) -
Next we invoke
fs.readFile
passing in a string containing the path to the file we wish to read, encoding options, and a callback function that acceptserr
anddata
arguments to match the arguments that thecallback
placeholder defines in step 1. The file is read and the placeholder callback is replaced with our callback function that we passed as an argument. -
After
fs.readFile
is done trying to read the file, it calls our callback function, passing to iterr
anddata
as arguments and our callback function then begins execution. If there was no error when reading the file,err
will beundefined
/falsy so we can easily test for errors and handle them, or pass them along.
Lets clean it up a little.
We replace the anonymous function with a named function logger
that we pass as the callback. It's nearly the exact same thing as the first example, except now the function can be implemented independently from where we invoke fs.readFile.
Ok, so it's not always practical to go digging around the internals of a module to locate the callback signature (e.g. the arguments) that it expects of your callback. This is where the module's README.md
comes in.
How did we know fs.readFile
expects a callback with (err, data)
arguments? We look it up!
Modules should come with a README.md
with similar documentation. If it doesn't have this, maybe look around for one that does.
We can write our own callback functions just as easily.
Here we write an async function that wraps the fs.readFile async function.
We write a function that accepts a callback as the last argument. We do some asynchronous work inside of it, modify the output and then pass it into our callback.
Easy!
Well, in theory. Callbacks take a bit of practice, but hopefully we can visualize callback flow and how variables and functions are passed around.
You can read more about callbacks, but the best way to learn how to use them is to read and write lots of them! While your doing that, peruse through these lovely links.
- maxogden/art-of-node#callbacks
- callbackhell.com
- Why callback hell can be your friend.
- Node In Action: 3.2 Asynchronous Programming Techniques p.46
- Eloquent Javascript: Http callbacks
- https://blog.domenic.me/youre-missing-the-point-of-promises/
- http://www.slideshare.net/domenicdenicola/callbacks-promises-and-coroutines-oh-my-the-evolution-of-asynchronicity-in-javascript
- https://github.com/petkaantonov/bluebird#what-are-promises-and-why-should-i-use-them
- https://spion.github.io/posts/why-i-am-switching-to-promises.html
Javascript is easy to learn, but hard to master. It's a flawed language with lots of subtle pitfalls to be navigated. These materials will help you master the more subtle and difficult aspects of the language.
Nodeschool.io: Tie it all together
If you haven't already, finish up the core nodeschool workshops. The last two are the most conceptually difficult:
scope-chains-closures
teaches you all about scopes and closures!
npm i -g scope-chains-closures
stream-adventure
teaches you about streams. This is a good place to start, but will leave you with a lot of questions.
npm i -g stream-adventure
REQUIRED READING
Effecitve Javascript is an excellent book focusing only on Javascritpt the Language. It has answers and clarifications for the more confusing aspects of the languages, explains the prototype chain in a clear way, and covers the more advanced aspects of JS. The only downside is the cost of the book. A+
This is a free e-book (paper version is available too). It seems to reside somewhere between Effective Javascript and Node.JS in Action with an environment agnostic approach to writing javascript. It has a lot of projects in it that run closer to your more traditional programming textbook.
JSTGP is one of Douglas Crockfords claim to fame (he also wrote down the JSON spec). It's pretty old at this point, and a difficult, dense and terse read. But its still really good, and has one of the better explanations about the different styles of object composition and inheritance (AKA Object Oriented Programming... or something kind of like it):
- Pseudoclassical Inheritance
- Prototypal Inheritance
- Functional Inheritance
- Talking about the bad and awful parts of js is straightforward and interesting.
- Setting the expectation of critical understanding and critique of different aspects of Javascript.
- The train track diagrams, while correct, literally don't help you think or understand how to write better JS. You get as much out of it as a you do completing a maze.
- The regex stuff might confirm your understanding, but don't try to learn regex from this book.
- Its discussion on callbacks. It doesn't even give one good example! It was written before JS was widely known for its ability to perform cooperative multitasking.
You will no doubt have questions about some of the suggestions in the book. It was written as ES5 was still not widely available. Quite a few of the polyfills noted in the book are actually widely available functions and methods... so double check MDN to see if its just a built in function now.
npm
is a package manager. If you have something (anything!) that could be apart of a larger project, you should version it and put it on npm
. Its the package manager for javascript, and is becoming the package manager for HTML and CSS components.
tl;dr npm
helps you download dependencies and publish reusable pieces of code/assets/software.
In addition to the how-to-npm nodeschool adventure, npm
has a great getting started page:
But here are the short notes version:
$ mkdir newproj ; cd newproj
$ git init
$ npm init
As we develop and use modules, we save them with:
$ npm i request --save
# e.g. install the request module and save it to package.json
When you are finally ready to put it out into the world
$ npm publish
If you clone in a project with a package.json
, typically this is where you start:
$ git clone [email protected] ; cd project
$ npm i
# i is short for install
# `npm install` would also work
This is downloads the dependencies of the project into a folder called node_modules
. The node_modules
folder should be added to your .gitignore
file.
A good next step is to test the project:
$ npm test
# this tests the module
Tests help ensure that others can help make changes and contributions the the module. There are no automated module interface testing tools, so project specific tests are the best thing we have right now.
Often you can start the module by running.
$ npm start
Finally, we can inspect other actions that project author added to the package.json
scripts
field:
$ npm run
This lists of other actions we can take on the module.
npm
is arguably more interesting than node itself:
node
is a tiny V8 javascript runtime engine with bindings to a set of high performance asynchronous C
libraries.
npm
is a system for authoring, packaging, and consuming reusable bits of code and assets, AND the worlds largest open source software repository:
It's analogous to a lego machine that produces unlimited copies of whatever kind of lego you can think of.
I want programming computers to be like coloring with crayons and playing with duplo blocks. --Ryan Dahl
npm
works better when using it when modules are small, focused bits of code that solve one problem well (instead of kitchen-sink, do-all modules).
npm
does a few things differently:
When you $ npm install
a package with npm
, it installs to the node_modules
folder specific to your project.
In nearly every other programming language, the package manager installs to a system wide (or user wide) dependency folder that all projects use. This runs into all sorts of permission errors and complexities that make it difficult to write software. Local-by-default node_modules
solve:
- Permission errors when users without admin access try to install project dependencies.
- Inconsistent and opaque module loading directories.
- Cognitive disconnect and confusion of where your modules are loading from.
- Eliminates the need for
$ENV
variables.
System wide dependency repositories are Global Variables. Nearly every language has systematic hacks to get around this issue like bundler.io and virtualenv. None of these actually solve the problem that local-by-default node_module
solves.
This is an important concept. Read more about it here:
- art-of-node: How
require
works - Folder Structures Used by npm
- Node API Docs:#modules_loading_from_node_modules_folders
- Node.js In Action: (p.43) Reusing modules in the node_modules folder
In most languages, if you depend on dependency [email protected]
and [email protected]
, and [email protected]
depends on [email protected]
, you are in dependency hell. You are unable to satisfy your dependency tree in a way that functions.
Node and npm
allow for nested dependencies. That means, your app gets [email protected]
and [email protected]
gets [email protected]
.
How does this magic work? See:
Nested dependencies solve the following issues:
- Dependency hell: Every dependency gets the dependencies it wants at the version it wants.
- Allows you to update your dependencies without worrying about breaking consumers of your module.
- Allows dependencies to update their dependencies without worry of breaking your module.
- Breaks undefined ties to global version state.
Nested dependencies introduce considerable complexity, and work is ongoing to improve its reliability. See what npm@3
is doing to improve this: [email protected] cangelog.
Many languages discourage the publishing of reusable code through various roadblocks (but primarily cultural ones).
npm
encourages publishing as much as possible. Publishing a module is a single command, and mostly automated.
This is unprecedented.
As a result, there are tons of bad modules. There are lots of good ones too. This solves the following issues:
- Community complacency with mediocre, monolithic packages. Everyone feels free to make something better, even if it sort of already exists.
- Stale standard libraries with lots of roadblocks in the way of improvements. Javascript doesn't have a standard library, and doesn't need one.
- Over designed and complicated do-all module APIs. The simplest and best modules usually win eventually!
- Denies Architecture Astronauts of Air, as they are side-stepped by multiple working modules during their never ending design debates.
- Lack of of solution diversity for different problem domains. Usually there are 3 - 100 different modules to choose from.
- Reduces unwarranted individual influence over language features and community culture to a minor degree. This is still a major cultural factor, but an open module system helps reduce this a little.
The package.json
is the source of truth about your module. It describes critical aspects of the module, like the name, version, license and entry point into the program.
It is worth reading the npm
docs page about this file in its entirety:
Specifics of npm's package.json handling
Here are some keys of interest:
main
: this is the name of the entry point of your application. When requiring the module, you get whatever this fileexports
ormodule.exports
. When reading a modules source code, this is the file you look at first.bin
: sometimes modules will include a executable bin. These get specified here. These end up in the./node_modules/.bin
folder and are available to the commands defined in thescripts
field. They also get installed to your$PATH
if you installed with the-g
flag.scripts
: this is where you define scripts. You should always include the following scripts in your module (if appropriate):test
: the command used to test your modulestart
: the command to run your module
dependencies
: these list off modules and their version ranges that your module needs in order to work.devDependencies
: these are modules needed to test, build and otherwise develop your module. Dependencies that are not required at runtime should live here. This includes all utility programs, test runners, task runners and build scripts.license
: This is the SPDX license identifier for the module.npm
complains if you leave this out.
AKA, how to navigate the world of javascript development tools without going crazy.
There are two factors to this problem.
-
There are tons of new javascript development tools to choose from with cool looking logos that get people really excited.
-
It isn't totally obvious what the best way to install and use these tools are.
- Install and save them as
devDependencies
in your package.json (e.g.npm i grunt --save-dev
) - Nail down the tool's project specific use in the
scripts
field. - Install the accompanying CLI with -g only when you feel the need to have it available system wide, and don't ever ask other developers to install a global tool for project specific use cases.
If a module comes with a bin
, that ends up in the .\node_modules\.bin
folder. .\node_modules\.bin
shouldn't be in your $PATH. When npm
runs a command out of the .package.json
scripts
field, it supplements the search path with .\node_modules\.bin
so that they are available for use from that interface.
This can be referred to as node_modules\.bin
PATH hoisting.
By hiding your toolchain behind a common interface, you shield yourself and other developers from these boring, toolchain details. Nobody actually cares what tools you used when they are looking to make a fix or change to your module, and you are not doing anyone any favors by promoting your favorite tools in this context by asking people to install a -g
tool to work on your module.
Read this article to learn more:
Grunt, Gulp, Broccoli --nodejsreactions.tumblr.com
Not every project needs a full task runner. Single purpose node utilities piped together with bash is a great way to get things done and should be considered.
Sometimes projects benefit from a task runner. In these projects, use a task runner.
When you decide you need a custom dev Tool, follow this general design process:
- Write your tool as a node module that can be tested and consumed by other node modules, in a generic and independent way.
- Write a CLI interface that consumes the library, and bundle it with the library (bonus points if you write it to accept
stdin
so that it can be piped together with other tools.) - Finally, write a separate, ecosystem specific module that requires your generic library and provides the correct interface to grunt/gulp etc.
Step 3 is often optional. Write these interface with task runners that you care about, and support contributors who want to write an task-runner specific interface for your module.
Global and environmental dependencies are an anti-pattern because it adds endless complexity to the process of writing and deploying applications.
The npm
community has some really big ideas about what npm
can and could do better. It has often been the tradition that you write a program, that talks to a web server like apache through CGI and also connects to a database that is assumed running.
These are all massive assumptions on your applications part.
Modern day ops teams have seen the value in codifying all of this server and service state using things like chef and puppet. Automating servers to achieve the correct state so that your application can run is great, but your app is still getting bundled without vital organs to make it work.
Utopia npm
is the idea that, with a working copy of npm
and a project repository with a package.json
, you can deploy your application by simply running the module's contract:
$ npm install ; npm test ; npm start
For example, instead of running a DB as a server, LevelDB is consumed no differently than any other library in your application:
- levelup
- leveldb-handbook (incomplete 😢)
Running a big, well-known database service has an ineffable heaviness to it: they bind to a custom port and the service must already be running before your program will work. Worse, if you install the database from a package manager like apt-get, the database will be auto-daemonized and put into your init scripts. Good luck hunting down where it decided to put the configuration file. -- substack: leveldb-handbook
Instead of downloading source dependencies and building them, they are packaged as pre-built binaries:
These are just some examples of projects that facilitate packaging non-js assets as npm
dependencies. This is an important design principle to keep in mind when building apps and modules. The more you can package into npm
, the easier it will be to use your app or module.
While the above examples are great, practical examples of patterns we can use today, these ideas are much larger than just versioning everything with npm
.
Semver is the versioning scheme of the node
community. Its not perfect, but it works mostly okay:
major.minor.patch
but you can also think about it like this
breaking.feature.bugfix
You start at version 1.0.0
, but sometimes people start modules at 0.0.1
to indicate 🚧experimental🚧 modules.
These links explain it pretty well.
In your package.json
, you can specify your dependencies using semver ranges.
The default range is:
^version "Compatible with version" See semver(7)
This range (in theory), should get you module patches that improve or fix bugs, and don't "break" the module interface in a range. In practice, sometimes they break anyway. This is why tests are critical.
Tools exist to help facilitate this patching process:
- next-update: run your tests against available updates without touching the current dep versions.
- npm.click: inspect a
package.json
for outdated packages. - David. DM: automatically fetches package versions status from a github url.
- Semver Calculator
This is an area that needs improvement and automation tools.
Still pulling together the primordial ooze below.
In node, you have access to the require
keyword that lets you load modules out of your ethereal node_modules
folder. This lets you write your programs in tiny, reusable modules of code.
Browserify is a program that lets you write javascript programs using the require
system. Instead of running the resulting programs in node
, browserify peforms a "build-step" and outputs a bundle that you serve up for a browser to run.
- Browserify-handbook
- http://wzrd.in/
- https://github.com/thlorenz/browserify-shim#multi-shim-example-including-dependencies
Browserify's scope is fairly limited compared to traditional front-end frameworks, as a result there are quite a few other tools that do similar things plus a whole lot more. Don't fall for it! YNGNI! Except when you do.
A module isn't complete without tests. Modules are fairly small, so 100% coverage is generally a modest goal to reach.
Tests force to you clearly define your module's interface, and allow for internal improvements to be made while ensuring external consumers don't break.
Testing your module will a mix of the following types of tools:
- Testing harness: some set of functions that keep track of how many tests are run and if they pass or fail.
- TAP Reporter: TAP is machine friendly. TAP reporters are machines that make your TAP output human friendly.
- Assertion library: this provides set of functions that are used to compare what actually happened to what you want to happen. Sometimes this comes along with the testing harness.
- Linter: This tool will perform static analysis on your code and point out mistakes, syntax errors, code smells or style issues.
- Formatter: Formats code so that its is formatted consistently.
- Continuous Integration (CI): You run your tests as you develop the module, but you can also have free services test your code for you.
- Coverage: Running your coverage tools along with your tests lets you keep track of which portion of your code is tested.
There are a lot of testing frameworks out there. For most modules, tape is a great choice. It's similar to writing tests in other languages like go
and python
so your ability to write tests will remain portable.
Here are some reasons why tape
is great:
- Test results are output as TAP
- Works in a browser, so you can test for browser compatibility
- Works like any other module. You install it and require it into your test files.
- Does not introduce non-standard language keywords, like
describe
- Provides a conservative but usefdul set of test assertions by default
- Does not extend default object prototypes in your testing environment
- Very stable, simple and complete
Some links on using tape:
Here's some tape tests
var test = require('tape')
var testString = 'normally i would be imported too'
function foo() {
return testString
}
test('lets test the foo function', function(t) {
setTimeout(function() {
t.equal(foo(), testString, 'foo() returns testString')
}, 100)
})
test('lets test the foo function', function(t) {
t.notEqual(foo(), 'boop', 'foo() returns testString')
})
Where do I go to learn these?
AKA OO AKA Object Oriented, AKA 👻👻 programming in Javascript. Also prototypes.
OO Programming is a way of organizing and writing code that breaks code up into "classes" that allow you to create object factories that create object instances that have instance specific properties and methods (aka a property of an object that is a function).
JavaScript doesn't subscribe to the purely object oriented world-view like other language (e.g. Java or Objective C). Object orientation is a valid, common and occasionally ideal way to organize code, but other times its not necessary.
Javascript lets you write code in a object oriented way. In fact, it lets you do it a number of different ways. This is a blessing, and a curse. While it allows for OO, often in a less verbose syntax than pure OO languages, you will have to learn all the different ways of writing OOJS in order to really do it effectively, as well as read code of other authors.
More often than not, you will find popular javascript libraries opt for the less ideal OO style for some reason. There is no good explanation for this other than OO programming is currently a predominant coding paradigm, and certain OO JS styles resemble classical OO more than other more ideal OOJS styles.
Its best to try to understand all the different OO styles, their strengths and weaknesses, and mix and match the parts that work with the problem domain you are attempting to solve.
Douglas Crockford wrote the book on the different types of object oriented inheritance you can have in JavaScript in 2008, and he boils it down to three types: (Pseudo)classical, Prototypal, and Functional. This breakdown is required reading and can be found in JavaScript: The Good Parts.
Eric Elliott revisits this issue in 2014 and breaks this issue down even more in Programming JavaScript Applications(Now a free e-book!) Eric identifies the techniques used in each of Doug's OO styles, explains how they work and what they are good and not so good for, and introduces Fluent-Style JavaScript and the concepts of stamps.
While its worth reading Eric and Doug's take on OOJS, lets check out a quick reference of the different strategies.
- https://medium.com/javascript-scene/what-is-webassembly-the-dawn-of-a-new-era-61256ec5a8f6
- https://medium.com/javascript-scene/the-two-pillars-of-javascript-ee6f3281e7f3
- https://medium.com/javascript-scene/the-two-pillars-of-javascript-pt-2-functional-programming-a63aa53a41a4
- https://medium.com/javascript-scene/common-misconceptions-about-inheritance-in-javascript-d5d9bab29b0a
- Streams handbook
Streams can help to separate your concerns because they restrict the implementation surface area into a consistent interface that can be reused.
- through
- through2
- duplexify - asynchronously assign readable and writable
- on streams1 vs 2-3
- pull stream
- stephenplusplus/stream-faqs
- https://nodejs.org/en/blog/feature/streams2/
- https://dl.dropboxusercontent.com/u/3685/presentations/streams2/streams2-ko.pdf
- https://iojs.org/api/stream.html
- https://nodejs.org/api/stream.html
- https://github.com/nodejs/readable-stream
- https://strongloop.com/strongblog/whats-new-io-js-beta-streams3/
- http://stackoverflow.com/questions/21538812/what-is-streams3-in-node-js-and-how-does-it-differ-from-streams2
- nodejs/node-v0.x-archive#5860
- https://nodejs.org/docs/v0.11.5/api/stream.html
- http://brycebaril.github.io/streams2-presentation/#/21/7
- https://www.npmjs.com/package/through2-spy
- https://www.npmjs.com/package/stream-meter
- https://github.com/brycebaril/node-stream-spigot
- https://www.npmjs.com/package/through2
- https://www.npmjs.com/package/through2-filter
- https://www.npmjs.com/package/stream-spigot#star
- https://twitter.com/rvagg/status/608577853601398784
- https://iojs.org/api/stream.html#stream_simplified_constructor_api
- http://words.jessekeane.me/front-end-streams/
- https://cloud.githubusercontent.com/assets/37303/5728694/f9a3e300-9b20-11e4-9e14-a6938b3327f0.png
- http://substack.net/node_aesthetic
- http://substack.net/many_things
- https://github.com/Raynos/http-framework
- http://maxogden.com/a-month-of-modules.html
- http://substack.net/finding_modules
- https://github.com/substack/node-mkdirp/blob/master/index.js
- http://www.ustream.tv/recorded/46670615
- https://gist.github.com/substack/68f8d502be42d5cd4942
- https://en.wikipedia.org/wiki/Turtles_all_the_way_down
- https://gist.github.com/substack/5075355
- http://0fps.net/2013/01/22/commonjs-why-and-how/
- https://en.wikipedia.org/wiki/Modular_programming
- https://gist.github.com/dominictarr/2401787
- https://github.com/felixge/node-style-guide
- http://www.quora.com/Whats-the-correct-way-to-write-nodejs-modules-There-seems-to-be-discrepancies-between-experts-like-substack-dominictarr-vs-the-newd-constructor-style-of-V8-optimization-recommendations-What-gives
- https://github.com/openopensource/openopensource.github.io
- https://github.com/ngoldman/open-2-contributing
- https://github.com/yoshuawuyts/knowledge
- https://github.com/npm-dom
When applications are done well, they are just the really application-specific, brackish residue that can't be so easily abstracted away. All the nice, reusable components sublimate away onto github and npm where everybody can collaborate to advance the commons. -- James Halliday-substack.net/how_I_write_modules
- http://blog.izs.me/post/104685388058/io-js
- https://gist.github.com/maxogden/d96123138522c84cdb25
- http://tableflip.io:1234/
- nodejs/node#37
- https://nodesource.com/blog/was-this-trip-really-necessary
- https://twitter.com/rvagg/status/598605393636429825
- https://twitter.com/mikeal/status/598595967928008705
- https://github.com/nodejs/node
- https://www.youtube.com/watch?v=UbYiFLf7MpU
- nodejs/node#1664 (comment)
- https://www.youtube.com/watch?v=hZJCnT7E1ts
To be consistent with node's emphasis on small, interchangeable modules, somehow a hex sticker specification was created. Other than looking super cool, and evenly dividing up valuable laptop surface real estate, one could understand hex stickers as node conventions actualizing as a distributed, shared community culture/art project. By following a few simple, basic rules, everyone can create an interesting and useful piece of the puzzle that works nicely together to make up the whole while progressing the commons. In this case, tessellating stickers across a laptop or water bottle without any wasted space!
Hexb.in Hex sticker registry
It's pretty easy to make hex stickers.
- stickermule.com: Order die-cut stickers and ensure that they understand the correct dimensions of the stickers.
- hexi.pics: a website dedicated to creating hex stickers. Lets you trivially create hex stickers and order them.
Oh yes. You heard right, a sticker standard.
“How standards are made on small scale”: A nice little history of the spec.
We quickly agreed that there should be a standard for them and that the most useful one seems to be 2”x1.75” hexagons. I tweeted it and the reaction was way more amazing than I expected.
Resources for teaching others and writing nodeschool adventures.
- http://bellard.org/jslinux/
- https://github.com/jashkenas/coffeescript/wiki/List-of-languages-that-compile-to-JS
- https://brendaneich.com/2015/06/from-asm-js-to-webassembly/
- https://medium.com/javascript-scene/what-is-webassembly-the-dawn-of-a-new-era-61256ec5a8f6
- https://medium.com/javascript-scene/why-we-need-webassembly-an-interview-with-brendan-eich-7fb2a60b0723
- https://medium.com/@brianleroux/es6-modules-amd-and-commonjs-c1acefbe6fc0
- http://espadrine.github.io/New-In-A-Spec/es6/
- https://en.wikipedia.org/wiki/ECMAScript
- http://blog.izs.me/post/25906678790/on-es-6-modules
- http://tomdale.net/2012/01/amd-is-not-the-answer/
- http://lebron.technology/
- http://hapijs.com/
- http://expressjs.com/
- https://github.com/Raynos/http-framework
- https://github.com/Raynos/mercury
- https://jshttp.github.io/
Every “why Node.js sucks” article ever --nodejs reactions
Node has its share of weak points. Articles that hate on it generally miss these points. A good way to learn about a tool is to analyze its criticism. Does it have merit? Is it correct? Is the the stated issue as bad as the critic makes it out to be?
- Dive into html5
- HTML5 and css Jon Ducket
- irc
- github
- conferences
- gitter/slack?
- http://lin-clark.com/blog/2014/07/01/authoring-nodejs-workshopper-lessons/
- http://nodeschool.io/
- orily git
- git for 5 year olds
- yay desktop apps
- electron handbook
- https://github.com/sindresorhus/awesome-electron
😎
- https://www.youtube.com/watch?v=84PE6EF3YWY&list=FLd8cLeIEVcsbhyX9FDph-Jg&index=3
- https://github.com/substack/dnode
- https://github.com/substack/upnode
- https://github.com/substack/fleet
- https://github.com/substack/seaport
- http://substack.net/multi_server_continuous_deployment_with_fleet
- http://substack.net/semver_your_services_with_seaport
- http://substack.net/shared_rendering_in_node_and_the_browser
This document was created after amassing a large collection of node related links helpful to learning and understanding node. Here is a partial linkdump until the rest of the guide can be written.
- http://nikhilm.github.io/uvbook/threads.html#core-thread-operations
- nodejs/node-v0.x-archive#5132 (comment)
- https://github.com/sindresorhus/awesome
- https://github.com/sindresorhus/awesome-nodejs
- https://gist.github.com/staltz/868e7e9bc2a7b8c1f754
- https://github.com/kriskowal/gtor
- https://gist.github.com/jhclark/2845836
- http://i.imgur.com/k0t1e.png
- https://github.com/essdot/js-lessons
- https://github.com/shama/base-element
- http://coodict.github.io/javascript-in-one-pic/
- https://github.com/hughsk/disc
- https://github.com/reddit/reddit-mobile/issues/247
- http://www.finnpauls.de/streams-editor/
- https://github.com/finnp/streams-editor
- http://dominictarr.com/post/25516279897/why-you-should-never-write-a-package-manager
- https://gist.github.com/chrisdickinson/0a236ce62097c806113d#file-wip-md
- https://gist.github.com/rgbkrk/91b40941a38daf700e61
- http://browserify.org/articles.html
- https://medium.com/@mjackson/universal-javascript-4761051b7ae9
- http://www.learnenough.com/command-line