intricatecloud

Creating a simple npm library to use in and out of the browser

Sharing some reusable javascript is hard - even moreso if you're trying to share between node and the browser. I'm just trying to share some 10 lines of code. Why is this taking me more than a day to figure out?! Its times like these I have to talk myself out of doing a copy/paste. I know its wrong, and reusable packages are better, but ugh... at what cost.

  • Some JS code is meant to run in the browser, some JS code can only run on the server-side.
  • es6 != es2016, es5 != 2015, esNext/es8, HOW OFTEN IS THIS CHANGING?! I found this explanation helpful
  • Webpack? Babel? I didn't think I'd need to deal with this just to share some code!

I mean, I get it. But why do I need 147 MB in node_modules just to share 10 lines of code!

Recently at work, I tried to extract some 20-something lines of javascript into a re-usable npm library to share between a few projects. I thought it would be easy and straightforward, and when I found out it wasn't, I started putting together this post to help guide people through what it takes to build a reusable npm package.

In this guide, I'm going to create a react app that loads random images of dogs, cats, and goats. Why those 3? :shrug: They were 3 random things I found on public-apis. Then I'll take that code, and turn it into a re-usable npm package that can be used in the browser and node with webpack, babel, and typescript.

I'm also including the intricatecloud/reusable-js-demo repo here so you can take a look at the project. Best way to follow along is to look at each commit to see each change that I'm specifically making (they are also linked after each section in the post). I've shortened some of the interesting bits here for brevity.

Building Random Animal Demo

Step 1. Create the app and run it

$ npx create-react-app random-animal-demo
$ cd random-animal-demo
$ npm start

Check out the full commit 582d2b

Step 2. Hook it up to some apis

In the app, I'm going to display a dog, cat, and goat picture. I'll tweak some of the boilerplate in the react demo app. I took the default react stuff that was there, and refactored it to show an image and some text. Now I'll integrate these free apis by using axios to make requests (it works in the browser, and in node).

We'll use these APIs which allow us to call them from the browser.

https://aws.random.cat/meow
https://random.dog/woof.json
http://placegoat.com/200

You'll notice each API works slightly differently. The cat API returns a JSON object with a file property. The dog API returns a JSON object with a url property. And the goat API returns the image itself. I added some logic to componentDidMount() to run it on a timer so that it cycles every few seconds

const DOG = 0
const CAT = 1
const GOAT = 2

switch(newIndex) {
case CAT:
    axios.get('https://aws.random.cat/meow').then((response) => {
        const imageSrc = response.data.file
        const text = 'CAT'
        this.setState({currentAnimal: {imageSrc, text}})
    })
    break;
case DOG:
    ...

Check out the full commit 3dd98c5

Step 3: Choose how you want to import your file

Once you want to pull the logic into another file, you have to decide how you want to import it. There are a few options for how you want to import it:

ES6 Imports - if you want to use import AnimalApi from 'animal-api'

animal-api.js

export default {
    getDog: () => ....
    getCat: () => ....
    getGoat: () => ....
}

ES6 Destructured Import - if you want to use import { getDog, getCat, getGoat } from 'animal-api'

animal-api.js

export const getCat = () => ....
export const getDog = () => ....
export const getGoat = () => ....

CommonJS - if you want to use const AnimalApi = require('animal-api')

animal-api.js

module.exports = {
    getDog, getCat, getGoat
}

When would you choose one over the other?

If your app only needs to work in a browser, and only in the context of React (or Angular2+ or environment that uses ES6 modules), then you're fine with using an ES6 import.

If your lib is meant to be used in the browser, and you need to include it in a vanilla JS HTML app, you need to use a bundler like webpack to bundle your app as a lib.

If you use webpack and take advantage of code splitting and tree shaking, you can use ES6 Destructured imports. What this means is rather than include all of lodash in your app, you can only include the functions you want and you'll have a smaller built app.

If you're writing an app or a library that needs to run in BOTH the browser, and in node, then you'll need to produce a few different versions of your library: one meant for the browser (as a script tag), one as an es6 module, and one for using in node.

For this guide, we're going to be writing an ES6 module, so we can target using import AnimalApi from 'animal-api'.

Step 4: Extract some of this out into an npm library

Would you really pull some of this out into a lib? Probably not. But this is an example of how complex something that looks so simple could be. So bear with me.

I can modify my App.js file to use this new file:

 import React from 'react';
 import axios from 'axios';
+import AnimalApi from './animal-api';
 import './App.css';

 class App extends React.Component {
@@ -28,23 +29,19 @@ class App extends React.Component {

       switch(newIndex) {
         case CAT:
-          axios.get('https://aws.random.cat/meow').then((response) => {
-            const imageSrc = response.data.file
-            const text = 'CAT'
-            this.setState({currentAnimal: {imageSrc, text}})
+          AnimalApi.getCat().then((animal) => {
+            this.setState({currentAnimal: animal})
           })
           break;
         case DOG:
         ...

src/animal-api.js

const getCat = () => {
    return axios.get('https://aws.random.cat/meow').then((response) => {
        const imageSrc = response.data.file
        const text = 'CAT'
        return {imageSrc, text}
    })
}
...
export default {
    getDog,
    getCat,
    getGoat
}

Check out the full commit 37c3a3a

Next, we're going to move this logic to its own npm package.

How to package your library so that it can be used

Create a new npm library locally

Create a new folder outside of your React project for the npm package we're going to make.

$ mkdir animal-api
$ cd animal-api
$ npm init

You can use all the defaults for the npm init command which creates a package.json file like this:

{
  "name": "animal-api",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
}

Then I'll copy paste all of the code thats in my random-animal-demo into animal-api/index.js, ready to be used with import AnimalApi from 'animal-api'.

Because this is a library, we'll want to add some tests to this to make sure this library is working. I like using jest, so I'll pull it in: npm install --save-dev jest

I'll then create animal-api/spec/index.spec.js:

import AnimalApi from './index'

describe('animal-api', () => {
    it('gets dogs', () => {
        return AnimalApi.getDog()
            .then((animal) => {
                expect(animal.imageSrc).not.toBeUndefined()
                expect(animal.text).toEqual('DOG')
            })
   })
})

Check out the full commit 6d797c

Now run jest:

☁  animal-api [master] ⚡ jest
 FAIL  spec/index.spec.js
  ● Test suite failed to run

    /Users/dperez/workspace/animal-api/spec/index.spec.js:1
    ({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,global,jest){import AnimalApi from '../index';
                                                                                                    ^^^^^^^^^

    SyntaxError: Unexpected identifier

      at Runtime.createScriptFromCode (node_modules/jest-runtime/build/index.js:1059:14)

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        0.834s
Ran all test suites.

And I thought this would have been the easiest part. Apparently, jest doesn't like the use of the import statement.

At the moment, with just an index.js file and jest, its going to be running inside a node.js environment where import is not yet supported. (This walkthrough is using node 10).

This is where babel comes into play. babel will transpile ("translate/compile") your js files from ES6 (where you can use import) and ES5 which node.js supports. Jest has a section on enabling babel support here. We'll have to install a few more packages, and a little configuration.

Run npm install --save-dev babel-jest @babel/core @babel/preset-env and then add babel.config.js

babel.config.js

module.exports = {
  presets: [
    [
      '@babel/preset-env',
      {
        targets: {
          node: 'current',
        },
      },
    ],
  ],
};

Then when you run jest, you get to the next error

☁  animal-api [master] ⚡ jest
 FAIL  spec/index.spec.js
  ● Test suite failed to run

    Cannot find module 'axios' from 'index.js'

Aha, progress. Now it knows how to read the import statement, and the test runs. Now i'm missing a dependency - axios. npm install --save axios. Now re-run jest and all green!

☁  animal-api [master] ⚡ jest
 PASS  spec/index.spec.js
  animal-api
    ✓ gets dogs (161ms)
    ✓ gets cats (69ms)
    ✓ gets goats

Test Suites: 1 passed, 1 total
Tests:       3 passed, 3 total
Snapshots:   0 total
Time:        1.148s
Ran all test suites.

Check out the full commit fafc23a

Next up, we'll go through a few scenarios for how to include this package in another project

Scenario 1: A lib that only needs to work in a browser, and you're using React/Angular2+ (ES6 modules).

I'm at a good spot now where I can try including this library in my react app. Rather than publishing the npm package publicly, I'll just "install" it from the folder on my machine. I'll switch back to my react app and run npm install ../animal-api

My package.json now looks like this:

     "@testing-library/user-event": "^7.2.1",
+    "animal-api": "file:../animal-api",
     "axios": "^0.19.2",

I can try using it from the React app. Back to App.js

 import React from 'react';
-import axios from 'axios';
-import AnimalApi from './animal-api';
+import AnimalApi from 'animal-api';
 import './App.css';

Check out the full commit a802898

Boom! Everything works! Fantastic, I've now got a publish-able package. By coincidence, theres already an animal-api published to npm so I wont bother adding yet-another-one. If all you need to do is share some code thats going to be used in a React/Angular framework (where you can use import), then you can stop right here. Move on with your life and be grateful you don't need to read Scenario 2.

Scenario 2. Using the library in a browser using a script tag

This is where we want to create a file that be included as a script tag.

In your animal-api directory, create index.html, run a local webserver with npx http-server and visit your page http://localhost:8080:

index.html

...
<div id="imageSrc"></div>
<div id="text"></div>
...
<script src="./index.js"></script>
<script>
    AnimalApi.getDog().then(function(animal) {
        document.querySelector('#imageSrc').textContent = animal.imageSrc
        document.querySelector('#text').textContent = animal.text
    })
</script>

Check out the full commit df79f05

The first error I get with this is:

index.js:1 Uncaught SyntaxError: Cannot use import statement outside a module
index.html:11 Uncaught ReferenceError: AnimalApi is not defined
    at index.html:11
(anonymous) @ index.html:11

This is saying that the browser can't run your file that contains import statements. We used babel before so that jest was able to use the import keyword. We have to do something similar here. The only difference is that we need to save the "transpiled" output so we can include it in the browser. Since we already have babel as a dependency in animal-api, we can use it to convert to a UMD module

$ npm install --save-dev @babel/plugin-transform-modules-umd @babel/core @babel/cli

Then we can run babel index.js -d lib which will create a lib/index.js file ready to be consumed as a script tag in your browser. Now I can update my script tag to point to this other file.

     </body>
-<script src="./index.js"></script>
+<script src="./lib/index.js"></script>
 <script>
     AnimalApi.getDog().then(function(animal) {

Once I refresh the page, one error goes away, the other one stays.

index.html:11 Uncaught ReferenceError: AnimalApi is not defined
    at index.html:11

We haven't configured the babel plugin that creates the UMD module to use the name that we want. Because our file is named index.js, we'll update our babel.config.js to this:

babel.config.js

 module.exports = {
+    plugins: [
+        ["@babel/plugin-transform-modules-umd", {
+        exactGlobals: true,
+        globals: {
+          index: 'AnimalApi'
+        }
+      }]
+    ],
     presets: [

Re-run babel index.js -d lib, and refresh the page

And now I get a new error! Progress!

Uncaught TypeError: AnimalApi.getDog is not a function
    at index.html:8

This one is a head scratcher. Whatever babel is doing to this file, I'm losing access to my functions. Lets take a look at what the transpiled file looks like lib/index.js. In this file, I notice 2 things:

global.AnimalApi = mod.exports;
...
var _default = {
    getDog,
    getCat,
    getGoat
  };
  _exports.default = _default;

This suggests that there is a .default property on my AnimalApi global object. This is a small detail thats usually invisible when you're using the import axios from 'axios' syntax. An ES6 package will provide a "default" export, and when you use the import statement, it knows to look at .default property for you, so you don't need to write it.

Lets update our code to match that:

AnimalApi.default.getDog().then(function(animal) {
    ...
})

Now, when we refresh the browser, we get our next error:

index.js:36 Uncaught TypeError: Cannot read property 'get' of undefined
    at Object.getDog (index.js:36)
    at index.html:8

Check out the full commit 5da97d8

This is failing to call axios.get because axios is undefined. All we did was transpile the index.js file. The code for axios isn't there, and the browser doesn't know that its supposed to search your node_modules directory for it (nor can it).

What we need to do is create one js file, that has all the dependencies of that library built into it, so that you only need to add the 1 script tag to your site. Babel can't handle your dependencies, it can only translate.

We now need to introduce our 2nd tool: webpack!

We'll use webpack to take a look at our file, and everything that is import'd/require'd will then get "packed" into one file.

We'll install webpack

$ npm install --save-dev webpack webpack-cli

And add the following to webpack.config.js

const path = require('path');

module.exports = {
  entry: './index.js',
  output: {
    path: path.resolve(__dirname, 'lib'),
    filename: 'animal-api.js',
    library: 'AnimalApi',
    libraryTarget: 'var'
  },
};

This config will look at ./index.js and create ./lib/animal-api.js which will make the variable AnimalApi available within your javascript, using the var target.

Now run webpack and you'll notice that lib/animal-api.js got a lot bigger, with more gibberish.

Go back to your index.html and update the script tag:

-<script src="./lib/index.js"></script>
+<script src="./lib/animal-api.js"></script>

Check out the full commit 8f865c5

Refresh the page, and success! The data shows up! If thats all you needed, then great. You can stop here. Otherwise....

Scenario 3: Using the library in both the browser AND node

I'll create a plain javascript file that I'll run with node: node node-test.js

node-test.js

const AnimalApi = require('./index.js')

AnimalApi.getCat().then(animal => {
    console.log(animal)
})

I get this error:

/Users/dperez/workspace/animal-api/index.js:1
(function (exports, require, module, __filename, __dirname) { import axios from 'axios';
                                                                     ^^^^^

SyntaxError: Unexpected identifier

We saw this when we tried to run our index.spec.js file with jest before using babel. We're trying to import an ES6 module with require which won't work because import is part of ES6 and node is running against ES5 (for node 10 anyways). We built a lib/index.js using babel earlier in Scenario 1 which contains an ES5 module (that was transpiled from ES6). Lets use that instead:

I now get this error:

☁  animal-api [master] ⚡ node node-test.js
/Users/dperez/workspace/animal-api/node-test.js:3
AnimalApi.getCat().then(animal => {
          ^

TypeError: AnimalApi.getCat is not a function

We saw this earlier in the browser, and saw that we needed to add AnimalApi.default.getCat() because of the way that babel transpiles your ES6 module to a UMD module. Once we update it....

☁  animal-api [master] ⚡ node node-test.js
{ logo: 'https://purr.objects-us-east-1.dream.io/i/v0SoPzk.jpg',
  text: 'CAT' }

Check out the full commit 3cdea87

Success! We can use the ES5 module lib/index.js that was created by babel in order to use the library in node.js.

Scenario 4: You want to use Typescript in your library, but still include it as normal in a React/node project

What if you like using Typescript and would rather create your library using Typescript, but still use it in other non-typescript projects? No fear, there's an easy solution.

I'm going to add an index.ts file which contains the library, only using Typescript. Here's what it looks like compared to our index.js file. We added an interface and added a function signature.

+interface Animal {
+    imageSrc: string
+    text: string
+}
+
+const getCat = (): Promise<Animal> => {
     return axios.get('https://aws.random.cat/meow').then((response) => {
         const imageSrc = response.data.file
         const text = 'CAT'
-        return {imageSrc, text}
+        return {imageSrc, text} as Animal
     })
 }

If we try to run this .ts file straight from react by adding:

App.js

 import React from 'react';
-import AnimalApi from 'animal-api';
+import AnimalApi from 'animal-api/index.ts';
 import './App.css';

When we refresh the app, we see this error:

Module parse failed: The keyword 'interface' is reserved (3:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
| import axios from 'axios';
|
> interface Animal {
|     logo: string
|     text: string

The problem is that React doesn't know what to do with your .ts file. So now you need to convert your index.ts file into something that React can read which would be either an ES6 module, or an ES5 module.

Typescript is able to do this without the help of babel. We can compile our file to an ES5 module that we can then import or require. First, install typescript npm install --save-dev typescript

Add a tsconfig.js to configure your Typescript compiler

{
    "compilerOptions": {
        "outDir": "./dist",
        "lib":["ES2015", "ES2016"],
        "sourceMap": true,
        "noImplicitAny": true,
        "target": "es5",
        "module": "commonjs"
    }
}

When we run tsc, we'll get dist/index.js and that file we'll be able to include from our react project or node-test.js file.

index.html

 import React from 'react';
-import AnimalApi from 'animal-api';
+import AnimalApi from 'animal-api/dist/index';
 import './App.css';

Check out the full commit 2625124

Now when you refresh your pages, success! Now, for the last bit...

Scenario 5: You want to use Typescript to build a lib included as a script tag

When we want to include a library or project using a script tag, we need to use a bundler to make sure that our projects dependencies are included with our library. We used webpack earlier to do this, and without much mucking around, we were able to produce a js bundle that can be included as a <script src="./lib/animal-api.js">

We'll update webpack.config.js to add a Typescript loader which will be able to read your Typescript file and do things with it. We'll still use webpack to bundle it.

Run npm install --save-dev ts-loader to install the loader and then update the webpack config.

webpack.config.js

 module.exports = {
-  entry: './index.js',
+  entry: './index.ts',
+  module: {
+    rules: [
+      {
+        test: /\.ts$/,
+        use: 'ts-loader',
+        exclude: /node_modules/,
+      }
+    ]
+  },
+  resolve: {
+    extensions: ['.ts', '.js'],
+  },
   output: {

Thats all we have to add to our webpack configuration to get this project built and bundled. Once you run webpack, you'll get a new file ./lib/animal-api.js. Go back to your test index.html file to test it out. Thats it!

Check out the last full commit cc18f80

Theres a few things to be aware of. Our very simple library has multiple entry points that need to be maintained.

  • If you can consume ES6 modules using import AnimalApi from 'animal-api' then that will rely on your ./index.js file.
  • If you want to consume your ES6 module in the browser, you need to use the ./lib/animal-api.js file which will make AnimalApi globally available using the babel plugin, and bundles all the dependencies along with it using webpack.
  • If you want to use your ES6 module in node, then you need to use ./lib/index.js, the UMD module that we built with babel.
  • If you want to use your Typescript file from another Typescript project, you need to use ./lib/index.ts
  • If you want to use your Typescript file in a node.js environment, you have to use tsc to compile it down to an ES5 module that you can then re-use.
  • If you want to use your Typescript file in a <script src="mylib.js">, you'll need to use webpack with ts-loader to compile your typescript, bundle all the dependencies, and produce a UMD bundle.

Both your source files, and built files can be distributed with your npm package and left to the user to choose which one they need. But it does require you to put a fair amount of tooling in place to create that convenience.

Wrapping up

Remember, you can follow along commit by commit by checking out the repo here intricatecloud/reusable-js-demo. Drop a ⭐️ if you found it helpful!

Thanks for reading along, hope this helped you understand the differences between the different module types, and how to go about extracting a library from our application that we can use both in and outside of the browser.

If you're interested in more things react - definitely check out my in-depth guide to adding google sign-in to a react app, or my guide to deploying a static website using S3 + Cloudfront with terraform

Tags:
Danny Aslam-Perez
Sr Software Developer and Engineering Manager at an edtech company writing about leading teams, automated testing, and building sites hosted on AWS.

You might be interested in…

Menu