JavaScript ESM is good, but it may not be so good right now

Leave a Comment / / rxliuli | 琉璃

Original link: https://blog.rxliuli.com/p/73331967c1814df480811eee598e714b/

foreword

Many front-end developers may know that since sindresorhus issued the esm only declaration for more than a year last year, many projects have turned to esm only, that is, only supporting esm but not cjs, in order to force the entire ecosystem to migrate to esm faster only.

Some popular projects already do this

  • Thousands of npm packages maintained by sindresorhus
  • node-fetch
  • remark series
  • More. . .

They claim: You can still use the existing version without upgrading to the latest version, major version updates will not affect you. How are the facts?

I have encountered the problem of not being able to use the esm only package several times before. Whenever I want to try esm only, there are always some problems. The most painful thing is that some packages are esm only, while others are cjs only. To choose to give up one side, fuck esm only. Some of the main problems are that the cjs only packages, and the must-compatible packages typescript/jest/ts-jest/wallaby do not properly support esm. Of course, we can choose to look for alternatives to esm only packages, such as globby => fast-glob, remark => markdown-it, node-fetch => node-fetch@2, lodash-es => lodash, but this is not A long-term option, not to mention that some bags are hard to really find a replacement for, such as the remark series.

So, what’s wrong with using an older version of the package?
The main problem is that it is difficult to find the correct version. Of course, if you are using a relatively independent package, such as node-fetch, you can use the v2 version directly. But if you’re using a project like vuepress/remark that contains many small packages in a monorepo, it’s hard to find the correct version for each subproject.

When I was working on an epub generator recently, I needed to do some conversions from markdown and ast, and finally convert it to html, so I used remark again, and decided to really try to use esm. The following are some attempts.

Target

Using esm must solve the following problems, otherwise it is impossible to use in production environment

  • Typescript support – basically all web projects use ts, not supporting it is unacceptable
  • jest support – also heavily used testing tool
    • wallaby support – a paid WYSIWYG testing tool
  • Allow references to cjs modules – need to support existing packages
  • The dual module package can still support both esm/cjs projects – need to support cjs project references
  • Support for unbundled modules – some private modules in monorepo will not be bundled
  • esbuild support – esbuild is becoming the lib bundle standard

Modify the package declaration

The first step is to modify the module type of the package. Modify "type": "module" to declare the package as esm, and all js code will run as esm module by default.

 { "type" : "module" }

TypeScript support

NodeNext is supported since ts4.7, so need to change tsconfig.json

 { "compilerOptions" : { "module" : "ESNext" , "moduleResolution" : "NodeNext" } }

In addition, importing other ts files in the ts file must use the .js suffix

This is an odd limitation, refer to the ts 4.7 release documentation

 import { helper } from "./foo.js" ; // works in ESM & CJS helper ( ) ;

Does it seem strange, but now it can only be written like this, typescript even prompts like this

jest/wallaby support

For example run the following code using the pnpm jest src/lodash.test.ts command

 import { uniq } from "lodash-es" ; it ( "uniq" , ( ) => { console . log ( uniq ( [ 1 , 2 , 1 ] ) ) ; } ) ;

An error occurred

 Jest encountered an unexpected token

From jest 28, experimental esm support is supported, and wallaby/ts-jest can also be supported by configuration. Follow the steps below to handle it

  1. Configure ts-jest

     { "jest" : { "preset" : "ts-jest/presets/default-esm" , "globals" : { "ts-jest" : { "useESM" : true } } , "moduleNameMapper" : { "^(\\.{1,2}/.*)\\.js$" : "$1" } , "testMatch" : [ "<rootDir>/src/**/__tests__/*.test.ts" ] } }

  2. Modify the command to node --experimental-vm-modules node_modules/jest/bin/jest.js src/lodash.test.ts

  3. Configure wallaby (you can run the test files in the __tests__ directory here, strange…)

     { "wallaby" : { "env" : { "params" : { "runner" : "--experimental-vm-modules" } } } }

  4. Since the import of esm is static, you also need to uninstall @types/jest use the @jest/globals package to import the functions required for testing, such as it/expect/describe/beforeEach , etc.

     import { it , expect } from "@jest/globals" ; it ( "basic" , ( ) => { expect ( 1 + 2 ) . toBe ( 3 ) ; } ) ;

nodejs support

Nodejs has supported esm since 14, but until now 18, the migration is still not smooth, mainly encountered the following problems.

import cjs only module

Unfortunately, a large number of existing packages are cjs only modules, it is impossible to migrate in a short time, and the interoperability of esm and cjs in nodejs is not very good, so it needs to be dealt with. Take fs-extra as an example below:

I used to write like this

 import { readdir } from "fs-extra" ; import path from "path" ; console . log ( await readdir ( path . resolve ( ) ) ) ;

The error SyntaxError: The requested module 'fs-extra' does not provide an export named 'readdir' when running with tsx appears to be a known bug, reference: https://github.com/esbuild-kit/tsx /issues/38

Now it needs to be changed to

 import fsExtra from "fs-extra" ; import path from "path" ; console . log ( await fsExtra . readdir ( path . resolve ( ) ) ) ;

Or modify the following code to run with ts-node --esm <file> (tsx does not support this method)

 import fsExtra = require ( "fs-extra" ) ; import path from "path" ; console . log ( await fsExtra . readdir ( path . resolve ( ) ) ) ;

use __dirname

Yes, you read that right, __dirname is not available under the esm module, instead import.meta.url is used. All in all, the way it is used now is

 import path from "path" ; import { fileURLToPath } from "url" ; const __filename = fileURLToPath ( import . meta . url ) ; const __dirname = path . dirname ( __filename ) ; console . log ( __dirname ) ;

Refer to the article https://flaviocopes.com/fix-dirname-not-defined-es-module-scope/ , and then talk about how to package cjs bundles to deal with import.meta.url when talking about esbuild (not supported in cjs, and is a choice of two).

lib maintenance and use

New esm and cjs dual package support configuration

Previously, we differentiated modules by the main/module field

 { "main" : "dist/index.js" , "module" : "dist/index.esm.js" , "types" : "dist/index.d.ts" }

But referencing in the esm project will report an error

 SyntaxError: The requested module 'cjs-and-esm-lib' does not provide an export named 'hello'

The esm project does not recognize this, it newly defines the exports field, so it needs to be added (note that the main field still needs to remain compatible with the old version of node) the exports field

 { "exports" : { "." : { "import" : "./dist/index.esm.js" , "require" : "./dist/index.js" , "types" : "./dist/index.d.ts" } } }

Refer to this answer: https://stackoverflow.com/a/70020984

esbuild support

I thought that esbuild naturally supports esm so it should be very simple, but in fact, it also encountered quite a few problems.

bundling the following code as cjs will give an error

 import path from "path" ; import { fileURLToPath } from "url" ; import fsExtra from "fs-extra" ; const { readdir } = fsExtra ; const __filename = fileURLToPath ( import . meta . url ) ; const __dirname = path . dirname ( __filename ) ; console . log ( __dirname ) ; console . log ( await readdir ( __dirname ) ) ;

Order

 esbuild src/bin.ts --platform=node --outfile=dist/bin.esm.js --bundle --sourcemap --format=esmesbuild src/bin.ts --platform=node --outfile=dist/bin.js --bundle --sourcemap --format=cjs

mistake

 [ERROR] Top-level await is currently not supported with the "cjs" output format

This is because cjs cannot contain top-level await, which is modified to

 import path from "path" ; import { fileURLToPath } from "url" ; import fsExtra from "fs-extra" ; const { readdir } = fsExtra ; ( async ( ) => { const __filename = fileURLToPath ( import . meta . url ) ; const __dirname = path . dirname ( __filename ) ; console . log ( __dirname ) ; console . log ( await readdir ( __dirname ) ) ; } ) ( ) ;

There is no problem with bundling, but running will give errors

 node dist/bin.js

first error first

 var import_path = __toESM(require("path"), 1); ^ReferenceError: require is not defined in ES module scope, you can use import insteadThis file is being treated as an ES module because it has a '.js' file extension and 'esm-demo\packages\esm-include-cjs-lib\package.json' contains "type": "module". Totreat it as a CommonJS script, rename it to use the '.cjs' file extension.

It says that this is an esm package, the default code is esm module, if you want to execute in cjs module, you need to modify it to cjs suffix.

Modify command

 esbuild src/bin.ts --platform=node --outfile=dist/bin.cjs --bundle --sourcemap --format=cjs

Then came the second error

 TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string or an instance of URL. Received undefined

Related code

 // src/bin.ts var import_path = __toESM ( require ( "path" ) , 1 ) ; var import_url = require ( "url" ) ; var import_fs_extra = __toESM ( require_lib ( ) , 1 ) ; var import_meta = { } ; var { readdir } = import_fs_extra . default ; ( async ( ) => { const __filename = ( 0 , import_url . fileURLToPath ) ( import_meta . url ) ; // 这里是关键，因为import.meta.url 在cjs 代码中是空的const __dirname = import_path . default . dirname ( __filename ) ; console . log ( __dirname ) ; console . log ( await readdir ( __dirname ) ) ; } ) ( ) ;

Based on the author’s answer in this issue , modify the command

 esbuild src/bin.ts --platform=node --outfile=dist/bin.cjs --inject:./import-meta-url.js --define:import.meta.url=import_meta_url --bundle --sourcemap --format=cjs

Unfortunately, this is no longer in effect, the code for the bundle is as follows

 // import-meta-url.js var import_meta_url2 = require ( "url" ) . pathToFileURL ( __filename ) ; console . log ( import_meta_url2 ) ; // src/bin.ts var import_path = __toESM ( require ( "path" ) , 1 ) ; var import_url = require ( "url" ) ; ( async ( ) => { const __filename2 = ( 0 , import_url . fileURLToPath ) ( import_meta_url ) ; const __dirname = import_path . default . dirname ( __filename2 ) ; console . log ( __dirname ) ; } ) ( ) ;

It is obvious that the variable name of the injected script has been modified, from import_meta_url => import_meta_url2 , which is strange problem. . .

Maybe replace --inject => --banner

 esbuild src/bin.ts --platform=node --outfile=dist/bin.cjs --define:import.meta.url=import_meta_url --bundle --sourcemap --banner:js="var import_meta_url = require('url').pathToFileURL(__filename)" --format=cjs

This will take effect

So, what about running esm bundle?
error also occurs

 throw new Error ( 'Dynamic require of "' + x + '" is not supported' ) Error : Dynamic require of "fs" is not supported

Modify the command as per the workaround here

 esbuild src/bin.ts --platform=node --outfile=dist/bin.esm.js --bundle --sourcemap --banner:js="import { createRequire } from 'module';const require = createRequire(import.meta.url);" --format=esm

Now, the bundled code can finally run.

Epilogue

Maybe esm only looks good, and tree shaking looks great, but right now, it’s not really available in production. Including a series of important projects are not migrated, including react/vscode/electron/vite and so on. In fact, before this, many people (and my generation) also used esm modules to write code, but the final bundle product may be cjs, such as iife in browsers, cjs in nodejs, but never Most application layer developers do not care about these, only lib maintainers will care, esm only transfers the complexity of using packages to users, and esm only packages referenced in cjs are not really available ‘s plan. Compared to projects like esbuild/vite that solve practical problems, the esm only movement is more like a carnival in the web front-end circle.

This article is reproduced from: https://blog.rxliuli.com/p/73331967c1814df480811eee598e714b/
This site is for inclusion only, and the copyright belongs to the original author.

Leave a Comment