Original link: http://claude-ray.com/2022/06/28/auditing_npm_packages/
Think about how long it has been since the project was created, how long have npm dependencies been upgraded?
How to know the “health” of the current project’s npm dependencies?
What are the precautions for upgrading npm dependencies for old projects?
core demands
- Improve maintainability. It is not easy to conflict with later-introduced dependencies. Introducing new features, the functional performance is close to the document description, and subsequent development can also be handy.
- Improve portability. It is convenient for old projects to migrate to higher versions of npm or pnpm.
- Improve reliability. As long as the dependencies are still in stable iteration, the upgrade will definitely introduce a series of bugfixes (but may also introduce new bugs).
- Improve security. The official community will promptly notify the security vulnerabilities that npm relies on, and keeping the version in the safe range can eliminate many hidden dangers.
Process method
- Use professional assessment tools. Manually upgrading @latest is equivalent to operating dependencies as a black box.
- Process by priority. Focus on upgrading core dependencies and libraries with security risks, otherwise the time investment can easily exceed expectations.
- Read the changelog to assess the impact of the upgrade.
- Regression testing is very important.
In addition to regression testing, those who lead governance must not only be familiar with the project content, but also have a solid understanding of the npm packages that are planned to be upgraded. If there is no suitable candidate, it is recommended to continue to persist in the code heap for a while. After all, the upgrade is risky, and the consequences are at your own risk.
search tool
The following uses npm as an example, pnpm and yarn have alternative commands.
outdated dependency npm outdated
The npm outdated command checks whether installed packages are outdated from the npm sources.
Just a few examples of packages:
1 |
Package Current Wanted Latest Location |
By default, only dependencies directly referenced in the project’s package.json are checked. The --all option can be used to match all dependencies. But it is not necessary. If you really want to upgrade completely, it is recommended to try to rebuild the lock file.
For outdated packages, use npm update or the update command corresponding to other package management tools to install the SemVer standard to perform the upgrade. If you want to span Major versions, you need to specify the upgrade version manually.
Risk depends on npm audit
npm audit command also makes a request to the npm source, which takes package-lock.json as a parameter and returns a list of dependencies with known vulnerabilities. In other words, audit can be executed without node_modules installed, and the result depends entirely on the current package-lock.json.
The return excerpt is as follows:
1 |
# Run npm install [email protected] to resolve 1 vulnerability |
If you find that the execution result is 404, it means that the current source does not support the audit interface, you can replace it with an official source that supports audit and execute it again.
1 |
npm http fetch POST 404 https://registry.npmmirror.com/-/npm/v1/security/audits 306ms |
Although the npm audit fix command is mentioned in the results, it is not always reliable, the number of dependencies it can fix is limited, and it is far less obvious than the number of indirect dependencies brought by upgrading root dependencies.
Implicit dependency on npx depcheck
The npm cli tool depcheck can help us find Unused dependencies (useless dependencies) and Phantom dependencies (phantom dependencies) in the project, which respectively indicate that they are written in package.json but not used by the project, and referenced by the project but not written in package.json.
depcheck is more like a filter to narrow down the scope of investigation, and its printed results cannot be trusted. For example, depcheck does not recognize specially mounted plugins by default.
1 |
Unused dependencies |
To delete a useless dependency, you must be familiar with the nature of the use of the npm package, and then confirm it repeatedly with the grep tool.
Zombies depend on npm install
Finally, beware of a Zombie dependencies (zombie dependencies). Unlike the implicit dependencies introduced earlier, it is very harmful.
First it is actually used by the project, but has been deprecated or archived by the maintainer. This means that the version is no longer updated, and the package name will not appear in the published list; most likely no bug has been reported, nor will it appear in the audited list. But the potential bug is not fixed, it will always be hidden in the project, waiting for an opportunity.
The author did not find a suitable tool to find zombie dependencies, so I had to pay more attention to the deprecated log of npm install.
Governance advice
HOW TO READ CHANGELOG
The changelog is usually located in CHANGELOG.md or History.md of the code repository. It may also be placed on the releases page of Github at will, and the official version will be placed in the Migrations category of the official website.
If you find that an npm package has no changelog, or the changelog is poorly written, It is recommended to switch to other more reliable alternatives , you can only rely on reading commits.
Keywords (welcome to add):
- BREAKING CHANGE
- !
- Node.js
Developers generally use the above method to mark incompatible changes.
lock file version management
This recommendation is for the development process of commercial software. Active open source scenarios do not require lock files, so that developers can find compatibility issues early in the process of iteration and testing.
The design document of package-lock.json bluntly recommends adding the lock file to the code repository:
- Ensure that team members and CI can use the exact same dependencies
- as a lightweight backup for node_modules
- Make changes to the dependency tree more visible
- Speed up the installation process
However, the strategy of npm dependency management varies by team and project. Whether to submit the lock file to the git repository can be chosen as needed, and there are many forms of version management.
For example, the research and development process is perfect, and the lock file released every time will remain in the product library or mirror, and can be restored at any time. However, if relevant measures are lacking, it is necessary to find a way to back up the lock file of the production environment to provide a basis for problem recurrence and fault recovery.
update hoisting
After years of updates, many versions of package-lock.json’s outer dependencies will lag behind child nodes, because npm currently does not rotate and deform the lock tree in order to maintain a minimum update range. Even if a newer project’s direct dependency is latest, its indirect dependency may still be old, so that the existing dependency boosting results deviate more and more from the default hoisting algorithm.
Some old projects can’t even install and build normally after they are separated from the package-lock.json file. At this point, the dependencies are already in a very unhealthy state. Developers need to worry about whether the newly introduced dependencies will break the balance, and they cannot migrate the npm package management tool, nor can they upgrade the Node.js version. But it’s not that complicated to fix it, it’s better than fixing a project without package-lock.json.
The easiest way to generate a reliable package-lock.json is to remove the old and welcome the new:
1 |
rm -rf package-lock.json node_modules |
A better solution is to switch to a dependency management tool that does not use hoisting.
The situation has been clarified, and when to rebuild can depend on your own needs. However, students who add the lock file to .gitignore should pay attention. If someone else has a problem that you cannot reproduce locally, remember to delete package-lock.json first.
Organize dependencies and devDependencies
The difference between dependencies and devDependencies in package.json does not need to be introduced, but will you make strict distinctions in the project?
First, devDependencies are designed to optimize dependencies for npm packages, and projects as applications are usually not packaged and released to npm; second, they do not distinguish and do not directly cause adverse consequences. Therefore, there are often small partners who install the tools that the development environment depends on directly into dependencies.
However, even for projects, devDependencies have positive implications:
- Semantically separate uses of dependencies
- Use
npm install --productionto ignore devDependencies, improve installation efficiency, and significantly reduce the size of node_modules
The second point needs to be supplemented. Since the build environment of static projects often needs to install most of the dependencies in devDependencies, generally only Node.js projects running on the server need to consider doing so. But with the popularity of TypeScript or the introduction of SSR, these server-side projects also need to perform a build before running. What’s the use of that? Don’t forget, there is also an npm prune --production that can be used as a post-project size optimization.
Of course, semantic partitioning is helpful enough, such as optimizing npm governance priorities and policies based on dependencies.
By the way, dependencies and devDependencies are not used to distinguish the importance. Please don’t put dependencies that are not necessary for running in devDependencies, but should be placed in optionalDependencies .
Epilogue
Most of the experience introduced above is an overview, mainly combined with the characteristics of npm dependency management tools, and failed to introduce the unique APIs and problems of tools such as yarn and pnpm. If readers want to know more, please refer to the relevant documents. In addition, projects that use multiple dependency management tools at the same time are quite complicated and relatively rare. This article does not analyze them, and does not recommend readers and friends to try. In the field of software engineering, there are still many points for dependency governance that we need to practice further, but the content focuses more on refactor.
Returning to the “health” of project dependencies mentioned in the slogan, it is actually a nonsense of the author, used to describe the chaotic degree of dependencies. It doesn’t matter if you don’t do these dependency governance, because the life cycle of software often doesn’t last until the day when the dependencies collapse. But messy dependency management can easily lead to code corruption.
This article is reprinted from: http://claude-ray.com/2022/06/28/auditing_npm_packages/
This site is for inclusion only, and the copyright belongs to the original author.