Start a Node project that uses at least five direct dependencies.
Leave it alone for three months.
Come back and try to install it.
Something in the dependency tree will yell at you that it is deprecated or discontinued. That thing will not be one of your direct dependencies.
NPM will tell you that you have at least one security vulnerability. At least one of the vulnerabilities will be impossible to trigger in your particular application. At least one of the vulnerabilities will not be able to be fixed by updating the versions of your dependencies.
(I am sure I exaggerate, but not by much!)
Why is it like this? How many hours per week does this running-to-stay-in-place cost the average Node project? How many hours per week of developer time is the minimum viable Node project actually supposed to have available?
My take: It’s because the “trust everything from everybody” model is fundamentally broken.
Note that trust is not only about avoiding malicious or vulnerable code, but also about dependability. Even if you ignore the “supply chain” security problems inherent in this model, it practically guarantees that the breakage you describe will happen eventually.
This is part of why I prefer languages with robust standard libraries, and why I am very picky about dependencies.
The fact that NPM can’t use multiple registries (yes, I know about scoped registries) is astounding. For every other language my org will separate artifacts into half a dozen or so virtual repos. The artifact team is quite annoyed that Node/JavaScript has to all go into one uber-repo.
I personally don’t trust NodeJS libraries that much - I always run projects inside Docker as a regular user with access to the working directory, just in case the supply chain is poisoned.
In the case of Python, particularly when I was testing out the LLaMA model, I just stood up a new VM for that. Back then safetensors wasn’t a thing, and the model tensor file format (python pickle-based) could potentially store arbitary code.
So many reasons. JS has a small standard library, a history of competing standards for things like asynchrony and modules, there are tons of different implementations against tons of different specs, running in tons of different environments (whose constraints and opportunities are also changing all the time), it tends to be the first language to receive an SDK for many services, packages tend to be almost-excessively granular because optimizing for size can be so important on certain platforms (tree-shaking and minification works, but takes time), there are many add-on languages like JSX and TS, there are tons of bundlers and transpilers which each have their own quirks… and also due to its unique position as the lingua franca of client-side web, it tends to be the primary battleground for researchers, tech firms, VC, FOSS, malicious users, and everything else.
To stay alive in an ecosystem like that, any project must become a “ship of theseus” kind of deal.
Something in the dependency tree will yell at you that it is deprecated or discontinued.
Only if you didn’t pinned the dependencies you actually consume, and expect that all your dependencies magically comply with semver.
Blindly replacing dependency versions never worked, at least reliably. If you do not put in the work to ensure things work, more often than not you’ll be surprised by them not working.
Then you’ll end up with tons of vulnerabilities within days.
You only end up with vulnerabilities if you refuse to update your dependencies. Updating a dependency is something you need to do yourself, along with running build validation tests after a successful update. Just because npm can download newer packages that does not mean those packages are good.
So, you still have to go through all the dependencies, vulnerabilities and incompatibilities. Yes, you can now kind of control the timing, but it’s still unacceptable in my opinion, that you have to spend so much time just to not be an active danger to everyone.
NPM is completely alone in that regard. Maven, Pypi, etc. don’t have these problems.
So, you still have to go through all the dependencies, vulnerabilities and incompatibilities.
That’s the point. There is no free lunch. Yet, you can live with stable, pinned versions and only upgrade the ones you really need to, and that means exercise your personal criteria instead of mindlessly upgrade everything just because there was a bump in a version number somewhere in an upstream dependency.
Yes, you can now kind of control the timing,
Timing, extent, scope, and impact. That’s the point.
but it’s still unacceptable in my opinion, that you have to spend so much time just to not be an active danger to everyone.
You only spend the time you decide to spend. There are already vulnerability scanners that pinpoint exactly which versions need to be updated, and do that automatically for you. Mindlessly opening the floodgate to each and every change is by definition idiocy and a liability.
Often it seems that people will make patch releases that add a “feature” of complaining at install time that that major release/minor release/entire package is bad now and should be replaced with something else. It still works, but it annoys everyone who transitively depends on it forevermore.
Welcome to my world. Not that I’m using node, but I’m using mediawiki. They manage to f-up something with about every update, and the documentation, if it exists at all, is often enough completely wrong or broken.
That is exactly the case here. I’ve got a private wiki with a rather large extension by now, and it is the only PHP project I have. So whenever mediawiki f-cks something up, which is nearly every update, I restart my PHP skills to find and fix the sh-t they did to my code this time.
For most projects or tools that I find on GitHub these days, I run them all in docker. Node, at least, is somewhat of a good guest. All it’s crap lives in node_modules, and so when I’m done, rm the directory and it’s all clean. Python seems to love leaving relics across my system
Also a great disadvantage with JavaScript being the official scripting language of the web so it needs to work in every browser, but not every browser will implement it the same way. Syntax between the browser and servers also gets conflated.
I’ve done frontend development in Java and it sucks major ass. The “advantage” of using one language for the whole project really ended up as a net negative. For any server-side project I’m totally fine using Java because for all it’s verbosity it’s secure, performant, and examples from 10 years ago aren’t useless.
Yes, that’s true, but JavaScript has very few core APIs aside from basic DOM manipulation. Even things like comparing timezones requires a third party dependency, for example.
Most of those are server-side languages and I’d disagree with the assessment. For web services Java needs some kind of server like Jetty, Undertow, or Tomcat. For testing you need JUnit or NGTest. And for common, everyday utilities you need something like Apache Commons or Guava. These things don’t “ship” with Java (and there’s actually a fair amount of runtimes now, it’s not only Oracle).
The thing that seems to benefit Java over Node is major corporate support (Oracle, RedHat, IBM) so for better or worse you can usually rely on a handful of essential tools being updated regularly.
I wouldn’t say you need no dependencies in a Java project, but by all means check the average number of dependencies you get with Java or Python and compare it to almost any Node project.
You could probably sample projects on GitHub, look at the dependency graph, and compare.
If you set the versions of your dependencies in the package.json file, you’ll reinstall exactly the same ones.
You may get new vulnerability warnings popping up, but what is the issue here ? You’d rather not be aware that a vulnerability has been found since your last development ?
If you are not happy with others modules, dev your own and no one will let you know about security issues 😝
If you mean computer science, then I’d say that webdevs have little in common with the industry that came up with stuff like ADA or functional correctness.
I’m struggling to understand how there can be so many security flaws, even in things that don’t seem to matter for security. I think the bar for a security problem might be too low; a lot of these look like footguns that could give my package a security hole, rather than genuine security flaws in the packages they are reported on.
Here’s a progress bar package with a “high” security vulnerability because it contains an internal utility that merges objects and doesn’t stop you writing to the prototype. Did the progress bar package ever promise to provide an object merge function that was safe to use on untrusted user input?
Here’s an arbitrary code execution vulnerability in sqlite3! High severity! The bug is that, if you tell sqlite3 to substitute an object into an SQL statement, it will run the ToString() method on the object. If an evil hacker has broken into your lead developer’s house and written a malicious ToString() method into one of the classes of object you use as a database query parameter, then that code would run! The fix here was, instead of letting the normal Javascript stringification rules apply, to hardcode all objects to be inserted into the database as “[object Object]”, because surely that is what the programmer meant to store.
The problem is that when you are alerted for trivial/non-actionable stuff it contributes to “alert-fatigue” and you just start ignoring all of the alerts.
As far as I’m aware, there’s also no way to triage an alert from an install other than to upgrade the offending package, which means you can’t really discriminate on the basis of “acceptable risk”
There’s a whole industry of bug bounty hunters making money off this trivial stuff. At work I had to fix a “bug” which could only be exploited if an attacker took control of facebook first, and even then it just meant a user could be redirected to a different website. And the company paid the clown that found the “vulnerability”.
I wonder if it has something to do with the framework you pick. There are probably frameworks that aim to be as stable as possible but lack the cutting edge.
They’re all that way: it’s just that Node is automatic enough to notice more easily, plus had an insane number of small dependencies
We started doing vulnerability scans on every build, which sounds like a good idea. However, now I know: Java is exactly the same. We need to constantly update but all too often there is no update available yet
In C# you can automatically generate (or manually write) binding redirects that let you say “anything using versions between x.y.0 and x.y.9 should use x.y.9”, which helps a lot with transitive dependencies. However, doing this manually is hard, and you can’t really rely on semver to be done “correctly.” This leads to subtle bugs. Occasionally not so subtle, but hard to diagnose.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !programming@programming.dev
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person’s post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
Follow the programming.dev instance rules
Keep content related to programming in some way
If you’re posting long videos try to add in some form of tldr for those who don’t want to watch videos
My take: It’s because the “trust everything from everybody” model is fundamentally broken.
Note that trust is not only about avoiding malicious or vulnerable code, but also about dependability. Even if you ignore the “supply chain” security problems inherent in this model, it practically guarantees that the breakage you describe will happen eventually.
This is part of why I prefer languages with robust standard libraries, and why I am very picky about dependencies.
What languages are those? And if you say C/C++ I’m going to laugh
The C++20 or so STL actually has things in it now.
It’s been better but still has a long way to go though, particularly regarding string handling
They still haven’t figured out string handling?
Thank god for proxy registries. I love Node, but it’s like using a house of cards as the foundation of a skyscraper.
The fact that NPM can’t use multiple registries (yes, I know about scoped registries) is astounding. For every other language my org will separate artifacts into half a dozen or so virtual repos. The artifact team is quite annoyed that Node/JavaScript has to all go into one uber-repo.
I personally don’t trust NodeJS libraries that much - I always run projects inside Docker as a regular user with access to the working directory, just in case the supply chain is poisoned.
In the case of Python, particularly when I was testing out the LLaMA model, I just stood up a new VM for that. Back then safetensors wasn’t a thing, and the model tensor file format (python pickle-based) could potentially store arbitary code.
Robust standard libraries are amazing to have.
So many reasons. JS has a small standard library, a history of competing standards for things like asynchrony and modules, there are tons of different implementations against tons of different specs, running in tons of different environments (whose constraints and opportunities are also changing all the time), it tends to be the first language to receive an SDK for many services, packages tend to be almost-excessively granular because optimizing for size can be so important on certain platforms (tree-shaking and minification works, but takes time), there are many add-on languages like JSX and TS, there are tons of bundlers and transpilers which each have their own quirks… and also due to its unique position as the lingua franca of client-side web, it tends to be the primary battleground for researchers, tech firms, VC, FOSS, malicious users, and everything else.
To stay alive in an ecosystem like that, any project must become a “ship of theseus” kind of deal.
You may be interested in yarn’s “zero-installs” option: https://yarnpkg.com/features/zero-installs
Only if you didn’t pinned the dependencies you actually consume, and expect that all your dependencies magically comply with semver.
Blindly replacing dependency versions never worked, at least reliably. If you do not put in the work to ensure things work, more often than not you’ll be surprised by them not working.
Then you’ll end up with tons of vulnerabilities within days. That can’t be the solution either.
You only end up with vulnerabilities if you refuse to update your dependencies. Updating a dependency is something you need to do yourself, along with running build validation tests after a successful update. Just because npm can download newer packages that does not mean those packages are good.
…which is the opposite of the comment above.
So, you still have to go through all the dependencies, vulnerabilities and incompatibilities. Yes, you can now kind of control the timing, but it’s still unacceptable in my opinion, that you have to spend so much time just to not be an active danger to everyone.
NPM is completely alone in that regard. Maven, Pypi, etc. don’t have these problems.
That’s the point. There is no free lunch. Yet, you can live with stable, pinned versions and only upgrade the ones you really need to, and that means exercise your personal criteria instead of mindlessly upgrade everything just because there was a bump in a version number somewhere in an upstream dependency.
Timing, extent, scope, and impact. That’s the point.
You only spend the time you decide to spend. There are already vulnerability scanners that pinpoint exactly which versions need to be updated, and do that automatically for you. Mindlessly opening the floodgate to each and every change is by definition idiocy and a liability.
Often it seems that people will make patch releases that add a “feature” of complaining at install time that that major release/minor release/entire package is bad now and should be replaced with something else. It still works, but it annoys everyone who transitively depends on it forevermore.
Welcome to my world. Not that I’m using node, but I’m using mediawiki. They manage to f-up something with about every update, and the documentation, if it exists at all, is often enough completely wrong or broken.
Been there, Mediawiki is truely painful, especially if you are trying to maintain an extension or two “on the side”.
That is exactly the case here. I’ve got a private wiki with a rather large extension by now, and it is the only PHP project I have. So whenever mediawiki f-cks something up, which is nearly every update, I restart my PHP skills to find and fix the sh-t they did to my code this time.
For most projects or tools that I find on GitHub these days, I run them all in docker. Node, at least, is somewhat of a good guest. All it’s crap lives in node_modules, and so when I’m done, rm the directory and it’s all clean. Python seems to love leaving relics across my system
Deno solves this problem :)
Because 99% of Node projects are web related, and if something’s vulnerable on the web it won’t take long to be abused
Also a great disadvantage with JavaScript being the official scripting language of the web so it needs to work in every browser, but not every browser will implement it the same way. Syntax between the browser and servers also gets conflated.
I’ve done frontend development in Java and it sucks major ass. The “advantage” of using one language for the whole project really ended up as a net negative. For any server-side project I’m totally fine using Java because for all it’s verbosity it’s secure, performant, and examples from 10 years ago aren’t useless.
JSP used to be the shit back in the day. Imagine server-side rendered html through Java. Nowadays it’s properly regarded as shit, fortunately.
I tought JSP was just one templating solution like mustache, jinja2, Thymeleaf etc. But guess theres more to it.
Fron-end development with Java? You mean Java applets or something?
Behold the horror of Vaadin: https://vaadin.com/docs/latest/guide/quick-start#add-code
Honestly looks like any other UI library/Framework for C#/Java.
However usually these elements are not instantiated/configured directly but from xml, yaml or json files that are built using some sort of GUI/Editor.
Google Web Toolkit was a thing at some point in time. ;y current company still maintains some apps which are written in it.
JSF for example. e.g. Primefaces https://www.primefaces.org/ I think there is something like Omnifaces too
Hey, a good answer! 🙌
Danke!
Prego!
At least part of it is that JavaScript is not really a batteries included language like Python or Java to even PHP.
You can’t really do anything productive without relying on a third party library.
All the languages you mentioned have some core APIs included and need to rely on third party libraries for more.
Yes, that’s true, but JavaScript has very few core APIs aside from basic DOM manipulation. Even things like comparing timezones requires a third party dependency, for example.
Most of those are server-side languages and I’d disagree with the assessment. For web services Java needs some kind of server like Jetty, Undertow, or Tomcat. For testing you need JUnit or NGTest. And for common, everyday utilities you need something like Apache Commons or Guava. These things don’t “ship” with Java (and there’s actually a fair amount of runtimes now, it’s not only Oracle).
The thing that seems to benefit Java over Node is major corporate support (Oracle, RedHat, IBM) so for better or worse you can usually rely on a handful of essential tools being updated regularly.
I wouldn’t say you need no dependencies in a Java project, but by all means check the average number of dependencies you get with Java or Python and compare it to almost any Node project.
You could probably sample projects on GitHub, look at the dependency graph, and compare.
This is not universally true. You totally can, especially today.
If you set the versions of your dependencies in the package.json file, you’ll reinstall exactly the same ones.
You may get new vulnerability warnings popping up, but what is the issue here ? You’d rather not be aware that a vulnerability has been found since your last development ?
If you are not happy with others modules, dev your own and no one will let you know about security issues 😝
I’d rather develop with dependencies that don’t have so many vulnerabilities.
Yeah we all wanna do that, but nobody writes perfect code.
That’s why people came up with defensive programming and functional correctness.
Just seems to be difficult for the webdev industry. Seems easier to push fixes from time to time.
Then you’re in the wrong industry…
If you mean web development, you’re right.
If you mean computer science, then I’d say that webdevs have little in common with the industry that came up with stuff like ADA or functional correctness.
I’m struggling to understand how there can be so many security flaws, even in things that don’t seem to matter for security. I think the bar for a security problem might be too low; a lot of these look like footguns that could give my package a security hole, rather than genuine security flaws in the packages they are reported on.
Here’s a progress bar package with a “high” security vulnerability because it contains an internal utility that merges objects and doesn’t stop you writing to the prototype. Did the progress bar package ever promise to provide an object merge function that was safe to use on untrusted user input?
Here’s a notification UI element that bills using HTML in your notification messages as a feature. It has a “medium” level “XSS” security vulnerability where the
messageparameter is not sanitized to remove HTML. A CVE was issued for this.Here’s an arbitrary code execution vulnerability in sqlite3! High severity! The bug is that, if you tell sqlite3 to substitute an object into an SQL statement, it will run the
ToString()method on the object. If an evil hacker has broken into your lead developer’s house and written a maliciousToString()method into one of the classes of object you use as a database query parameter, then that code would run! The fix here was, instead of letting the normal Javascript stringification rules apply, to hardcode all objects to be inserted into the database as “[object Object]”, because surely that is what the programmer meant to store.You don’t need to do anything when you get a security advisory if you’re not impacted or the risk is acceptable. They’re just there to advise you.
The problem is that when you are alerted for trivial/non-actionable stuff it contributes to “alert-fatigue” and you just start ignoring all of the alerts.
As far as I’m aware, there’s also no way to triage an alert from an install other than to upgrade the offending package, which means you can’t really discriminate on the basis of “acceptable risk”
There’s a whole industry of bug bounty hunters making money off this trivial stuff. At work I had to fix a “bug” which could only be exploited if an attacker took control of facebook first, and even then it just meant a user could be redirected to a different website. And the company paid the clown that found the “vulnerability”.
Or if you leave that alone, and update your package-lick.json so can install exact dependencies with
npm cideleted by creator
I wonder if it has something to do with the framework you pick. There are probably frameworks that aim to be as stable as possible but lack the cutting edge.
And these frameworks will fall out of fashion in three days.
They’re all that way: it’s just that Node is automatic enough to notice more easily, plus had an insane number of small dependencies
We started doing vulnerability scans on every build, which sounds like a good idea. However, now I know: Java is exactly the same. We need to constantly update but all too often there is no update available yet
C# is also exactly the same, you just don’t get yelled at when things are out of date and you only see that if you go to manually install packages
In C# you can automatically generate (or manually write) binding redirects that let you say “anything using versions between x.y.0 and x.y.9 should use x.y.9”, which helps a lot with transitive dependencies. However, doing this manually is hard, and you can’t really rely on semver to be done “correctly.” This leads to subtle bugs. Occasionally not so subtle, but hard to diagnose.
I think NPM has the same feature, it’s called overrides https://docs.npmjs.com/cli/v9/configuring-npm/package-json?v=true#overrides
It’s a job machine. Maintenance is easier than improving code quality or adding features.