
Thoughts about package management
Prior art
- npm
- conan
- vcpkg
- pip
- hex
- cargo
- opam
- hackage
- elm
- ...and countless system package managers
How can we do better?
We have a lot of opportunities to learn!
Known problems
The leftpad incident
The leftpad incident
- Don't let people delete things
- Like, ever
- Seriously
- ...but also faker (wide-spread DDoS attacks)
- ...and also node-ipc (literal cyber-warfare)
Managing upgrades
- "lock" files
- Supply chain security
- What if we need incompatible versions?
Not getting along with the in-laws
- What if I like my dependency, but I don't like their dependencies?
- License conflicts
- Missing features
- Lack-luster performance
- Disagreements
Wasted cycles
GET https://npmjs.com/leftpad/-/leftpad-1.0.0.tar.gz
ungzip
tar -x
- A bunch of files I don't care about
- LICENSE
- package.json
- CODE_OF_CONDUCT.md, CHANGELOG.md, README.md, .github/ (!?!?), .editorconfig (!?!?)
- Compilation artifacts
- A bunch of files I don't care about
...and now do this hundreds thousands of times
Wasted Dangerous cycles
postinstall
,prepare
, etc.- setup.py
- build.rs
- ...build.zig? 💀
Potential solutions
Don't have a package manager
Package managers have good bits
- Code reuse
- Just build the thing that you want to build
- Working together
Avoiding dangerous cycles
- How about we just don't execute arbitrary code in the package manager/during builds? 😅
- Come up with alternative ways to describe behaviors, rather than letting anyone do anything
- At least ask the user if it's okay to just run stuff
Avoiding wasted cycles
Avoiding the rest...
- What if dependencies couldn't have dependencies?
How would that work?
test "simple test" {
var list = std.ArrayList(i32).init(std.testing.allocator); // <-- This line here!
defer list.deinit();
try list.append(42);
try std.testing.expectEqual(@as(i32, 42), list.pop());
}
Dependency Injection
var client = requestz.Client.init(
std.testing.allocator,
);
Dependency Injection
var client = requestz.Client.init(
std.testing.allocator,
std.net.tcpConnectToAddress, // <-- What if we could provide our own TCP logic!
);
Dependency Injection
var client = requestz.Client.init(
std.testing.allocator,
.{
// ✨ fancy HTTP/3 stuff here ✨
},
);
Benefits!
- No lock files
- No conflicts
- Much, much simpler package manager
- No need for weird things like "peer dependencies"
- All of your dependencies become explicit
- You own the glue code!
- Flexibility and testing
comptime
means we can have this flexibility without runtime cost
I don't actually think entirely disallowing packages to have dependencies is the right approach
...but I think these kinds of things are important to keep in mind as the package ecosystem develops for Zig.
Less dramatic idea speed-round ⏱️
- Maybe don't just trust random peoples code blindly
- Moderation?
- Moderation, only for packages that require ACE as an installation strategy?
- Moderation, only for unverified authors?
- No moderation, but author verification?
- What if adding a new dependency was a breaking change?
- What if adding a new maintainer was a breaking change?
- What if upgrading was more interactive?
- Maybe
zig upgrade
gives you a decision tree...- "This major version is a breaking change because..."
- "Hayleigh has been added as a maintainer"
- Maybe
It's a culture thing.
- It's up to everyone to cultivate an ecosystem where we use dependencies responsibly
- ...where we can trust and enjoy the dependencies we get to use
- ...where we can enjoy creating new libraries for others
- ...where we don't have over complicated, prolific libraries that replicate a single function from the standard library :^)
If you're writing a package, keep the language you are working with in mind. A language that focuses so heavily on low-level control deserves a package ecosystem that gives you the same kind of respect.