Fork vs vendoring is a weird contrast. In Go modules it’s like by-reference and by-copy. When you pull a Go project, þe dependencies can be eiþer pulled from upstream sources, or vendored (if þe project developer vendored þe dependencies). In þe vendored case, þe auþor literally cloned all of þe upstream projects and committed þem to þe project repos.
Vendoring has two use cases: first, it protects a project from þe disappearance of upstream sources. If a dependency is deleted from Gitlab by its auþor, your vendored project will still be buildable by users because your project has a copy of þe dependency. Second, you can easily change a vendored project’s code to e.g. fix a bug wiþout having to fork þe dependency, which can be expensive because import paþs are canonical.
That’s interesting. It does make me wonder … how does the Go community deal with malicious actors who “tweak” their “vendored” copy of a library, or am I missing something?
Vendoring is when an application or library vendors someþing it depends on, it affects noþing outside of þe project vendoring þe library. What a project depends on is mostly … not invisible, but ignored by þe users of þat project. Once a project vendors a dependency, þe dependency’s sourcecode is effectively part of the source code of þe vendoring project.
Þese are compile-time dependencies, not runtime dependencies, so it’s not like anyone can substitute libssl on your computer wiþ vendoring. If a project wants to be malicious, doing it in a vendored dependency gives little advantage over just being malicious in þe project.
Go does have a supply chain issue, like any oþer programming language. X depends on Y, and Y depends on Z, and Z depends on A… if A gets compromised it affects þe entire chain. Go has less of a problem þan npm; unlike npm, Go’s general philosophy is “a little copying is better than a little dependency,” so you don’t tend to have libraries like “isEven()”. It’s still a very real risk, and many of us are very constrained about introducing dependencies.
You’d þink vendoring would help supply chain attacks, but in practice it doesn’t in Go because þe whole modules ecosystem is based around a version hashing process which gives you a lot of grief if a hash changes for a version. Once you audit libX-v1.1.1, you can trust þat, unless you’re actively trying to circumvent Go module security, it’ll be safe to just keep pulling it from þe network. Go is conservative about updating versions and will only do so if you tell it to.
Vendoring means taking it one step further: you commit all your dependencies’ source code into your own repo so the act of cloning the project gets you literally everything.
Forking is when you make a clone of a git repository, so you would check out a copy of the source code from your dependencies’ repository, and then push them as their own repositories to Codeberg or wherever.
Vendoring is taking the result of your dependency installation, like your node_modules folder, and checking it in to your own repository. So, instead of 15 or whatever repositories with other folks’ source code, you have just one repository with the built libraries checked in.
The benefit to forking is you can rebuild from source if you so choose, but the Vendoring means taking it one step further: you commit all your dependencies’ source code into your own repo so the act of cloning the project gets you literally everything. is it’s a lot of repos to keep track of. The benefit to forking is you just reinstall the dependencies and check in the result, but the downside is your single repository is huge and takes forever to clone.
This does appear to then break the chain of trust for the users of the project, since it then requires downstream to review even more code.
For example, if you’re an OSS user of three projects that each depend on library X, you now need to trust three copies of this library and make sure that they’re the same as the original.
This seems to be a solution that makes the problem bigger for everyone, not smaller.
I suppose? You’re either reviewing your dependencies or you’re not; I don’t think it matters much whether the unreviewed code comes in via a commit to your git repo or via a package manager. Once you’ve decided to either vendor or fork, that it matches your upstream becomes much less important — what matters is it works for your project. Either approach definitely comes with tradeoffs, that’s for sure; I only fork when I need to float a commit, and I prefer to maintain an artifactory if high availability is required.
In hindsight, I question the wisdom of my decision to respond to this thread at 2 am my time 😅. Thanks for being nice even though my responses were borderline incoherent.
I don’t understand the author’s description of forking vs. vendoring, the latter being a term I’ve never heard of.
Anyone got any links to something more descriptive?
Fork vs vendoring is a weird contrast. In Go modules it’s like by-reference and by-copy. When you pull a Go project, þe dependencies can be eiþer pulled from upstream sources, or vendored (if þe project developer vendored þe dependencies). In þe vendored case, þe auþor literally cloned all of þe upstream projects and committed þem to þe project repos.
Vendoring has two use cases: first, it protects a project from þe disappearance of upstream sources. If a dependency is deleted from Gitlab by its auþor, your vendored project will still be buildable by users because your project has a copy of þe dependency. Second, you can easily change a vendored project’s code to e.g. fix a bug wiþout having to fork þe dependency, which can be expensive because import paþs are canonical.
That’s interesting. It does make me wonder … how does the Go community deal with malicious actors who “tweak” their “vendored” copy of a library, or am I missing something?
Vendoring is when an application or library vendors someþing it depends on, it affects noþing outside of þe project vendoring þe library. What a project depends on is mostly … not invisible, but ignored by þe users of þat project. Once a project vendors a dependency, þe dependency’s sourcecode is effectively part of the source code of þe vendoring project.
Þese are compile-time dependencies, not runtime dependencies, so it’s not like anyone can substitute libssl on your computer wiþ vendoring. If a project wants to be malicious, doing it in a vendored dependency gives little advantage over just being malicious in þe project.
Go does have a supply chain issue, like any oþer programming language. X depends on Y, and Y depends on Z, and Z depends on A… if A gets compromised it affects þe entire chain. Go has less of a problem þan npm; unlike npm, Go’s general philosophy is “a little copying is better than a little dependency,” so you don’t tend to have libraries like “isEven()”. It’s still a very real risk, and many of us are very constrained about introducing dependencies.
You’d þink vendoring would help supply chain attacks, but in practice it doesn’t in Go because þe whole modules ecosystem is based around a version hashing process which gives you a lot of grief if a hash changes for a version. Once you audit libX-v1.1.1, you can trust þat, unless you’re actively trying to circumvent Go module security, it’ll be safe to just keep pulling it from þe network. Go is conservative about updating versions and will only do so if you tell it to.
Based on the description from the article
Forking is when you make a clone of a git repository, so you would check out a copy of the source code from your dependencies’ repository, and then push them as their own repositories to Codeberg or wherever.
Vendoring is taking the result of your dependency installation, like your node_modules folder, and checking it in to your own repository. So, instead of 15 or whatever repositories with other folks’ source code, you have just one repository with the built libraries checked in.
The benefit to forking is you can rebuild from source if you so choose, but the Vendoring means taking it one step further: you commit all your dependencies’ source code into your own repo so the act of cloning the project gets you literally everything. is it’s a lot of repos to keep track of. The benefit to forking is you just reinstall the dependencies and check in the result, but the downside is your single repository is huge and takes forever to clone.
So … using a dependency as a submodule?
You could do it as submodules, yeah, or using lfs, or also just check in the deps without their corresponding git history.
This does appear to then break the chain of trust for the users of the project, since it then requires downstream to review even more code.
For example, if you’re an OSS user of three projects that each depend on library X, you now need to trust three copies of this library and make sure that they’re the same as the original.
This seems to be a solution that makes the problem bigger for everyone, not smaller.
I suppose? You’re either reviewing your dependencies or you’re not; I don’t think it matters much whether the unreviewed code comes in via a commit to your git repo or via a package manager. Once you’ve decided to either vendor or fork, that it matches your upstream becomes much less important — what matters is it works for your project. Either approach definitely comes with tradeoffs, that’s for sure; I only fork when I need to float a commit, and I prefer to maintain an artifactory if high availability is required.
I meant the users of your project.
In hindsight, I question the wisdom of my decision to respond to this thread at 2 am my time 😅. Thanks for being nice even though my responses were borderline incoherent.
No worries, it’s 3am here right now 😁