To manage the complexity we have entire teams dedicated to aspects of it, and an insanely complex build system that makes use of remote builders and caches. A change to a single python file requires about fifteen seconds of the build system determining it needs to do no work, with all of this caching, and the cache can be invalidated unexpectedly and take twenty minutes instead. Ordinary language features in ides are routinely broken, I assume because of the difficulty of maintaining an index of that stuff on such a huge codebase. Ordinary tools like grep -R or find can only be used with care.
On the other hand, using ordinary tools like find and grep are exactly what I like about monorepos! Yes, they may take a while, but at least I know I’ll find a file or code that I’m looking for!
With multi-repos I’m constantly searching, but not finding where a particular piece of code comes from. Yes, it’s from library X, but where there heck does that live? Now I really can’t use ordinary tools. I have to rely on coworkers, docs, or GitLab to search for where a piece of code is actually defined.
Yeah, I’m sure. It’s not something I would do frequently. My work had us on beefy desktops. But, I was totally fine with letting find+parallel+grep run for 30 minutes in the background while I searched docs or messaged people on slack. Depending on your team, getting a response from slack could easily take 24 hours so. Eh.
The other thing I liked to do is directly edit the libraries in the monorepo! No need to figure out how hack some random decency manager. You have the code! Just edit and build!
consider looking at ripgrep and semantic search tools.
i maintain a gigantic monorepo using policies, cicd gating, pre-commit, and lots of scripting. its not unworkable, just takes process. i don’t really agree with the creators pov. but i guess i may not be a new engineer entering a team with a monolith and a bone.
I don’t see how ripgrep would help with the monorepo situation. We have tooling for an equivalent of grep, but it’s based on an index, not what’s on your filesystem.
IIRC, ripgrep used a faster algorithm than grep, but more recent versions of grep are now shipping with a faster (the same?) algorithm. So the above suggestion shouldn’t help much unless you use a very old grep.
We have a gigantic monorepo at work.
To manage the complexity we have entire teams dedicated to aspects of it, and an insanely complex build system that makes use of remote builders and caches. A change to a single python file requires about fifteen seconds of the build system determining it needs to do no work, with all of this caching, and the cache can be invalidated unexpectedly and take twenty minutes instead. Ordinary language features in ides are routinely broken, I assume because of the difficulty of maintaining an index of that stuff on such a huge codebase. Ordinary tools like grep -R or find can only be used with care.
On the other hand, using ordinary tools like find and grep are exactly what I like about monorepos! Yes, they may take a while, but at least I know I’ll find a file or code that I’m looking for!
With multi-repos I’m constantly searching, but not finding where a particular piece of code comes from. Yes, it’s from library X, but where there heck does that live? Now I really can’t use ordinary tools. I have to rely on coworkers, docs, or GitLab to search for where a piece of code is actually defined.
there is a size of monorepo where that becomes infeasible ;)
Yeah, I’m sure. It’s not something I would do frequently. My work had us on beefy desktops. But, I was totally fine with letting find+parallel+grep run for 30 minutes in the background while I searched docs or messaged people on slack. Depending on your team, getting a response from slack could easily take 24 hours so. Eh.
The other thing I liked to do is directly edit the libraries in the monorepo! No need to figure out how hack some random decency manager. You have the code! Just edit and build!
consider looking at ripgrep and semantic search tools.
i maintain a gigantic monorepo using policies, cicd gating, pre-commit, and lots of scripting. its not unworkable, just takes process. i don’t really agree with the creators pov. but i guess i may not be a new engineer entering a team with a monolith and a bone.
I don’t see how ripgrep would help with the monorepo situation. We have tooling for an equivalent of grep, but it’s based on an index, not what’s on your filesystem.
IIRC, ripgrep used a faster algorithm than grep, but more recent versions of grep are now shipping with a faster (the same?) algorithm. So the above suggestion shouldn’t help much unless you use a very old grep.