I’ve got a non trivial project in Rust, and it takes like 5 min to compile on my machine. Personally, I don’t know how anybody can call this fun. I find it insane to have to wait minutes to see the changes and to iterate. And like sure you can break shit up into crates to speed up compilation, but to do that you already have to have a design you’re happy with and that’s stable.
100k loc without a reasonable architecture is… problematic.
I have run into problems with C++ linker/optimizers which will choke on big (50k loc) .cpp files which are auto-generated from definitions, they get slow like that, build times around 20 minutes for the worst of them. Keep the files more like 10k loc and down if you don’t want this kind of issue.
Nowhere did I say there’s no reasonable architecture. Please don’t put words in my mouth. What I said, is that I haven’t found a good way to get quick compiles end to end. And already I don’t have any files more than 3-4k loc, most are under 1k.
Yeah I doubt it. I’ve worked in large Rust codebases too. I’d do some benchmarking. Cargo has some nice flags that let you see exactly what it’s doing. You can open the compilation report up in a browser and zoom in on different steps of the process and see everything.
Make sure you’re using a fast linker. Although I think lld is the default now, so that should be less of a concern these days. But see if mold or wild would help.
More relevantly, you should be using cranelift for codegen in dev builds.
If you have codegen-units = 1 in your release profile, make sure you have a separate dev profile that doesn’t.
If you still experience decapitating slowness (doubtful), and it’s actually the rust compiler being slow (super doubtful), you can try the parallel frontend. Apparently there are almost™ no bugs left in its implementation (It’s one of the project priorities now).
Needless to say, but you should be checking your --timings to really know what’s going on.
I find it insane to have to wait minutes to see the changes and to iterate.
i did devops between 2015 & 2025 and got used to the cadence of waiting between 15 & 180 minutes for testing/production pipelines to finish vetting the work i submitted to them.
i went back to doing IT last year and setup similar pipelines to update the code base my predecessor left behind and my new boss expressed the same consternation about waiting for your changes to iterate.
i’m thankful for it because it gives me 30-ish minute windows to browse and annoy people on lemmy throughout the day. lol
back around 2010, i did IT at a place that got a new isilon cluster to alleviate storage space woes with our netapps and the software engineers used this comic to make fun of the work we did when it caused a bottleneck over the network. lol
Gosh. I remember working with this back in 2019 and earlier. Our legacy products ran on it. I remember a new service we made (before Spring Boot really took off) used TomCat. It was so much easier to mess with. I asked why we don’t use that for our main product. Imagine my surprise when they said we actually used to.
I’m sure they had their reasons, I was a younger dev at the time and didn’t have insight into why they changed it. But still. Everything is so simple now. Java to run your jar. Your jar has your server built in. Done. And that’s not even including containerization.
I do miss Jenkins though. GitHub Actions seems to be the new hotness. Maybe I just don’t have the muscle memory yet, but it can be annoying.
Yeah, not gonna miss app servers. I do find the JVM was kind of made for a different era though. It’s basically designed to act like a VM on top of which all your apps run. So, startup time isn’t really a problem, and it wants to grab as much memory as it can by default. But nowadays everybody just makes small self contained apps that you can scale horizontally, so this whole model the JVM is tailored for isn’t really used outside big enterprise. And Jenkins was alight, I used to use it back in the day too. For the most part, I do find GitHub actions are an improvement though. You just make a script and magic happens.
5 minutes sounds like way too much, unless you mean a fresh compile. But then you shouldn’t need to wait that long between changes, since incremental compilation should kick in then.
And like sure you can break shit up into crates to speed up compilation, but to do that you already have to have a design you’re happy with and that’s stable.
I mean, if you have your modules structured in a tree structure and with proper visibility, then it isn’t a particularly big leap to put it into a separate crate. You just move the files, maybe fix some visibility modifiers still, and then a bit of boilerplate to add it to the workspace.
It’s only really when you’re publishing to crates.io, that you don’t particularly want to keep changing the names/scopes of the crates, as they’ll stick around on there for the foreseeable future.
5m seems about right. Lemmy’s from scratch builds are ~3.5m on a fast machine, and we have ~60k lines of code, and are using some with large libraries with lots of features enabled.
But you really should only ever have to do a from scratch build either at the beginning, or when you deploy. When developing, your IDE should only ever really run check or clippy, which should take seconds at most.
I recall looking at mold, but it didn’t end up helping much in my case. And haven’t looked at sccache, that might actually help if it can cache compilation incrementally in an intelligent way.
Or maybe your architecture evolves over time as it actually does in real world large projects. Sounds like somebody’s never worked on a large long running project before.
I’ve got a pretty large Rust/Tauri program going. Breaking things into crates help, but if you touch enough bits of common libraries, everything needs to get rebuilt anyway.
I use Makefiles for everything. What I found helps is breaking out the parts that don’t change much into separate Make commands (like generating icons, linting, frontend, tests, etc), then leaving “make go” as the simplest inner iteration loop. That way, it’s a lot more bearable.
There’s also “make build” and “make fullclean.” You start one of those and take a long break.
Right, you basically have to start building a rube goldberg machine for compiling your app just so you don’t have to wait for minutes on end. And to me it is absolutely bananas that you can make a compiler this slow on modern hardware. Coming from largely working with Clojure where I have a live application I can interactively load code into as I edit it, this whole process feels insane.
I’ve got a non trivial project in Rust, and it takes like 5 min to compile on my machine. Personally, I don’t know how anybody can call this fun. I find it insane to have to wait minutes to see the changes and to iterate. And like sure you can break shit up into crates to speed up compilation, but to do that you already have to have a design you’re happy with and that’s stable.
Five minutes with build cache? Or from cold build?
With cache, then there’s almost certainly something wrong with your computer or project.
Also… you aren’t building with --release every time are you?
No, I’m not using --release every time of course. And sometimes it’s minutes even with the cache.
That’s highly abnormal to have minute long debug build times.
for 100k loc?
100k loc without a reasonable architecture is… problematic.
I have run into problems with C++ linker/optimizers which will choke on big (50k loc) .cpp files which are auto-generated from definitions, they get slow like that, build times around 20 minutes for the worst of them. Keep the files more like 10k loc and down if you don’t want this kind of issue.
Nowhere did I say there’s no reasonable architecture. Please don’t put words in my mouth. What I said, is that I haven’t found a good way to get quick compiles end to end. And already I don’t have any files more than 3-4k loc, most are under 1k.
Yeah I doubt it. I’ve worked in large Rust codebases too. I’d do some benchmarking. Cargo has some nice flags that let you see exactly what it’s doing. You can open the compilation report up in a browser and zoom in on different steps of the process and see everything.
I’ve tried looking before and couldn’t get a significant improvement. I’ll have to investigate it again I guess.
Make sure you’re using a fast linker. Although I think
lldis the default now, so that should be less of a concern these days. But see ifmoldorwildwould help.More relevantly, you should be using cranelift for codegen in dev builds.
If you have
codegen-units = 1in yourreleaseprofile, make sure you have a separate dev profile that doesn’t.If you still experience decapitating slowness (doubtful), and it’s actually the rust compiler being slow (super doubtful), you can try the parallel frontend. Apparently there are almost™ no bugs left in its implementation (It’s one of the project priorities now).
Needless to say, but you should be checking your
--timingsto really know what’s going on.thanks will see what happens
The various linkers do make a difference, but all of them choke, eventually, when individual files get too massive.
i did devops between 2015 & 2025 and got used to the cadence of waiting between 15 & 180 minutes for testing/production pipelines to finish vetting the work i submitted to them.
i went back to doing IT last year and setup similar pipelines to update the code base my predecessor left behind and my new boss expressed the same consternation about waiting for your changes to iterate.
i’m thankful for it because it gives me 30-ish minute windows to browse and annoy people on lemmy throughout the day. lol
https://xkcd.com/303/
this comic will forever make me flinch. lol
back around 2010, i did IT at a place that got a new isilon cluster to alleviate storage space woes with our netapps and the software engineers used this comic to make fun of the work we did when it caused a bottleneck over the network. lol
haha this reminds me of my days working with websphere :)
indeed
Gosh. I remember working with this back in 2019 and earlier. Our legacy products ran on it. I remember a new service we made (before Spring Boot really took off) used TomCat. It was so much easier to mess with. I asked why we don’t use that for our main product. Imagine my surprise when they said we actually used to.
I’m sure they had their reasons, I was a younger dev at the time and didn’t have insight into why they changed it. But still. Everything is so simple now. Java to run your jar. Your jar has your server built in. Done. And that’s not even including containerization.
I do miss Jenkins though. GitHub Actions seems to be the new hotness. Maybe I just don’t have the muscle memory yet, but it can be annoying.
Yeah, not gonna miss app servers. I do find the JVM was kind of made for a different era though. It’s basically designed to act like a VM on top of which all your apps run. So, startup time isn’t really a problem, and it wants to grab as much memory as it can by default. But nowadays everybody just makes small self contained apps that you can scale horizontally, so this whole model the JVM is tailored for isn’t really used outside big enterprise. And Jenkins was alight, I used to use it back in the day too. For the most part, I do find GitHub actions are an improvement though. You just make a script and magic happens.
5 minutes sounds like way too much, unless you mean a fresh compile. But then you shouldn’t need to wait that long between changes, since incremental compilation should kick in then.
I mean, if you have your modules structured in a tree structure and with proper visibility, then it isn’t a particularly big leap to put it into a separate crate. You just move the files, maybe fix some visibility modifiers still, and then a bit of boilerplate to add it to the workspace.
It’s only really when you’re publishing to crates.io, that you don’t particularly want to keep changing the names/scopes of the crates, as they’ll stick around on there for the foreseeable future.
It’s around 100k loc sized project, so from what I’ve seen that’s about what you can expect with Rust.
5m seems about right. Lemmy’s from scratch builds are ~3.5m on a fast machine, and we have ~60k lines of code, and are using some with large libraries with lots of features enabled.
But you really should only ever have to do a from scratch build either at the beginning, or when you deploy. When developing, your IDE should only ever really run
checkorclippy, which should take seconds at most.I should really revisit this. I know I looked at it again. The compiling crops up I you actually want to run the app and test drive the changes live.
100k lines and you don’t yet have a design that’s mostly settled?
Yes, I move things around.
mold and sccache help a lot with the compile times
I recall looking at mold, but it didn’t end up helping much in my case. And haven’t looked at sccache, that might actually help if it can cache compilation incrementally in an intelligent way.
Or, you can modularize your architecture before you start implementation…
Do you even write requirements ahead of the code, bro?
Or maybe your architecture evolves over time as it actually does in real world large projects. Sounds like somebody’s never worked on a large long running project before.
I’ve got a pretty large Rust/Tauri program going. Breaking things into crates help, but if you touch enough bits of common libraries, everything needs to get rebuilt anyway.
I use Makefiles for everything. What I found helps is breaking out the parts that don’t change much into separate Make commands (like generating icons, linting, frontend, tests, etc), then leaving “make go” as the simplest inner iteration loop. That way, it’s a lot more bearable.
There’s also “make build” and “make fullclean.” You start one of those and take a long break.
Right, you basically have to start building a rube goldberg machine for compiling your app just so you don’t have to wait for minutes on end. And to me it is absolutely bananas that you can make a compiler this slow on modern hardware. Coming from largely working with Clojure where I have a live application I can interactively load code into as I edit it, this whole process feels insane.