stacks: use environment with groups #3175
Conversation
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
alalazo
requested review from
alecbcs,
becker33,
tgamblin and
tldahlgren
as code owners
alalazo
added
the
pipelines:urgent
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
stacks/tutorial/spack.yaml
Outdated
|
|
||
| - group: "gcc@11 specs" | ||
| specs: | ||
| - $gcc_system_packages |
There was a problem hiding this comment.
maybe inline those? the group already has a name
| - $gcc_system_packages | ||
| - group: compiler | ||
| specs: | ||
| - gcc@12 |
There was a problem hiding this comment.
| - gcc@12 | |
| - gcc@12.3 |
was on purpose
There was a problem hiding this comment.
Wasn't the purpose to have the same version that we would get from:
$ apt install gcc-12
on Ubuntu 22.04? Do we still need that now that compilers are dependencies?
There was a problem hiding this comment.
As far as I'm aware we'd used apt before compilers as dependencies to bootstrap gcc@12 for the environment. With groups we can drop the usage of apt and use a spack build gcc@12 to more closely mirror the steps in the tutorial.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Use the group feature to have a single spack.yaml for radiuss. Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
alalazo
requested review from
kwryankrattiger and
zackgalbreath
as code owners
| - openmpi | ||
|
|
||
| concretizer: | ||
| unify: false |
There was a problem hiding this comment.
@adrienbernede Is there any reason why we use unify:false for radiuss on the cpu? Can it be changed to unify:true?
There was a problem hiding this comment.
Here are my concerns:
This stack is not meant to be built together (e.g. as dependencies of a unique project), so it may over-constrain the specs and fail.
Could this result in building only older versions of certain packages ?
E.g. if A@latest requires B@latest-1, then B@latest will not be built in the stack, correct ?
There was a problem hiding this comment.
Could this result in building only older versions of certain packages ?
Yes, that might happen. In the case you make there will be just B@<latest-1>. I'll keep unify:false then.
There was a problem hiding this comment.
Another question. Do you have any concern if the same specs that are now in radiuss-cuda would be built in a single pipeline together with the specs of radiuss (that's what this PR is experimenting with)?
There was a problem hiding this comment.
My only concern is that it will not help to decipher where a failing job comes from. What are the benefits of merging the pipelines ?
There was a problem hiding this comment.
Less duplicate builds, less runners used, faster pipelines. Also, since the spack.yaml is unique also more consistency in the configuration for the common parts.
There was a problem hiding this comment.
OK, I vote the benefits outweighs the drawbacks :)
There was a problem hiding this comment.
To make it clear:
Yes to merging pipelines, but no to unify:true to ensure we build the latest release of each package.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
There was a problem hiding this comment.
Currently don't see a huge advantage since we still end up duplicating all config. What I was hoping for in spack/spack#32893 and spack/spack#32894 was the ability to create a cross-product/matrix of:
- packages
- variants
- arches
Ideally, these would also reuse shared dependencies instead of building the same non-GPU Python libs 3x.
5 participants
Meant to try spack/spack#51891