-
Notifications
You must be signed in to change notification settings - Fork 16
Make all shell initialisations more robust #153
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ocaisa
wants to merge
20
commits into
EESSI:main
Choose a base branch
from
ocaisa:better_init
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+349
−71
Open
Changes from all commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
143341c
Make all shell initialisations more robust
ocaisa a428f68
Fix typo and tests
ocaisa 6b057e8
Merge branch 'main' into better_init
ocaisa c3d3159
See where the error is
ocaisa 1466e39
Merge branch 'better_init' of github.com:ocaisa/software-layer-script…
ocaisa 3692346
Add new capabilities and new tests
ocaisa 46c878f
Try to fix csg
ocaisa 3a6c117
csh has very short maximum length for envvar names, stick to <32 chars
ocaisa 785ae13
Some fancy new runners available!
ocaisa 61db941
Actually use the CPU override
ocaisa ec6f4e9
Merge branch 'main' into better_init
ocaisa dcce340
Use specific commit for assert.sh, also run tests on system with Lmod…
ocaisa 5e5ed24
Merge branch 'better_init' of github.com:ocaisa/software-layer-script…
ocaisa a97140a
Don't clobber an existing .cshrc
ocaisa d779b54
Add a sanity check to make sure module command works
ocaisa d91ded8
Set explicit MODULEPATH
ocaisa fc13853
No quotes on ~
ocaisa 7d621f1
Make test init script listen to EESSI_SOFTWARE_SUBDIR_OVERRIDE
ocaisa 78e286b
Remove copy/pasted comment
ocaisa 4d97256
Add licence and more extensive documentation
ocaisa File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| setenv("INSIDE_GITHUB_ACTIONS", "true") | ||
| -- Interfere with PATH so Lmod keeps a record | ||
| prepend_path("PATH", "/snap/bin") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| setenv("INSIDE_GITHUB_ACTIONS", "true") | ||
| -- Interfere with PATH so Lmod keeps a record | ||
| prepend_path("PATH", "/snap/bin") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,19 +1,83 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| # SPDX-License-Identifier: GPL-2.0-only | ||
| # Copyright (C) 2020-2026 EESSI contributors | ||
| # | ||
| # EESSI - European Environment for Scientific Software Installations | ||
| # | ||
| # This file is a template for initialising EESSI via Lmod for the environment indicated by the shebang. | ||
| # | ||
| # Assumptions: | ||
| # - EESSI accessible under $EESSI_CVMFS_REPO or /cvmfs/software.eessi.io | ||
| # - expected to be true as this file is also meant to be shipped under that location | ||
| # - Lmod must the shipped in the repository in the correct location | ||
| # | ||
| # Options: | ||
| # - The script listens to environment variables so that it can be silently configured by a site | ||
| # - EESSI_CVMFS_REPO: perform the initialisation from a respository other than /cvmfs/software.eessi.io | ||
| # - EESSI_VERSION: loads a specific EESSI version ignoring the default in the file | ||
| # - EESSI_DEFAULT_MODULES_PREPEND: environment variable allows you to prepend modules to the defaults (loaded last) | ||
| # - EESSI_DEFAULT_MODULES_APPEND: environment variable allows you to append modules to the defaults (loaded first) | ||
| # - EESSI_EXTRA_MODULEPATH: environment variable allows a site to append to MODULEPATH (lower priority than EESSI MODULEPATH) | ||
| # | ||
| # Other options that can be set to influence the end result: | ||
| # - The EESSI module also listens to environment variables so that it can be silently configured by a site | ||
| # - EESSI_MODULE_FAMILY_NAME: use the value of the environment variable to set an Lmod family for the EESSI module | ||
| # - EESSI_MODULE_STICKY: make the EESSI module sticky | ||
| # - EESSI_MODULE_UPDATE_PS1: have the EESSI module update PS1 to give a prompt that is prepended with "{EESSI/...} " | ||
| # | ||
| # Effects: | ||
| # - Should always succeed | ||
| # - Initialises Lmod from specific version of EESSI | ||
| # - Clobbers any existing Lmod configuration | ||
| # - Some special environment variables that are internal to Lmod (__LMOD_REF_COUNT_MODULEPATH and | ||
| # _ModuleTable001_) are used to force a hard reset of an existing Lmod installation. This | ||
| # approach may be brittle. | ||
| # - Then loads module EESSI/... to initialise EESSI | ||
| # - use `module show EESSI/...` to see the environment variables set by the EESSI module | ||
| # | ||
| # Reverting the effects: | ||
| # - EESSI initialisation via `module unload EESSI/...` | ||
| # - Lmod initialisation cannot be undone | ||
|
|
||
| # Choose an EESSI CVMFS repository | ||
| EESSI_CVMFS_REPO="${EESSI_CVMFS_REPO:-/cvmfs/software.eessi.io}" | ||
| # Choose an EESSI version | ||
| EESSI_VERSION_DEFAULT="__EESSI_VERSION_DEFAULT__" | ||
|
|
||
| # Choose an EESSI version (the default is only used if the EESSI_VERSION environment variable is not provided) | ||
| EESSI_VERSION_DEFAULT="${__EESSI_VERSION_USED_FOR_INIT:-__EESSI_VERSION_DEFAULT__}" | ||
| EESSI_VERSION="${EESSI_VERSION:-${EESSI_VERSION_DEFAULT}}" | ||
| # Path to top-level module tree | ||
| export MODULEPATH="${EESSI_CVMFS_REPO}/versions/${EESSI_VERSION}/init/modules" | ||
| . "${EESSI_CVMFS_REPO}/versions/${EESSI_VERSION}/compat/linux/$(uname -m)/usr/share/Lmod/init/bash" | ||
| # On the first run we want to record the EESSI version used for init as an environment variable so that if a different | ||
| # version of this script is called (e.g, for a a different EESSI version) it retains a memory which EESSI | ||
| # version was actually used in the initialisation. This is useful as __Init_EESSI_Default_Modules used below will | ||
| # be defined on the first call and Lmod initialisation will not happen twice. | ||
| # This sets the value only on first execution, if the variable already exists in the environment | ||
| # the original value is retained. | ||
| export __EESSI_VERSION_USED_FOR_INIT="${__EESSI_VERSION_USED_FOR_INIT:-${EESSI_VERSION}}" | ||
|
|
||
| # Ability to predefine elsewhere the default list of modules to load (colon separated): | ||
| # - EESSI_DEFAULT_MODULES_PREPEND environment variable allows you to prepend modules (loaded last) | ||
| # - EESSI_DEFAULT_MODULES_APPEND environment variable allows you to append modules (loaded first) | ||
| LMOD_SYSTEM_DEFAULT_MODULES="${EESSI_DEFAULT_MODULES_PREPEND:+$EESSI_DEFAULT_MODULES_PREPEND:}EESSI/$EESSI_VERSION${EESSI_DEFAULT_MODULES_APPEND:+:$EESSI_DEFAULT_MODULES_APPEND}" | ||
| export LMOD_SYSTEM_DEFAULT_MODULES | ||
|
|
||
| if [ -z "$__Init_EESSI_Default_Modules" ]; then | ||
| export __Init_EESSI_Default_Modules=1; | ||
|
|
||
| # Lmod version in 2023.06 has a problem with newer Lmod caches, so let's stick to more recent Lmod | ||
| # (has no effect except on Lmod itself, and compatible caches are still created/supported by EESSI) | ||
| LMOD_EESSI_VERSION=${EESSI_VERSION/2023.06/2025.06} | ||
|
|
||
| if [ -z "$__Init_Default_Modules" ]; then | ||
| export __Init_Default_Modules=1; | ||
| # Lmod may have been initialised so we need to clear some internal variables to allow for a full reset | ||
| # - make it forget about the system set MODULEPATH | ||
| unset __LMOD_REF_COUNT_MODULEPATH | ||
| # - and clear out any memory Lmod might have | ||
| unset _ModuleTable001_ | ||
|
|
||
| ## ability to predefine elsewhere the default list | ||
| LMOD_SYSTEM_DEFAULT_MODULES=${LMOD_SYSTEM_DEFAULT_MODULES:-"EESSI/$EESSI_VERSION"} | ||
| export LMOD_SYSTEM_DEFAULT_MODULES | ||
| module --initial_load --no_redirect restore | ||
| # Path to top-level module tree | ||
| # - EESSI_EXTRA_MODULEPATH environment variable allows a site to append to MODULEPATH (lower priority than EESSI MODULEPATH) | ||
| export MODULEPATH="${EESSI_CVMFS_REPO}/init/modules${EESSI_EXTRA_MODULEPATH:+:$EESSI_EXTRA_MODULEPATH}" | ||
| . "${EESSI_CVMFS_REPO}/versions/${LMOD_EESSI_VERSION}/compat/linux/$(uname -m)/usr/share/Lmod/init/bash" | ||
| module --initial_load --no_redirect restore | ||
| else | ||
| module refresh | ||
| module reset | ||
| fi |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,24 +1,70 @@ | ||
| #!/usr/bin/env csh | ||
|
|
||
| # SPDX-License-Identifier: GPL-2.0-only | ||
| # Copyright (C) 2020-2026 EESSI contributors | ||
| # | ||
| # EESSI - European Environment for Scientific Software Installations | ||
| # | ||
| # This file is a template for initialising EESSI via Lmod for the environment indicated by the shebang. | ||
| # | ||
| # Please refer to the reference bash implementation for full documentation, only minimal comments are included here | ||
|
|
||
| # Choose an EESSI CVMFS repository | ||
| if (! $?EESSI_CVMFS_REPO) then | ||
| if ( ! $?EESSI_CVMFS_REPO) then | ||
| set EESSI_CVMFS_REPO = "/cvmfs/software.eessi.io" | ||
| endif | ||
| # Choose an EESSI version | ||
| setenv EESSI_VERSION_DEFAULT "__EESSI_VERSION_DEFAULT__" | ||
| if (! $?EESSI_VERSION) then | ||
| set EESSI_VERSION = "${EESSI_VERSION_DEFAULT}" | ||
|
|
||
| # Choose an EESSI version (default is used if EESSI_VERSION is not provided) | ||
| if ( ! $?__EESSI_VERSION_USED_FOR_INIT ) then | ||
| set EESSI_VERSION_DEFAULT = "__EESSI_VERSION_DEFAULT__" | ||
| else | ||
| set EESSI_VERSION_DEFAULT = "$__EESSI_VERSION_USED_FOR_INIT" | ||
| endif | ||
| if ( ! $?EESSI_VERSION ) then | ||
| set EESSI_VERSION = "$EESSI_VERSION_DEFAULT" | ||
| endif | ||
| # On first run, record the EESSI version used for init as an environment variable. | ||
| # We use setenv to ensure it is available to child processes (equivalent to export). | ||
| if ( ! $?__EESSI_VERSION_USED_FOR_INIT ) then | ||
| setenv __EESSI_VERSION_USED_FOR_INIT "$EESSI_VERSION" | ||
| endif | ||
| # Path to top-level module tree | ||
| setenv MODULEPATH "${EESSI_CVMFS_REPO}/versions/${EESSI_VERSION}/init/modules" | ||
| source "${EESSI_CVMFS_REPO}/versions/${EESSI_VERSION}/compat/linux/`uname -m`/usr/share/Lmod/init/csh" | ||
|
|
||
| if (! $?__Init_Default_Modules ) then | ||
| setenv __Init_Default_Modules 1 | ||
| # ability to predefine elsewhere the default list (with options to append or prepend) | ||
| set LMOD_SYSTEM_DEFAULT_MODULES = "EESSI/${EESSI_VERSION}" | ||
| if ( $?EESSI_DEFAULT_MODULES_PREPEND ) then | ||
| set LMOD_SYSTEM_DEFAULT_MODULES = "${EESSI_DEFAULT_MODULES_PREPEND}:${LMOD_SYSTEM_DEFAULT_MODULES}" | ||
| endif | ||
| if ( $?EESSI_DEFAULT_MODULES_APPEND ) then | ||
| set LMOD_SYSTEM_DEFAULT_MODULES = "${LMOD_SYSTEM_DEFAULT_MODULES}:${EESSI_DEFAULT_MODULES_APPEND}" | ||
| endif | ||
| setenv LMOD_SYSTEM_DEFAULT_MODULES "${LMOD_SYSTEM_DEFAULT_MODULES}" | ||
|
|
||
| if ( ! $?__Init_EESSI_Default_Modules ) then | ||
| setenv __Init_EESSI_Default_Modules 1 | ||
|
|
||
| # ability to predefine elsewhere the default list | ||
| if (! $?LMOD_SYSTEM_DEFAULT_MODULES) then | ||
| setenv LMOD_SYSTEM_DEFAULT_MODULES "EESSI/${EESSI_VERSION}" | ||
| # Lmod version in 2023.06 has a problem with newer Lmod caches, so let's stick to more recent Lmod | ||
| # (has no effect except on Lmod itself, and compatible caches are still created/supported by EESSI) | ||
| set LMOD_EESSI_VERSION = "${EESSI_VERSION}" | ||
| if ( "${LMOD_EESSI_VERSION}" == "2023.06" ) then | ||
| set LMOD_EESSI_VERSION = "2025.06" | ||
| endif | ||
|
|
||
| # If there is a local Lmod, make it forget about the system set MODULEPATH | ||
| unsetenv __LMOD_REF_COUNT_MODULEPATH | ||
| # and clear out any memory Lmod might have | ||
| unsetenv _ModuleTable001_ | ||
| # Path to top-level module tree | ||
| set modulepath = "${EESSI_CVMFS_REPO}/init/modules" | ||
| if ( $?EESSI_EXTRA_MODULEPATH ) then | ||
| # Now that we know it exists, check IF it is not empty | ||
| if ( "$EESSI_EXTRA_MODULEPATH" != "" ) then | ||
| set modulepath = "${modulepath}:${EESSI_EXTRA_MODULEPATH}" | ||
| endif | ||
| endif | ||
| setenv MODULEPATH "$modulepath" | ||
| source "${EESSI_CVMFS_REPO}/versions/${LMOD_EESSI_VERSION}/compat/linux/`uname -m`/usr/share/Lmod/init/csh" | ||
|
|
||
| module --initial_load --no_redirect restore | ||
| else | ||
| module refresh | ||
| module reset | ||
| endif |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,18 +1,62 @@ | ||
| #!/usr/bin/env fish | ||
|
|
||
| # SPDX-License-Identifier: GPL-2.0-only | ||
| # Copyright (C) 2020-2026 EESSI contributors | ||
| # | ||
| # EESSI - European Environment for Scientific Software Installations | ||
| # | ||
| # This file is a template for initialising EESSI via Lmod for the environment indicated by the shebang. | ||
| # | ||
| # Please refer to the reference bash implementation for full documentation, only minimal comments are included here | ||
|
|
||
| # Choose an EESSI CVMFS repository | ||
| set EESSI_CVMFS_REPO (set -q EESSI_CVMFS_REPO; and echo "$EESSI_CVMFS_REPO"; or echo "/cvmfs/software.eessi.io") | ||
|
|
||
| # Choose an EESSI version | ||
| set EESSI_VERSION_DEFAULT "__EESSI_VERSION_DEFAULT__" | ||
| set EESSI_VERSION (set -q EESSI_VERSION; and echo "$EESSI_VERSION"; or echo "$EESSI_VERSION_DEFAULT") | ||
| # Path to top-level module tree | ||
| set -x MODULEPATH "$EESSI_CVMFS_REPO"/versions/"$EESSI_VERSION"/init/modules | ||
| . "$EESSI_CVMFS_REPO"/versions/"$EESSI_VERSION"/compat/linux/(uname -m)/usr/share/Lmod/init/fish | ||
|
|
||
| if test -z "$__Init_Default_Modules" | ||
| export __Init_Default_Modules=1; | ||
|
|
||
| ## ability to predefine elsewhere the default list | ||
| set -x LMOD_SYSTEM_DEFAULT_MODULES (set -q LMOD_SYSTEM_DEFAULT_MODULE; and echo "$LMOD_SYSTEM_DEFAULT_MODULE"; or echo "EESSI/$EESSI_VERSION") | ||
| module --initial_load --no_redirect restore | ||
| if not set -q __EESSI_VERSION_USED_FOR_INIT | ||
| set EESSI_VERSION_DEFAULT "__EESSI_VERSION_DEFAULT__" | ||
| else | ||
| set EESSI_VERSION_DEFAULT "$__EESSI_VERSION_USED_FOR_INIT" | ||
| end | ||
| if not set -q EESSI_VERSION | ||
| set EESSI_VERSION "$EESSI_VERSION_DEFAULT" | ||
| end | ||
|
|
||
| # Record version used for init; -x exports it to the environment | ||
| if not set -q __EESSI_VERSION_USED_FOR_INIT | ||
| set -x __EESSI_VERSION_USED_FOR_INIT "$EESSI_VERSION" | ||
| end | ||
|
|
||
| # ability to predefine elsewhere the default list (with options to append or prepend) | ||
| set LMOD_SYSTEM_DEFAULT_MODULES "EESSI/$EESSI_VERSION" | ||
| if set -q EESSI_DEFAULT_MODULES_PREPEND | ||
| set LMOD_SYSTEM_DEFAULT_MODULES "$EESSI_DEFAULT_MODULES_PREPEND:$LMOD_SYSTEM_DEFAULT_MODULES" | ||
| end | ||
| if set -q EESSI_DEFAULT_MODULES_APPEND | ||
| set LMOD_SYSTEM_DEFAULT_MODULES "$LMOD_SYSTEM_DEFAULT_MODULES:$EESSI_DEFAULT_MODULES_APPEND" | ||
| end | ||
| set -x LMOD_SYSTEM_DEFAULT_MODULES $LMOD_SYSTEM_DEFAULT_MODULES | ||
|
|
||
| if test -z "$__Init_EESSI_Default_Modules" | ||
| set -x __Init_EESSI_Default_Modules 1 | ||
|
|
||
| # Lmod version in 2023.06 has a problem with newer Lmod caches, so let's stick to more recent Lmod | ||
| # (has no effect except on Lmod itself, and compatible caches are still created/supported by EESSI) | ||
| set LMOD_EESSI_VERSION (string replace 2023.06 2025.06 -- $EESSI_VERSION) | ||
|
|
||
| # If there is a local Lmod, make it forget about the system set MODULEPATH | ||
| set -e __LMOD_REF_COUNT_MODULEPATH | ||
| # and clear out any memory Lmod might have | ||
| set -e _ModuleTable001_ | ||
| # Path to top-level module tree | ||
| set modulepath "$EESSI_CVMFS_REPO/init/modules" | ||
| if set -q EESSI_EXTRA_MODULEPATH; and test -n "$EESSI_EXTRA_MODULEPATH" | ||
| set modulepath "$modulepath:$EESSI_EXTRA_MODULEPATH" | ||
| end | ||
| set -x MODULEPATH $modulepath | ||
| . "$EESSI_CVMFS_REPO"/versions/"$LMOD_EESSI_VERSION"/compat/linux/(uname -m)/usr/share/Lmod/init/fish | ||
|
|
||
| module --initial_load --no_redirect restore | ||
| else | ||
| module refresh | ||
| module reset | ||
| end |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.