A defense-in-depth analysis of a PATH sanitization flaw in SingularityCE OCI runtime discovery.
Overviewh2
SingularityCE OCI mode contained a design issue related to executable path resolution. The runtime discovery logic did not sanitize the PATH environment variable before calling exec.LookPath, which allowed untrusted binaries in user-controlled directories to potentially be selected and executed as root.
This issue only manifests when SingularityCE is run with elevated privileges (via sudo or setuid), which is a necessary precondition for exploitation in most scenarios.
Root Causeh2
The flaw is in internal/pkg/util/bin/bin_singularity.go, inside the findOnPath implementation.
Instead of restricting the search to trusted system directories, the code prepended the current environment PATH to the default search path and then called:
os.Setenv("PATH", oldPath+":"+env.DefaultPath)path, err = exec.LookPath(name)Because exec.LookPath searches from left to right, any malicious runtime binary placed in an untrusted location such as /tmp could be found and executed before the system binary.
This is a classic untrusted search path vulnerability (CWE-426). A privileged process should never inherit untrusted execution order from a low-privileged environment.
Impacth2
This design issue could potentially arise in specific scenarios where SingularityCE is launched with elevated privileges:
- Restricted sudo configurations: If a user is permitted via sudoers to run
sudo singularity oci createor similar operations, they could theoretically poisonPATHto cause a maliciousrunc/crunbinary to be executed as root. However, this requires explicit sudo permissions granted by an administrator. - SUID installations: If SingularityCE is installed with the setuid bit, unprivileged users could theoretically attempt the same path hijacking. This is a less common deployment model.
- Automated systems: Headless systems running SingularityCE in OCI mode as root (cron jobs, CI/CD runners) could inherit contaminated
PATHvalues from their environment.
It is important to note that SingularityCE is fundamentally designed to accept root privileges when operating in OCI mode. The issue is not the privilege requirement itself, but rather the execution path design during privilege context.
Why This Mattersh2
While SingularityCE is not intended to prevent host modification when run as host-root, there is a valid defense-in-depth argument for improving execution path safety.
In this case, SingularityCE was inheriting the execution environment’s PATH value without explicit configuration controls. From a defense-in-depth perspective, a privileged application can benefit from explicit, configurable search path rules rather than implicit reliance on environment variables.
Maintainer Response & Implementationh2
The SingularityCE team accepted the defense-in-depth case and implemented explicit search path configuration. The fix is available in PR #4095.
Prior behavior: SingularityCE searched the environment $PATH plus sensible default directories for external binaries (except those with explicit entries in singularity.conf).
New configuration: Two configurable search paths were introduced in singularity.conf:
- root search path: Used when SingularityCE starts as host root. By default, excludes the environment
$PATHto avoid inherited untrusted directories. - user search path: Used when started as non-root or fake-root. By default, still includes
$PATH, preserving previous behavior for unprivileged execution.
Restoration path: If legacy behavior is required, administrators can add $PATH: to the beginning of the root search path in singularity.conf.
This design makes the security model explicit and auditable, allowing operators to decide whether environment inheritance is acceptable for their threat model.
Technical Recommendationsh2
From a software design perspective, privileged processes benefit from explicit executable resolution policies:
- Resolve critical runtime binaries using explicit absolute paths whenever possible.
- Sanitize or explicitly control
PATHvalues before performing privileged executable discovery. - Add configuration options for root and user search paths so behavior is explicit and auditable.
- Treat
PATHlookups as a design consideration when privilege context is involved.
Takeawayh2
The SingularityCE OCI runtime discovery behavior highlighted the value of explicit, configurable search paths in privileged applications. Rather than relying on implicit environment inheritance, the SingularityCE team implemented a defense-in-depth approach by introducing configurable root and user search paths.
This change allows operators to make informed decisions about environment inheritance while providing secure defaults for privileged execution. The maintainers’ response demonstrates how design refinements in privilege-boundary code can improve both security posture and administrative control.
Acknowledgmentsh2
Thanks to the SingularityCE maintainers for their thoughtful review and constructive response. The implementation of configurable search paths is an excellent example of how to incorporate security feedback in a defense-in-depth manner.
The fix is tracked in SingularityCE PR #4095 on GitHub.