Let’s begin by enumerating the techniques can be used:

• Enforcement of code signing
• Execution by a custom class loader
• Use of a custom security manager
• Execution under a custom policy

The use of code signing by the JRE to secure it is most commonly used on applets that are deployed as part of a web application. In that context, Oracle use code signing to warn users about untrusted code and to limit the APIs that untrusted code can make use of. Of course, historically, this technique has been bypassed by luring users into accepting the untrusted code (these days Java will prompt even if code is signed and browsers often offer their own prompt too) and identifying flaws in the JRE which allow the untrusted code to call APIs that would otherwise be restricted. Whilst the latter is still a potential cause for concern, running untrusted in a J2EE application server won’t result in a popup that can be ignored, making this technique more far more effective.

However, in the case of an application running in a J2EE application server, this distinction between trusted and untrusted code is not typically present and all code is considered trusted. To enable the use of signed code, the developers therefore opted to make use of a custom class loader. The custom class loader they implemented extended the SecureClassLoader class, not just to enforce code signing as outlined above but also to limit where classes could be loaded from. Taking this approach, an malicious code would not only need to be correctly signed but would also need to have been deployed to a specific directory path. Further more attempts by plugin developers to load arbitrary bytecode, either from the JRE or from other untrusted sources could be reviewed and granted on a case by case basis. By doing so, the developers hoped to minimise the APIs that the plugins can make use of as well as limit the potential for an installed plugin to fetch and execute malicious code after it had been initially reviewed and found to be suitable for deployment.

An additional side effect of implementing a custom class loader as described is that it can be used in combination with a custom security manager to limit the APIs that the 3rd parties can call. The security manager is designed to determine whether any sensitive operation should be allowed. For example, it can limit access to files and network resources to prevent any unauthorised operations from occurring by mediating on behalf of the underlying JRE when such operations are attempted. The developers ensured that setSecurityManager() was called early with a custom security manager which overrode the default and implemented checkRead() etc in a restricted form. All in all, the custom security manager they wrote needed to implement checks for manipulation of the JVM (such as whether new custom class loaders are allowed), security managers (such as whether the custom class loader is allowed to instantiate a given class), system resources, threads, files and network resources.

Since bugs in the security manager (and access controllers present in later releases of the JRE) can be effectively exploited to gain access to privileged calls by untrusted code, the implementation called for an access controller where only a small subset of allowed operations were permitted. By doing so the developers were able to isolate one installed plugin’s ability to access resource that were only intended to be access that of other installed plugins. In particular, to complete the code signing protection checkAccess() was implemented to allow only a small set of trusted code signers.

Whilst the developers we worked with chose to implement their own security manager, I feel that there is benefit in discussing an alternate approach. Underneath the surface of the default access controller present within the JVM are a series of permissions which can be assigned or modified by the use of a custom security policy. Indeed, it would be possible to make use of these permissions within the developers custom security manager as an alternative to implementing the checks as code. In my mind, utilising a security policy is preferable since much of the complexity is abstracted away however it gives less fine grained access controls since you’re limited to the policy language as implemented within the JRE.

Remember that whilst privilege escalation vulnerabilities are relatively common in the JRE as a whole (hence the raft of remote code execution bugs we’ve seen with applets), what we’ve looked at here is a security in depth measure that can be applied to so-called trusted code supplied by a 3rd party developer that runs in a server environment. Thankfully, the threat actors in a position to exploit any bugs you may leave behind is significantly reduced.

In conclusion, the concepts I’ve touched on in this blog post are are quite complex and leave a fair degree of discretion to developers which may allow for vulnerabilities to be introduced. If you’re interested in implementing such a solutions, I would strongly recommend that you acquire a copy of Java Security which covers these and other concepts in far greater depth. Useful too, is the security research of Security Explorations, a Polish research company who have had a lot of fun with Java.

The post Building a sandpit appeared first on Portcullis Labs.