For Upsun (and Platform.sh) customers , the current situation is unchanged:
- All versions below 7.10 are available and deprecated.
- Elasticsearch 7.17 and 8.5 come with an additional charge, as premium services.
- Opensearch is a viable alternative to Elasticsearch.
- New versions of Elasticsearch will be released as premium services.
The issue at hand
Over the years, the Elasticsearch licenses have evolved: Before 2021: the majority of the code was under an Apache 2.0 license.After 2021, dual-licensed: The Apache 2.0 licensed code changed to a dual license under the Service Side Public License (SSPL) and the Elastic License (EL).\ In August 2024, triple-licensed: The GNU Affero General Public License (AGPLV3) is another option, alongside ELv2 and SSPL. All our services are offered as managed services. From that perspective, here are the main differences between the four licenses:
- Apache 2.0: A permissive license that can be provided as a managed service.
- SSPL: A server side public license that requires releasing the source code for the entirety of the service. This includes all “management software, user interfaces, application program interfaces, automation software, monitoring software, backup software, storage software and hosting software, all such that a user could run an instance of the service using the Service Source Code you make available”. For Upsun, this means all our tooling must be released to manage the service. The SSPL is the license used by MongoDB and now Redis too.
- AGPLV3: This license is compatible with managed services and requires that any modifications made to softwares under the same license must also be published.
- Elastic License V2: a commercial license that is compatible with reselling Elasticsearch as a managed service.
- All versions below 7.10: all of these versions have been deprecated for several years but are still available to be used with no additional charge.
- Elasticsearch 7.17 and 8.5: There is an additional charge when using Elasticsearch 7.17 and 8.5 as these are classed as premium services.
The licensing issue to using new Elasticsearch releases
Our customers asked us to provide Elasticsearch 8.16, free of charge, so we started to do our homework to provide it. At Upsun, services are built using internal tooling, driven by a configuration file with multiple attributes and the required packages. For elasticsearch “premium”, it looks like this:
Let’s build it!
In the images team, we are a bit stubborn. If we can’t use the available packages, we build our own. It’s not because it’s in Java, using Gradle that we can’t do it. We started by a nice discussion with our legal team, to discuss what we could do and agreed on:- build Elasticsearch from the unmodified sources
- publish the resulting Debian package under AGPL V3 in a publicly accessible Github repository
- Remove all OSS distribution references from Elasticsearch build #68797
- Remove OSS specific distribution packaging #69775
“changes to Elasticsearch were sent as upstream pull requests (#42066, #42658, #43284, #43839, #53643, #57271, #59563, #61400, #64513), and we then included the “oss” builds offered by Elastic in our distribution. This ensured that we were collaborating with the upstream developers and maintainers, and not creating a “fork” of the software.”So, it’s not easy to build an AGPL v3 package for Elasticsearch; the code simply doesn’t have the tooling.
Let’s modify the build tooling!
We went back to our legal team, to discuss what we could do, and agreed on:- fork the Elasticsearch repository
- update the Gradle tooling to build an OSS distribution
- on a public repository, under AGPL V3:
- share all the code
- share the build artifacts
- send an email to Elasticsearch explaining what we did
Have you met x-pack?
x-pack is an Elastic extension that provides various capabilities, including the security layer. It has been in Elasticsearch for a long time, as a paid extension, and it has a “turbulent” life of its own: “Doubling Down on Open”.
As previously mentioned, we had to remove the x-pack folder to build an open source package.
However, once we removed x-pack, Elasticsearch wouldn’t start anymore; the code is coupled to x-pack’s security features. You can’t turn that off.
Secure, very secure
We noticed Elasticsearch sets up security before executing the server. The code below callssecureSettingsLoader.supportsSecurityAutoConfiguration() to decide if it enables security or not.
null. Unfortunately, when security is not configured, the secrets ends up being null!
Finding ourselves at a crossroads again, we realized something.
Wait, are we really forking Elasticsearch?
At that point, we took a step back, looking at what we were doing. We realized that “we are forking Elasticsearch”:- We forked the repository
- We deeply modified the code, involving complicated maintenance and version upgrades
- We were proposing a new build