Key takeaway
- NiFi 1 and Airflow 2 are out of support and Spark 4.0 is next. How to build a lifecycle risk register for Hadoop, Spark, NiFi, Airflow and Druid estates.
What is a data platform lifecycle risk register? It is a single, owned record of every open-source component in your data platform, the version you run, when community support for that version ends and what you plan to do about it. Open-source projects do not send renewal notices. Support for a release line ends on the project's schedule, whether or not anyone in the enterprise is watching.
The 60-second brief
- The problem: a typical private cloud data platform runs five or more open-source projects, each with its own release and support calendar.
- The evidence: NiFi 1 left community support in December 2024. Airflow 2 reached end of life in April 2026. Spark 4.0 is listed to follow in November 2026.
- The gap: infrastructure monitoring shows whether a service is running. It does not show whether the version is still supported.
- The fix: a register with seven fields per component, reviewed quarterly, owned by one named person.
- The payoff: upgrades become planned change, not audit findings.
Part of a series. Start with the overview: Open-Source Data Platforms on Private Cloud: Why Managed Services and Governance Must Evolve Together.
Why Open-Source Lifecycles Catch Enterprises Out
With a commercial product, a vendor account manager tells you when support ends. With community open source, the information is public but nobody is responsible for delivering it to you. Three patterns make this worse:
- Support windows are getting shorter. Projects are moving to faster release cycles. A version adopted as "new" can be near the end of its window by the time it reaches production.
- Major versions are architecture changes. NiFi 2 and Airflow 3 both removed features that older estates depend on. The upgrade is a project, not a patch.
- Dependencies age underneath. A component can be in support while the Java version or library it needs is not.
Open source gives you freedom from a vendor. It also gives you the vendor's job of tracking the lifecycle.
Where the Common Stack Stands
The table below reflects public project information and lifecycle trackers at the time of writing, November 2026. Use it as a starting point and confirm each line against the project's own pages.
| Component | Lifecycle signal | What to check |
|---|---|---|
| Apache NiFi | The 1.x line left community support on 8 December 2024. Development is on 2.x | Templates, Variable Registry use, removed processors, Java 21 |
| Apache Airflow | The 2.x line reached end of life on 22 April 2026. The 3.x line is current | Custom operators that access the metadata database, plugins, provider versions |
| Apache Spark | Trackers list 3.4 as end of life since October 2024, 4.0 ending on 23 November 2026 and 3.5 supported into late 2027 | Which line you run, and whether your distribution follows its own dates |
| Apache Druid | Hadoop-based ingestion is deprecated and scheduled for removal in upcoming releases. Recent releases require Java 17 | Any ingestion that still uses Hadoop-based tasks; the Java runtime |
| Apache Hadoop (HDFS, YARN) | Only recent 3.x release lines are actively maintained by the community | Your exact release line and its Java support |
| Runtime and platform | Java, operating system and Kubernetes versions each have their own end dates | Whether a supported component is sitting on an unsupported base |
If you run a commercial or packaged distribution, the distributor's support dates apply to you, not the community's. Record both.
The Seven Fields of the Register
| Field | What it records |
|---|---|
| Component and version | Exact version in each environment, not just "Spark 3" |
| Support status and date | Supported, limited maintenance or end of life, with the date and source |
| Business dependency | Which critical data journeys rely on it |
| Upgrade path | Target version, known breaking changes and prerequisite steps |
| Interim controls | Compensating controls while the exposure is open |
| Owner | One named person accountable for the plan |
| Decision and date | Upgrade, replace or accept, with a target date and approval |
Running the Register
- Build it from discovery, not memory. Pull versions from the running platform. Estates often hold more versions than anyone expects.
- Review quarterly. Add new end-of-life announcements and move dates as projects publish them.
- Rank by business impact. An unsupported component on a regulatory reporting path outranks one on a sandbox.
- Sequence upgrades together. Components depend on each other. Plan the Java, Hadoop, Spark and orchestration moves as one roadmap so one upgrade does not block another.
- Report it upward. Give technology and risk leadership a one-page view: components past support, components within twelve months of it and the plan for each.
Who Should Own It
Lifecycle risk falls between teams. Infrastructure owns the operating system, a platform team owns Kubernetes, data engineering owns pipelines and security owns vulnerability findings. The register needs one owner with authority across those boundaries, usually the head of the data platform. A co-managed support partner can maintain it, but accountability stays inside the enterprise. See Who Owns a Cross-Team Pipeline Failure for the wider responsibility model.
Go Deeper on Specific Components
- Apache NiFi 1 End of Support: How to Move to NiFi 2
- Airflow 2 End of Life: An Upgrade Plan
- Modernise in Place: HDFS to Apache Iceberg on Private Cloud
The Bottom Line
You cannot govern what you have not listed. A lifecycle risk register is a small document with a large effect: it turns "we found out during the audit" into "we planned it last quarter". Start with the versions actually running today.
Build Your Register with XenonStack
XenonStack's platform discovery covers component inventory, support status, critical dependencies and an upgrade roadmap for open-source data platforms on private cloud. Explore Data Managed Services and the Open-Source Data Platform solution.
Frequently Asked Questions
What does end of life mean for an open-source project?
The community stops releasing fixes, including security fixes, for that version line. The software keeps running, but new vulnerabilities stay unpatched.
Which data platform components are already out of support?
At the time of writing, the Apache NiFi 1.x and Apache Airflow 2.x lines are both past community support, along with older Spark and Hadoop release lines.
How often should the register be reviewed?
Quarterly, and whenever a project announces a new major version or an end-of-life date.
Does a commercial distribution change the dates?
Yes. A distributor sets its own support dates for the versions it ships. Record the distributor date and the community date.
Who should own lifecycle risk?
One named leader with authority across infrastructure, data engineering and security, typically the head of the data platform.
Support dates change. Confirm each component against its project pages, for example the Apache Spark versioning policy, before making decisions.
Explore the XenonStack Platform and Agent Network
Platforms: ElixirData · ElixirClaw · ElixirHub · NexaStack · MetaSecure · Neural AI · AutonomousOps · CohortIQ · AI Aviator · SAIF Aviator
AI agents: AgentAnalyst · AgentEvaluation · AgentForce · AgentGRC · AgentHR · AgentInstruct · AgentLabel · AgentPixel · AgentQA · AgentRAI · AgentScrum · AgentSearch · AgentSketch · AgentSRE
XenonStack: xenonstack.com · xenonstack.ai · xenonstack.ae · xenonstack.jobs
Not sure which decision to hand to governed agents first? Book a Context Workshop to map one AI decision in half a day and leave with a go/no-go.
Explore the XenonStack platform family: ElixirData · ElixirClaw · ElixirHub · NexaStack · MetaSecure · Neural AI · AgentGRC · AgentSRE · AgentAnalyst · AgentInstruct · AgentRAI · AgentScrum · AgentSearch · AgentForce · AI Aviator · SAIF Aviator · CohortIQ · XenonStack AI · XenonStack UAE