xenonstack-logo

Big Data Engineering

Enterprise Data Management

Big Data Governance

XenonStack BlogThe Open-Source Data Platform Lifecycle Risk Register

| Published

Key takeaway

  • NiFi 1 and Airflow 2 are out of support and Spark 4.0 is next. How to build a lifecycle risk register for Hadoop, Spark, NiFi, Airflow and Druid estates.

What is a data platform lifecycle risk register? It is a single, owned record of every open-source component in your data platform, the version you run, when community support for that version ends and what you plan to do about it. Open-source projects do not send renewal notices. Support for a release line ends on the project's schedule, whether or not anyone in the enterprise is watching.

The 60-second brief

  • The problem: a typical private cloud data platform runs five or more open-source projects, each with its own release and support calendar.
  • The evidence: NiFi 1 left community support in December 2024. Airflow 2 reached end of life in April 2026. Spark 4.0 is listed to follow in November 2026.
  • The gap: infrastructure monitoring shows whether a service is running. It does not show whether the version is still supported.
  • The fix: a register with seven fields per component, reviewed quarterly, owned by one named person.
  • The payoff: upgrades become planned change, not audit findings.

Part of a series. Start with the overview: Open-Source Data Platforms on Private Cloud: Why Managed Services and Governance Must Evolve Together.

Why Open-Source Lifecycles Catch Enterprises Out

With a commercial product, a vendor account manager tells you when support ends. With community open source, the information is public but nobody is responsible for delivering it to you. Three patterns make this worse:

  • Support windows are getting shorter. Projects are moving to faster release cycles. A version adopted as "new" can be near the end of its window by the time it reaches production.
  • Major versions are architecture changes. NiFi 2 and Airflow 3 both removed features that older estates depend on. The upgrade is a project, not a patch.
  • Dependencies age underneath. A component can be in support while the Java version or library it needs is not.

Open source gives you freedom from a vendor. It also gives you the vendor's job of tracking the lifecycle.

Where the Common Stack Stands

The table below reflects public project information and lifecycle trackers at the time of writing, November 2026. Use it as a starting point and confirm each line against the project's own pages.

ComponentLifecycle signalWhat to check
Apache NiFiThe 1.x line left community support on 8 December 2024. Development is on 2.xTemplates, Variable Registry use, removed processors, Java 21
Apache AirflowThe 2.x line reached end of life on 22 April 2026. The 3.x line is currentCustom operators that access the metadata database, plugins, provider versions
Apache SparkTrackers list 3.4 as end of life since October 2024, 4.0 ending on 23 November 2026 and 3.5 supported into late 2027Which line you run, and whether your distribution follows its own dates
Apache DruidHadoop-based ingestion is deprecated and scheduled for removal in upcoming releases. Recent releases require Java 17Any ingestion that still uses Hadoop-based tasks; the Java runtime
Apache Hadoop (HDFS, YARN)Only recent 3.x release lines are actively maintained by the communityYour exact release line and its Java support
Runtime and platformJava, operating system and Kubernetes versions each have their own end datesWhether a supported component is sitting on an unsupported base

If you run a commercial or packaged distribution, the distributor's support dates apply to you, not the community's. Record both.

The Seven Fields of the Register

FieldWhat it records
Component and versionExact version in each environment, not just "Spark 3"
Support status and dateSupported, limited maintenance or end of life, with the date and source
Business dependencyWhich critical data journeys rely on it
Upgrade pathTarget version, known breaking changes and prerequisite steps
Interim controlsCompensating controls while the exposure is open
OwnerOne named person accountable for the plan
Decision and dateUpgrade, replace or accept, with a target date and approval

Running the Register

  1. Build it from discovery, not memory. Pull versions from the running platform. Estates often hold more versions than anyone expects.
  2. Review quarterly. Add new end-of-life announcements and move dates as projects publish them.
  3. Rank by business impact. An unsupported component on a regulatory reporting path outranks one on a sandbox.
  4. Sequence upgrades together. Components depend on each other. Plan the Java, Hadoop, Spark and orchestration moves as one roadmap so one upgrade does not block another.
  5. Report it upward. Give technology and risk leadership a one-page view: components past support, components within twelve months of it and the plan for each.

Who Should Own It

Lifecycle risk falls between teams. Infrastructure owns the operating system, a platform team owns Kubernetes, data engineering owns pipelines and security owns vulnerability findings. The register needs one owner with authority across those boundaries, usually the head of the data platform. A co-managed support partner can maintain it, but accountability stays inside the enterprise. See Who Owns a Cross-Team Pipeline Failure for the wider responsibility model.

Go Deeper on Specific Components

The Bottom Line

You cannot govern what you have not listed. A lifecycle risk register is a small document with a large effect: it turns "we found out during the audit" into "we planned it last quarter". Start with the versions actually running today.

Build Your Register with XenonStack

XenonStack's platform discovery covers component inventory, support status, critical dependencies and an upgrade roadmap for open-source data platforms on private cloud. Explore Data Managed Services and the Open-Source Data Platform solution.

Talk to a Specialist

Frequently Asked Questions

What does end of life mean for an open-source project?

The community stops releasing fixes, including security fixes, for that version line. The software keeps running, but new vulnerabilities stay unpatched.

Which data platform components are already out of support?

At the time of writing, the Apache NiFi 1.x and Apache Airflow 2.x lines are both past community support, along with older Spark and Hadoop release lines.

How often should the register be reviewed?

Quarterly, and whenever a project announces a new major version or an end-of-life date.

Does a commercial distribution change the dates?

Yes. A distributor sets its own support dates for the versions it ships. Record the distributor date and the community date.

Who should own lifecycle risk?

One named leader with authority across infrastructure, data engineering and security, typically the head of the data platform.

Support dates change. Confirm each component against its project pages, for example the Apache Spark versioning policy, before making decisions.

Explore the XenonStack Platform and Agent Network

Platforms: ElixirData · ElixirClaw · ElixirHub · NexaStack · MetaSecure · Neural AI · AutonomousOps · CohortIQ · AI Aviator · SAIF Aviator

AI agents: AgentAnalyst · AgentEvaluation · AgentForce · AgentGRC · AgentHR · AgentInstruct · AgentLabel · AgentPixel · AgentQA · AgentRAI · AgentScrum · AgentSearch · AgentSketch · AgentSRE

XenonStack: xenonstack.com · xenonstack.ai · xenonstack.ae · xenonstack.jobs

Not sure which decision to hand to governed agents first? Book a Context Workshop to map one AI decision in half a day and leave with a go/no-go.

Table of Contents

navdeep-singh-gill

Navdeep Singh Gill

Global CEO and Founder of XenonStack

LinkedIn profile

Navdeep Singh Gill is serving as Chief Executive Officer and Product Architect at XenonStack. He holds expertise in building SaaS Platform for Decentralised Big Data management and Governance, AI Marketplace for Operationalising and Scaling. His incredible experience in AI Technologies and Big Data Engineering thrills him to write about different use cases and its approach to solutions.

Get the latest articles in your inbox

Subscribe Now

Interested in Solving your Challenges with XenonStack Team

Get Started

Get Started with your requirements and primary focus, that will help us to make your solution

Please Select your Industry
Banking
Fintech
Payment Providers
Wealth Management
Discrete Manufacturing
Semiconductor
Machinery Manufacturing / Automation
Appliances / Electrical / Electronics
Elevator Manufacturing
Defense & Space Manufacturing
Computers & Electronics / Industrial Machinery
Motor Vehicle Manufacturing
Food and Beverages
Distillery & Wines
Beverages
Shipping
Logistics
Mobility (EV / Public Transport)
Energy & Utilities
Hospitality
Digital Gaming Platforms
SportsTech with AI
Public Safety - Explosives
Public Safety - Firefighting
Public Safety - Surveillance
Public Safety - Others
Media Platforms
City Operations
Airlines & Aviation
Defense Warfare & Drones
Robotics Engineering
Drones Manufacturing
AI Labs for Colleges
AI MSP / Quantum / AGI Institutes
Retail Apparel and Fashion

Proceed Next