.webp?width=1921&height=622&name=usecase-banner%20(1).webp)
Key takeaway
- DPDP Rule 6 security safeguards apply from May 2027. A checklist mapping encryption, access control, logging and backup to Hadoop, Spark, NiFi and Airflow.
What does DPDP require of a big data platform? India's Digital Personal Data Protection Rules, 2025 were notified on 13 November 2025. Rule 6, which sets out reasonable security safeguards, comes into force on 13 May 2027. It turns a general duty to protect personal data into a named list of measures: data security such as encryption and masking, access control, logging and monitoring, backups, one-year log retention and security terms in processor contracts. Each of these has to be true on the Hadoop, Spark, NiFi and Airflow platforms where personal data is processed at scale.
The 60-second brief
- The date: Rule 6 applies from 13 May 2027.
- The stake: the DPDP Act provides for penalties of up to Rs 250 crore for failing to take reasonable security safeguards.
- The scope: the measures apply to the data fiduciary and, through contract, to its data processors.
- The platform challenge: open-source data platforms were often built for throughput first. Authentication, encryption and audit were added unevenly.
- The checklist: six control areas, each mapped to platform components and to the evidence an assessor would expect.
Part of a series. Start with the overview: Open-Source Data Platforms on Private Cloud: Why Managed Services and Governance Must Evolve Together.
Why Analytical Platforms Deserve Their Own Review
Privacy programmes usually begin with customer-facing applications and consent. Analytical platforms get attention later, yet they hold the widest collection of personal data in the enterprise: raw feeds, joined datasets, model features and exports. They also have the most internal boundaries: ingestion, storage, compute, orchestration and serving, each with its own identity and encryption settings.
A safeguard that covers the application but not the data lake behind it covers the smaller half of the risk.
The Checklist
1. Data security: encryption, masking and tokenisation
| Check | Where it applies |
|---|---|
| Data in transit is encrypted between services, not only at the edge | HDFS data transfer, RPC, Spark shuffle, NiFi site-to-site, JDBC connections |
| Management interfaces and APIs use TLS | NameNode, ResourceManager, Spark history, NiFi, Airflow and Druid consoles |
| Sensitive datasets are encrypted at rest with managed keys | HDFS encryption zones or storage-level encryption, with a key management service |
| Personal data is masked or tokenised where full values are not needed | Column masking policies; tokenisation at ingestion |
| Secrets are not stored in code, variables or flow definitions | Airflow connections, NiFi sensitive properties, Spark job configuration |
2. Access control
| Check | Where it applies |
|---|---|
| Every user and service authenticates strongly | Kerberos or enterprise SSO across the cluster; no anonymous or simple authentication |
| Authorisation is least-privilege and centrally managed | Policy enforcement such as Apache Ranger across HDFS, Hive, YARN queues and Kafka |
| Service accounts are individual, scoped and reviewed | Pipelines, schedulers and ingestion flows; no shared superuser accounts |
| Privileged access is time-bound and approved | Cluster administrators, emergency access to consoles |
3. Logging, monitoring and review
| Check | Where it applies |
|---|---|
| Access to personal data is logged | HDFS audit logs, Ranger audit, query engine logs |
| Administrative and configuration changes are logged | Cluster management, policy changes, DAG and flow changes |
| Logs are centralised and protected from alteration | A log platform separate from the cluster administrators |
| Unauthorised access attempts are detected and reviewed | Alerts on denied access, unusual volumes and privilege use |
4. Log retention
Rule 6 requires logs and relevant personal data to be retained for one year to support detection, investigation and remediation, unless another law requires longer. Many clusters rotate audit logs on a storage-cost cycle far shorter than that. Check retention settings per component, and budget the storage.
5. Continuity: backup and recovery
| Check | Where it applies |
|---|---|
| Critical data and metadata are backed up | NameNode metadata, Hive metastore, Ranger policies, Airflow database, NiFi flow configuration |
| Restoration is tested, not assumed | Documented recovery runbooks with test evidence |
| Processing can continue after a compromise | High availability and recovery procedures for critical pipelines |
6. Processors and partners
The rule expects contracts with data processors to include security safeguard provisions. If a managed service provider, system integrator or analytics partner can access the platform, the same controls must bind them: named identities, scoped access, logged activity and the one-year log retention, written into the agreement.
Evidence an Assessor Will Expect
- A current inventory of components, data flows and where personal data resides.
- Configuration baselines showing encryption and authentication settings.
- Access review records for users, administrators and service accounts.
- Samples of audit logs with retention proof.
- Backup and restoration test results.
- Records of security exceptions, with owners and closure dates.
Enabling a control is not the finish line. The control must be shown to work, be supportable and leave evidence.
How to Introduce Controls Without Breaking Pipelines
Turning on Kerberos, TLS or restrictive policies in a platform that has run for years can stop business-critical jobs. Sequence the work:
- Assess current controls and gaps per component.
- Design the target, checking compatibility across versions.
- Implement in nonproduction first, then by approved production change window.
- Validate with negative-access tests, connectivity checks, performance regression tests and audit evidence.
Unsupported component versions complicate this, because security features and fixes arrive only on supported lines. Check the lifecycle risk register first. For banks and NBFCs, plan this together with the RBI draft data governance guidance, since both depend on the same capabilities.
The Bottom Line
May 2027 is close in platform terms. Security hardening of a live distributed platform is measured in quarters, not weeks. Start with the assessment, fix identity and logging first, and keep the evidence as you go.
Harden Your Data Platform with XenonStack
XenonStack provides security and governance enablement for open-source data platforms on private cloud, from assessment and design through controlled implementation and validation, with co-managed L2/L3 support afterwards. Explore Data Managed Services and the Data Protection readiness assessment.
Frequently Asked Questions
When do the DPDP security safeguard requirements apply?
Rule 6 of the DPDP Rules, 2025 comes into force on 13 May 2027, eighteen months after the Rules were notified.
What safeguards does Rule 6 list?
Data security measures such as encryption, obfuscation, masking or tokens; access control; logging, monitoring and review; continuity measures including backups; one-year retention of logs; security provisions in processor contracts; and supporting technical and organisational measures.
How long must logs be kept?
For one year, unless another law requires a longer period.
Does DPDP apply to analytical and big data platforms?
Yes. The safeguards apply to personal data in the fiduciary's possession or control, wherever it is processed, including data lakes and analytical clusters.
What is the penalty for failing to take security safeguards?
The DPDP Act provides for a penalty of up to Rs 250 crore for breach of the obligation to take reasonable security safeguards.
This article is a technology perspective, not legal advice. Refer to the official DPDP Rules, 2025 and your legal advisers for authoritative requirements.
Explore the XenonStack Platform and Agent Network
Platforms: ElixirData · ElixirClaw · ElixirHub · NexaStack · MetaSecure · Neural AI · AutonomousOps · CohortIQ · AI Aviator · SAIF Aviator
AI agents: AgentAnalyst · AgentEvaluation · AgentForce · AgentGRC · AgentHR · AgentInstruct · AgentLabel · AgentPixel · AgentQA · AgentRAI · AgentScrum · AgentSearch · AgentSketch · AgentSRE
XenonStack: xenonstack.com · xenonstack.ai · xenonstack.ae · xenonstack.jobs
Not sure which decision to hand to governed agents first? Book a Context Workshop to map one AI decision in half a day and leave with a go/no-go.
Explore the XenonStack platform family: ElixirData · ElixirClaw · ElixirHub · NexaStack · MetaSecure · Neural AI · AgentGRC · AgentSRE · AgentAnalyst · AgentInstruct · AgentRAI · AgentScrum · AgentSearch · AgentForce · AI Aviator · SAIF Aviator · CohortIQ · XenonStack AI · XenonStack UAE
