Skip to main content
XenonStack

Cognitive Automation

XenonStack Insights Capsule Networks: Architecture and Dynamic Routing

and | Published | Updated

Capsule Networks Explained: a stack from convolutional features and capsule vectors to routing by agreement and a pilot decision.

Key takeaway

  • Capsule networks explained: how routing by agreement works, what the 2017 and 2018 papers report, how they compare with CNNs and a 3 to 5 year outlook.

What is a capsule network?

A capsule network is a neural network built from capsules, which are small groups of neurons whose output is a vector rather than a single number. In the original paper, Dynamic Routing Between Capsules by Sara Sabour, Nicholas Frosst and Geoffrey Hinton (arXiv, October 2017), the length of that vector represents the probability that an entity such as an object or object part is present, and its orientation encodes the entity's parameters.

The idea was proposed as an alternative way to represent objects than convolutional neural networks alone. For enterprise leaders the short answer is this: capsule networks are an important research idea in computer vision, but not a mainstream production choice today. This guide explains how they work, what the evidence says, and where they fit.

How does routing by agreement work?

Capsules in one layer use transformation matrices to predict the parameters of capsules in the layer above. When several lower-level capsules make predictions that agree, the higher-level capsule becomes active. According to the paper's abstract, this is done by an iterative routing-by-agreement mechanism: a lower-level capsule sends its output preferentially to higher-level capsules whose activity vectors have a large scalar product with its prediction.

Two-column diagram: eye, nose and mouth capsules and transformation matrices on the left; predictions compared, routing by agreement, object capsule active and a pilot baseline on the right.

Think of a face made of parts. A capsule for an eye, a nose and a mouth each predicts where a face capsule should sit and how it should be oriented. If those predictions line up, the face capsule switches on. If they disagree, it stays quiet. The same idea applies to any object made of parts.

Routing also decides where information goes. Instead of every lower-level unit feeding every higher-level unit, output is sent to the capsules that agree with it. That is the mechanism the paper describes; it does not by itself promise lower compute cost, so treat any speed claim as something to measure on your own workload.

How do capsule networks differ from CNNs?

QuestionConvolutional networkCapsule network (as proposed in the 2017 paper)
What does a unit output?A scalar activation per featureA vector whose length is presence and whose direction encodes entity parameters
How are parts combined?Learned filters and pooling or striding across layersPrediction by transformation matrices, then routing by agreement
Where is it proven?Widely deployed across industryReported results on MNIST and overlapping digits in the original paper; see the 2018 and 2019 follow-ups below
Production maturityHigh, with mature toolingMostly research code and benchmarks

Both ideas come from the same family of deep learning methods. If you are new to them, our guides on artificial neural network applications and generative adversarial networks give useful background.

What are the benefits and limits of capsule networks?

The original paper reports that a discriminatively trained, multi-layer capsule system achieves state-of-the-art performance on MNIST and is considerably better than a convolutional net at recognising highly overlapping digits. The follow-up Matrix capsules with EM routing (ICLR 2018, Hinton, Sabour and Frosst) reports 30% lower test error than the best reported CNN on the smallNORB benchmark and much greater resistance to white-box adversarial attacks. Both are the authors' own reports on specific benchmarks.

PointWhat the sources sayCaution
Entity parametersCapsule vectors encode parameters such as pose (2017 paper)Benefit depends on the task and data
Overlapping objectsBetter than a CNN on highly overlapping digits (2017 paper abstract)Result is on a digit benchmark, not a general guarantee
Viewpoint changesmallNORB test error 30% lower than best reported CNN (2018 paper)Benchmark reported by the authors; test it on your images
Adversarial attacksMuch more resistant to white-box attacks (2018 paper)Confirm independently before relying on it for security
CostRouting adds iterative steps between layersMeasure training and inference time yourself

How do you implement a capsule network?

Capsule networks can be built in any deep learning framework, for example TensorFlow, PyTorch or Keras. The sequence below follows the structure described in the 2017 paper at a high level. Check the paper and its reference code for exact layer sizes before you copy anything.

  1. Start with ordinary convolutional layers that turn the image into low-level features.
  2. Group those features into primary capsules, each producing a vector.
  3. Apply a squashing non-linearity so each vector length stays between 0 and 1 and can be read as a presence probability.
  4. Use transformation matrices so each lower capsule predicts the higher capsules.
  5. Run the routing-by-agreement loop between adjacent capsule layers.
  6. Train with a loss on the output capsules and evaluate on held-out and overlapping-object data.

Useful skills are deep learning basics, the framework you choose and a clear understanding of the routing step. Our guides to machine learning model visualisation and ONNX help with inspecting and exporting trained models.

What will change in capsule networks over the next 3 to 5 years?

Nobody can promise dates for research ideas, so this outlook is limited to what the published papers show as at November 2026. It is our reading of those sources, not a forecast from them.

Source and dateWhat it showsWhat it suggests for the next 3 to 5 years
Matrix capsules with EM routing, ICLR 2018Capsules with pose matrices and EM routing, with smallNORB and adversarial results reported by the authorsExpect continued benchmark work on viewpoint change and robustness; independent replication is the thing to watch
Stacked Capsule Autoencoders, arXiv June 2019 (Kosiorek, Sabour, Teh, Hinton)Unsupervised capsule model reporting 98.7% on MNIST and 55% on SVHN classification, with code releasedLikely direction is less labelled data and standard encoders, not new hand-built routing
How to represent part-whole hierarchies in a neural network, arXiv February 2021 (Hinton)GLOM, a conceptual proposal that says it does not describe a working systemPart-whole ideas may return inside transformer-style models rather than as stand-alone capsule nets
An Image is Worth 16x16 Words, arXiv October 2020Vision Transformers perform competitively with leading CNNs when pre-trained on large datasetsTransformers set the production baseline that capsule ideas must beat

In plain terms: expect capsule ideas to influence research on viewpoint robustness and interpretability, while production vision stays on CNNs and transformers unless a capsule method shows independent, repeatable gains on your data.

Should an enterprise adopt capsule networks?

For most teams the answer is to run a small, bounded pilot rather than a roll-out. A sensible sequence is: define the vision task and the failure you care about (for example viewpoint change), set a CNN or transformer baseline, test a capsule model on the same data, and compare accuracy, cost and explainability before any decision.

Whichever model you choose, governance matters more than the architecture. ElixirData Context OS keeps governed context, versioned policy and decision traces in one decision layer above AI agents, and AgentAnalyst lets teams ask questions of enterprise data in plain language. If the model runs on cameras or machines at the edge, NexaStack is an agentic operating platform for physical AI that runs across on-premises, cloud and hybrid environments.

To map one vision or model decision your AI should make, and the context and policy it depends on, book a Context Workshop. See what happens in the workshop.

Capsule networks FAQ

What is a capsule network in simple terms?

It is a neural network made of capsules, groups of neurons that output a vector. The vector length shows whether an entity is present and its direction encodes the entity's parameters, as described in the 2017 paper by Sabour, Frosst and Hinton.

What is routing by agreement?

Lower-level capsules predict the parameters of higher-level capsules, and a higher-level capsule becomes active when several predictions agree. Output is sent preferentially to the capsules whose activity vectors have a large scalar product with the prediction.

Are capsule networks better than CNNs?

On some benchmarks the authors report better results, such as overlapping digits in 2017 and smallNORB in 2018. CNNs and transformers remain the mainstream production choice, so test on your own data before deciding.

Are capsule networks used in production today?

This post found no source to confirm wide production use. The papers read are research results, so treat capsule networks as a pilot candidate, not a default.

What will change in capsule networks over the next 3 to 5 years?

Based on the papers read, expect work on unsupervised capsule models and part-whole ideas inside transformer-style systems. Whether any of it beats CNNs and transformers in production is still open.

Ready to put AI agents to work? See an ElixirClaw demo: the agentic OS to build, orchestrate and govern AI agents across your enterprise systems.

Table of Contents

navdeep-singh-gill

Navdeep Singh Gill

Global CEO and Founder of XenonStack

LinkedIn profile

Navdeep Singh Gill is serving as Chief Executive Officer and Product Architect at XenonStack. He holds expertise in building SaaS Platform for Decentralised Big Data management and Governance, AI Marketplace for Operationalising and Scaling. His incredible experience in AI Technologies and Big Data Engineering thrills him to write about different use cases and its approach to solutions.

Get the latest articles in your inbox

Subscribe Now

Interested in solving your challenges with the XenonStack team?
Get Started

Tell us about you and your priorities so we can prepare the right conversation. It takes about 2 minutes.

Please Select your Industry
Banking
Fintech
Payment Providers
Wealth Management
Discrete Manufacturing
Semiconductor
Machinery Manufacturing / Automation
Appliances / Electrical / Electronics
Elevator Manufacturing
Defense & Space Manufacturing
Computers & Electronics / Industrial Machinery
Motor Vehicle Manufacturing
Food and Beverages
Distillery & Wines
Beverages
Shipping
Logistics
Mobility (EV / Public Transport)
Energy & Utilities
Hospitality
Digital Gaming Platforms
SportsTech with AI
Public Safety - Explosives
Public Safety - Firefighting
Public Safety - Surveillance
Public Safety - Others
Media Platforms
City Operations
Airlines & Aviation
Defense Warfare & Drones
Robotics Engineering
Drones Manufacturing
AI Labs for Colleges
AI MSP / Quantum / AGI Institutes
Retail Apparel and Fashion

Continue