ML and DL Model Testing Best Practices and Tools - XenonStack

What is Machine Learning?

Machine learning is a part of Artificial Intelligence (AI) that give power to the systems to automatically determine and boost from experience without being particularly programmed. Machine learning targets on the advancement of computer models that can admission datasets and use it train for themselves. There are different types of Machine Learning, some of them are –

  • Supervised Learning
  • Unsupervised Learning
  • Reinforcement Machine Learning
  • Semi-Supervised Learning
  • Semi- unsupervised Learning.

Example of Machine Learning –

Detect the face and tag on Facebook. In games, movement of person change, then replicate the action in the virtual world.


What is Deep Learning?

Deep Learning concerns with algorithms inspired by the structure and function of the brain called Artificial Neural Networks. Deep learning considered as the subset of Machine learning. Machine learning and Deep learning models differ in automatic acquires information’s representations from datasets which contain images, video or text, without recommending well-programmed rules or human domain expertise in Deep Learning.

Example of Deep Learning –

Deep learning is a crucial technology behind driverless cars, enabling them to recognize a stop sign or to distinguish a pedestrian from a lamp post.


Machine Learning Model Testing Training

Software quality in Machine learning and Deep learning systems are different. In this, accuracy, robustness, learning efficiency and adaptation and performance of the system checked.

To understand and determine the quality requirements of Machine Learning systems is an important step. To verify them will be another new challenge.

Testing With ML Models?

The elements which considered for testing in the case of ML Models.

Quality of data and quality of features Quality of ML algorithms Labels related to the Images

Quality Assurance of Data Used for Training the Model

One of the aspects of building a Machine Learning model is to check whether the data used for training and testing the model belong to an adversary dataset. The adversary data sets are that can be used to skew the results of the model by training the model using incorrect data called as Data Poisoning Attack.

Quality Assurance put test mechanisms in place to validate whether the data used for training sanitized. The tests need to be performed to identify whether there are instances of data poisoning attacks unintentionally or intentionally.

To achieve this, one of the techniques to have QA work with product management and product consultant teams for some of the following –

Understanding the statistics related to data (median, mean, mode, etc.) Understand the data and relationships at a high-level. Build tests using scripts to check the statistics and correlations. Run the tests during regular intervals. The parameters listed would need to be tracked at regular intervals and verified with the help of consultants before every release.

Quality Assurance of Features

Many times, one or more features become redundant/irrelevant, and, in turn, impact the prediction error rates where QA/testing practices should be in place to proactively evaluate features using feature engineering techniques such as dimensionality reduction and feature selection

Quality Assurance of ML Algorithms

Evolving datasets as a result of data poisoning attacks could result in increased prediction error rates. As the ML model gets retrained, the increased prediction error rates result in the re-evaluation of ML models result in the discovery of new algorithms giving accuracy over the existing ones.

Ways to go about ML algorithms testing with new data is the following –

Many times, ML models are built using different algorithms and get discarded once and for all after the most accurate model gets selected. Retrain all of the models and track the performance of models with new data set at regular intervals. Raise the defect if another model gives greater accuracy or performs better than the existing model.


Top Benefits of Machine Learning and Deep Learning Model Testing

When considering the strategy of Machine Learning testing, think accuracy and efficiency as a primary goal in the quality assurance. Benefits such as detecting redundant unsuccessful tests, and keeping untested code out of production, prediction, and prevention ultimately reduce much of the risk in the deployment phase. Some of the critical contributions quality assurance include –

  • Defect alerts Enhanced analytics Faster predictions Improved optimization Cleaner traceability Real-time feedback
  • The sooner can implement an in-house AI platform to assist in application testing; will discover a more accurate and efficient deployment with reduced effort. Using defined test metrics and analytics will launch application development to new heights.

Why Model Testing Matters?

The Machine Learning field provides tools to make decisions automatically from data to achieve any goal or requirement.

Some problems resist a manually specified solution. Machine learning matters as it provides methods to create solutions for complex problems.

Machine learning promises to solve problems automatically, faster and more accurately than a manually specified solution and at a larger scale.

Machine learning applications are not 100% accurate, and approx never will be. There are some of the reasons why testers cannot ignore learning about Machine learning and Deep learning. The fundamental reason is that these applications learning limited by data they have used to build algorithms. As Machine learning apps are managing almost daily activity performed by humans – one error can lead to severe losses. You may also love to read more about System Testing in this insight.


How to Adopt Model Testing?

  • Based on historical data ( previously verified defects) create a training data set.
  • Train predictive model using the classification algorithm that meets defined quality criteria.
  • The model should expose as an online REST service, so we can quickly call it and make prediction requests for new incoming defects.
  • Defect records that store in a code repository automatically updated with the scoring result: prediction and probability of prediction.
  • Defects should sort by prediction probability, so QA engineer could start testing bugs with the highest probability of fix(incorrect).
  • If new bugs successfully verified, also label value is known (correctly fixed or not), such records are marked as new training data. Then it stored in the feedback data store.

Best Practices for ML and DL Model Testing

Both testing practices and results have changed to accommodate applications that don’t behave the same as traditional software. Traditional testing techniques based on fixed inputs. Testers believe that given inputs x and y, the output will be z and this will be constant until the application changes. It is not true in machine learning systems. The output not fix. It will change over time. It is similar to the model built on Machine Learning system evolving as more data fed. It forces the testing professional to think differently and adapt test strategies that are very different from traditional testing techniques.

Critical activities that will be essential to test machine learning systems –

  • Developing training data sets – This refers to a data set of examples used for training the model. In this data set, you have the input data with the expected output. This data usually prepare by collecting data in a semi-automated way. The goal of such a continuous learning system is to ensure the highest possible quality of the exposed model. The feedback data store is used to evaluate the quality of the server model, automatically retrain the deployed model, and finally to re-deploy the new model version.
  • Developing test data sets – It is a subset or part of the training dataset that is built to test all the possible combinations, also estimates how well the model trains. Based on the test data set results, the model fine-tuned.

Develop validation test suites based on algorithms and test datasets. The example of DNA, test scenarios include categorizing patient outcomes based on DNA sequences and creating patient risk profiles based on behaviors and demographics.

The key to building validation suites is to understand the algorithm based on calculations that create a model from the training data. The algorithm analyzes the data provided for model creation, looks for specific patterns, and results of the analysis to develop optimal parameters for creating the model. The model defines as the number of iterations and the richness of the data increase. Some algorithms in widespread use are regression algorithms that predict one or more continuous numeric variables such as return on investment.

Another example is the association of algorithms. It uses for portfolio analysis in capital markets. Another illustration in digital applications is sequence algorithms that predict customer behavior based on a series of clicks or paths on the platform. Communicating test results in statistical terms.

Traditionally testers are used to expressing the results of testing concerning quality such as the severity of defects or defect leakage. Models validation based on machine algorithms will not exact results but produce approximations. The testing community needs to determine the level of confidence within a certain range for each outcome and articulate the same. Learn more about Sanity Testing in this insight.


Best Tools for Machine Learning and Deep Learning Model Testing



For manual testing of a Machine learning the tools which can be used to develop a machine learning model can also be used for testing the model. The tools are –

However, there are tools which can be for automating testing concerning Artificial Intelligence –


[wpforms id="8104"]
<div class="wpforms-container wpforms-container-full optin-monster-forms" id="wpforms-8104"><form id="wpforms-form-8104" class="wpforms-validate wpforms-form" data-formid="8104" method="post" enctype="multipart/form-data" action="/insights/machine-learning-model-testing/"><noscript class="wpforms-error-noscript">Please enable JavaScript in your browser to complete this form.</noscript><div class="wpforms-page-indicator progress" data-indicator="progress" data-indicator-color="#72b239" data-scroll="1"><span class="wpforms-page-indicator-page-title" ></span><span class="wpforms-page-indicator-page-title-sep" style="display:none;"> - </span><span class="wpforms-page-indicator-steps">Step <span class="wpforms-page-indicator-steps-current">1</span> of 2</span><div class="wpforms-page-indicator-page-progress-wrap"><div class="wpforms-page-indicator-page-progress" style="width:50%;background-color:#72b239;"></div></div></div><div class="wpforms-field-container"><div class="wpforms-page wpforms-page-1 "><div id="wpforms-8104-field_10-container" class="wpforms-field wpforms-field-pagebreak" data-field-id="10"></div><div id="wpforms-8104-field_24-container" class="wpforms-field wpforms-field-html form-popup-header-wrapper" data-field-id="24"><div id="wpforms-8104-field_24"><div class="form-popup-header"> <h2>Modernise Data Operations with DataOps</h2> </div></div></div><div id="wpforms-8104-field_21-container" class="wpforms-field wpforms-field-radio custom-radio-btn-wrapper wpforms-list-2-columns" data-field-id="21"><label class="wpforms-field-label wpforms-label-hide" for="wpforms-8104-field_21">Sevices <span class="wpforms-required-label">*</span></label><ul id="wpforms-8104-field_21" class="wpforms-field-required wpforms-image-choices wpforms-image-choices-modern"><li class="choice-1 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8104-field_21_1" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/data-catalog-delivery-platform.png" alt="Data Catalog and Discovery Platform" title="Data Catalog and Discovery Platform"></span><input type="radio" id="wpforms-8104-field_21_1" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Data Catalog and Discovery Platform" tabindex="-1" required ><span class="wpforms-image-choices-label">Data Catalog and Discovery Platform</span></label></li><li class="choice-2 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8104-field_21_2" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/real-time-analytics-platform.png" alt="Real Time Analytics Platform" title="Real Time Analytics Platform"></span><input type="radio" id="wpforms-8104-field_21_2" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Real Time Analytics Platform" tabindex="-1" required ><span class="wpforms-image-choices-label">Real Time Analytics Platform</span></label></li><li class="choice-3 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8104-field_21_3" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/migration-delta-lake.png" alt="Migration to Delta Lake" title="Migration to Delta Lake"></span><input type="radio" id="wpforms-8104-field_21_3" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Migration to Delta Lake" tabindex="-1" required ><span class="wpforms-image-choices-label">Migration to Delta Lake</span></label></li><li class="choice-4 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8104-field_21_4" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/cloud-data-warehouse.png" alt="Cloud Data warehouse" title="Cloud Data warehouse"></span><input type="radio" id="wpforms-8104-field_21_4" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Cloud Data warehouse" tabindex="-1" required ><span class="wpforms-image-choices-label">Cloud Data warehouse</span></label></li><li class="choice-5 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8104-field_21_5" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/enterprise-data-strategy.png" alt="Enterprise Data Strategy" title="Enterprise Data Strategy"></span><input type="radio" id="wpforms-8104-field_21_5" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Enterprise Data Strategy" tabindex="-1" required ><span class="wpforms-image-choices-label">Enterprise Data Strategy</span></label></li></ul></div><div id="wpforms-8104-field_23-container" class="wpforms-field wpforms-field-pagebreak" data-field-id="23"><div class="wpforms-clear wpforms-pagebreak-left"><button class="wpforms-page-button wpforms-page-next" data-action="next" data-page="1" data-formid="8104">Next</button></div></div></div><div class="wpforms-page wpforms-page-2 last " style="display:none;"><div id="wpforms-8104-field_25-container" class="wpforms-field wpforms-field-html form-popup-header-wrapper" data-field-id="25"><div id="wpforms-8104-field_25"><div class="form-popup-header"> <h2>Lets Discuss and Understand the Requirements</h2> </div></div></div><div id="wpforms-8104-field_20-container" class="wpforms-field wpforms-field-name col-12 col-sm-12 col-md-12 form-group" data-field-id="20"><label class="wpforms-field-label" for="wpforms-8104-field_20">Name <span class="wpforms-required-label">*</span></label><input type="text" id="wpforms-8104-field_20" class="wpforms-field-large wpforms-field-required" name="wpforms[fields][20]" placeholder="Name" required></div><div id="wpforms-8104-field_2-container" class="wpforms-field wpforms-field-email col-12 col-sm-12 col-md-12 form-group" data-field-id="2"><label class="wpforms-field-label" for="wpforms-8104-field_2">Email <span class="wpforms-required-label">*</span></label><input type="email" id="wpforms-8104-field_2" class="wpforms-field-large wpforms-field-required" name="wpforms[fields][2]" placeholder="Email" required></div><div id="wpforms-8104-field_3-container" class="wpforms-field wpforms-field-text col-12 col-sm-12 col-md-12 form-group" data-field-id="3"><label class="wpforms-field-label" for="wpforms-8104-field_3">Organization <span class="wpforms-required-label">*</span></label><input type="text" id="wpforms-8104-field_3" class="wpforms-field-large wpforms-field-required" name="wpforms[fields][3]" placeholder="Organization" required></div><div id="wpforms-8104-field_11-container" class="wpforms-field wpforms-field-pagebreak" data-field-id="11"><div class="wpforms-clear wpforms-pagebreak-left"></div></div></div></div><div class="wpforms-field wpforms-field-hp"><label for="wpforms-8104-field-hp" class="wpforms-field-label">Phone</label><input type="text" name="wpforms[hp]" id="wpforms-8104-field-hp" class="wpforms-field-medium"></div><input type="hidden" name="wpforms[recaptcha]" value=""><div class="wpforms-submit-container" style="display:none;"><input type="hidden" name="wpforms[id]" value="8104"><input type="hidden" name="wpforms[author]" value="2"><input type="hidden" name="wpforms[post_id]" value="204"><button type="submit" name="wpforms[submit]" class="wpforms-submit om-trigger-conversion mon-btn" id="wpforms-submit-8104" value="wpforms-submit" aria-live="assertive" data-alt-text="Submitting..." data-submit-text="Submit">Submit</button></div></form></div> <!-- .wpforms-container -->
[wpforms id="8095"]
<div class="wpforms-container wpforms-container-full optin-monster-forms" id="wpforms-8095"><form id="wpforms-form-8095" class="wpforms-validate wpforms-form" data-formid="8095" method="post" enctype="multipart/form-data" action="/insights/machine-learning-model-testing/"><noscript class="wpforms-error-noscript">Please enable JavaScript in your browser to complete this form.</noscript><div class="wpforms-page-indicator progress" data-indicator="progress" data-indicator-color="#72b239" data-scroll="1"><span class="wpforms-page-indicator-page-title" ></span><span class="wpforms-page-indicator-page-title-sep" style="display:none;"> - </span><span class="wpforms-page-indicator-steps">Step <span class="wpforms-page-indicator-steps-current">1</span> of 2</span><div class="wpforms-page-indicator-page-progress-wrap"><div class="wpforms-page-indicator-page-progress" style="width:50%;background-color:#72b239;"></div></div></div><div class="wpforms-field-container"><div class="wpforms-page wpforms-page-1 "><div id="wpforms-8095-field_10-container" class="wpforms-field wpforms-field-pagebreak" data-field-id="10"></div><div id="wpforms-8095-field_24-container" class="wpforms-field wpforms-field-html form-popup-header-wrapper" data-field-id="24"><div id="wpforms-8095-field_24"><div class="form-popup-header"> <h2>RPA and Cognitive Automation</h2> <p>Making business processes smarter and more efficient</p> </div></div></div><div id="wpforms-8095-field_21-container" class="wpforms-field wpforms-field-radio custom-radio-btn-wrapper wpforms-list-3-columns" data-field-id="21"><label class="wpforms-field-label wpforms-label-hide" for="wpforms-8095-field_21">Sevices <span class="wpforms-required-label">*</span></label><ul id="wpforms-8095-field_21" class="wpforms-field-required wpforms-image-choices wpforms-image-choices-modern"><li class="choice-1 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8095-field_21_1" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/automation-anywhere.png" alt=""></span><input type="radio" id="wpforms-8095-field_21_1" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Choice 1" tabindex="-1" required ><span class="wpforms-image-choices-label"></span></label></li><li class="choice-2 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8095-field_21_2" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/uipath.png" alt=""></span><input type="radio" id="wpforms-8095-field_21_2" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Choice 2" tabindex="-1" required ><span class="wpforms-image-choices-label"></span></label></li><li class="choice-3 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8095-field_21_3" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/blueprism.png" alt=""></span><input type="radio" id="wpforms-8095-field_21_3" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Choice 3" tabindex="-1" required ><span class="wpforms-image-choices-label"></span></label></li><li class="choice-4 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8095-field_21_4" tabindex="0"><input type="radio" id="wpforms-8095-field_21_4" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Intelligent Automation" tabindex="-1" required ><span class="wpforms-image-choices-label">Intelligent Automation</span></label></li><li class="choice-5 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8095-field_21_5" tabindex="0"><input type="radio" id="wpforms-8095-field_21_5" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="RPA with AI" tabindex="-1" required ><span class="wpforms-image-choices-label">RPA with AI</span></label></li><li class="choice-6 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8095-field_21_6" tabindex="0"><input type="radio" id="wpforms-8095-field_21_6" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Open Source RPA Implementation" tabindex="-1" required ><span class="wpforms-image-choices-label">Open Source RPA Implementation</span></label></li></ul></div><div id="wpforms-8095-field_23-container" class="wpforms-field wpforms-field-pagebreak" data-field-id="23"><div class="wpforms-clear wpforms-pagebreak-left"><button class="wpforms-page-button wpforms-page-next" data-action="next" data-page="1" data-formid="8095">Next</button></div></div></div><div class="wpforms-page wpforms-page-2 last " style="display:none;"><div id="wpforms-8095-field_25-container" class="wpforms-field wpforms-field-html form-popup-header-wrapper" data-field-id="25"><div id="wpforms-8095-field_25"><div class="form-popup-header"> <h2>Let Us Discuss and Understand the Requirement</h2> </div></div></div><div id="wpforms-8095-field_20-container" class="wpforms-field wpforms-field-name col-12 col-sm-12 col-md-12 form-group" data-field-id="20"><label class="wpforms-field-label" for="wpforms-8095-field_20">Name <span class="wpforms-required-label">*</span></label><input type="text" id="wpforms-8095-field_20" class="wpforms-field-large wpforms-field-required" name="wpforms[fields][20]" placeholder="Name" required></div><div id="wpforms-8095-field_2-container" class="wpforms-field wpforms-field-email col-12 col-sm-12 col-md-12 form-group" data-field-id="2"><label class="wpforms-field-label" for="wpforms-8095-field_2">Email <span class="wpforms-required-label">*</span></label><input type="email" id="wpforms-8095-field_2" class="wpforms-field-large wpforms-field-required" name="wpforms[fields][2]" placeholder="Email" required></div><div id="wpforms-8095-field_3-container" class="wpforms-field wpforms-field-text col-12 col-sm-12 col-md-12 form-group" data-field-id="3"><label class="wpforms-field-label" for="wpforms-8095-field_3">Organization <span class="wpforms-required-label">*</span></label><input type="text" id="wpforms-8095-field_3" class="wpforms-field-large wpforms-field-required" name="wpforms[fields][3]" placeholder="Organization" required></div><div id="wpforms-8095-field_11-container" class="wpforms-field wpforms-field-pagebreak" data-field-id="11"><div class="wpforms-clear wpforms-pagebreak-left"></div></div></div></div><div class="wpforms-field wpforms-field-hp"><label for="wpforms-8095-field-hp" class="wpforms-field-label">Phone</label><input type="text" name="wpforms[hp]" id="wpforms-8095-field-hp" class="wpforms-field-medium"></div><input type="hidden" name="wpforms[recaptcha]" value=""><div class="wpforms-submit-container" style="display:none;"><input type="hidden" name="wpforms[id]" value="8095"><input type="hidden" name="wpforms[author]" value="2"><input type="hidden" name="wpforms[post_id]" value="204"><button type="submit" name="wpforms[submit]" class="wpforms-submit om-trigger-conversion mon-btn" id="wpforms-submit-8095" value="wpforms-submit" aria-live="assertive" data-alt-text="Submitting..." data-submit-text="Submit">Submit</button></div></form></div> <!-- .wpforms-container -->
[wpforms id="8085"]
<div class="wpforms-container wpforms-container-full optin-monster-forms" id="wpforms-8085"><form id="wpforms-form-8085" class="wpforms-validate wpforms-form" data-formid="8085" method="post" enctype="multipart/form-data" action="/insights/machine-learning-model-testing/"><noscript class="wpforms-error-noscript">Please enable JavaScript in your browser to complete this form.</noscript><div class="wpforms-page-indicator progress" data-indicator="progress" data-indicator-color="#72b239" data-scroll="1"><span class="wpforms-page-indicator-page-title" ></span><span class="wpforms-page-indicator-page-title-sep" style="display:none;"> - </span><span class="wpforms-page-indicator-steps">Step <span class="wpforms-page-indicator-steps-current">1</span> of 2</span><div class="wpforms-page-indicator-page-progress-wrap"><div class="wpforms-page-indicator-page-progress" style="width:50%;background-color:#72b239;"></div></div></div><div class="wpforms-field-container"><div class="wpforms-page wpforms-page-1 "><div id="wpforms-8085-field_10-container" class="wpforms-field wpforms-field-pagebreak" data-field-id="10"></div><div id="wpforms-8085-field_24-container" class="wpforms-field wpforms-field-html form-popup-header-wrapper" data-field-id="24"><div id="wpforms-8085-field_24"><div class="form-popup-header"> <img src="https://www.xenonstack.com/images/wednesday-digitally-transform-popup-img.png" /> <h2>Digitally Transform <span>your Organization </span></h2> </div></div></div><div id="wpforms-8085-field_21-container" class="wpforms-field wpforms-field-radio custom-radio-btn-wrapper wpforms-list-2-columns" data-field-id="21"><label class="wpforms-field-label wpforms-label-hide" for="wpforms-8085-field_21">Sevices <span class="wpforms-required-label">*</span></label><ul id="wpforms-8085-field_21" class="wpforms-field-required wpforms-image-choices wpforms-image-choices-modern"><li class="choice-1 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8085-field_21_1" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/continous-delivery-e1594278564177.png" alt="Continous Delivery and DevSecOps" title="Continous Delivery and DevSecOps"></span><input type="radio" id="wpforms-8085-field_21_1" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Continous Delivery and DevSecOps" tabindex="-1" required ><span class="wpforms-image-choices-label">Continous Delivery and DevSecOps</span></label></li><li class="choice-2 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8085-field_21_2" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/robotic-process.png" alt="Robotic Process Automation" title="Robotic Process Automation"></span><input type="radio" id="wpforms-8085-field_21_2" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Robotic Process Automation" tabindex="-1" required ><span class="wpforms-image-choices-label">Robotic Process Automation</span></label></li><li class="choice-3 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8085-field_21_3" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/big-data-analytics.png" alt="Big Data Analytics" title="Big Data Analytics"></span><input type="radio" id="wpforms-8085-field_21_3" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Big Data Analytics" tabindex="-1" required ><span class="wpforms-image-choices-label">Big Data Analytics</span></label></li><li class="choice-4 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8085-field_21_4" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/cloud-security.png" alt="Cloud Security" title="Cloud Security"></span><input type="radio" id="wpforms-8085-field_21_4" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Cloud Security" tabindex="-1" required ><span class="wpforms-image-choices-label">Cloud Security</span></label></li><li class="choice-5 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8085-field_21_5" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/data-science-ai-e1594278553510.png" alt="Data Science and AI" title="Data Science and AI"></span><input type="radio" id="wpforms-8085-field_21_5" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Data Science and AI" tabindex="-1" required ><span class="wpforms-image-choices-label">Data Science and AI</span></label></li><li class="choice-6 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8085-field_21_6" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/cloud-migration-e1594278582638.png" alt="Cloud Migration" title="Cloud Migration"></span><input type="radio" id="wpforms-8085-field_21_6" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Cloud Migration" tabindex="-1" required ><span class="wpforms-image-choices-label">Cloud Migration</span></label></li></ul></div><div id="wpforms-8085-field_23-container" class="wpforms-field wpforms-field-pagebreak" data-field-id="23"><div class="wpforms-clear wpforms-pagebreak-left"><button class="wpforms-page-button wpforms-page-next" data-action="next" data-page="1" data-formid="8085">Next</button></div></div></div><div class="wpforms-page wpforms-page-2 last " style="display:none;"><div id="wpforms-8085-field_25-container" class="wpforms-field wpforms-field-html form-popup-header-wrapper" data-field-id="25"><div id="wpforms-8085-field_25"><div class="form-popup-header"> <img src="https://www.xenonstack.com/images/wednesday-digitally-transform-popup-img.png" /> <div> <h2>How can we get in Touch</h2> <p>Fill the form and we will revert back to you soon.<p></div> </div></div></div><div id="wpforms-8085-field_20-container" class="wpforms-field wpforms-field-name col-12 col-sm-12 col-md-12 form-group" data-field-id="20"><label class="wpforms-field-label" for="wpforms-8085-field_20">Name <span class="wpforms-required-label">*</span></label><input type="text" id="wpforms-8085-field_20" class="wpforms-field-large wpforms-field-required" name="wpforms[fields][20]" placeholder="Name" required></div><div id="wpforms-8085-field_2-container" class="wpforms-field wpforms-field-email col-12 col-sm-12 col-md-12 form-group" data-field-id="2"><label class="wpforms-field-label" for="wpforms-8085-field_2">Email <span class="wpforms-required-label">*</span></label><input type="email" id="wpforms-8085-field_2" class="wpforms-field-large wpforms-field-required" name="wpforms[fields][2]" placeholder="Email" required></div><div id="wpforms-8085-field_3-container" class="wpforms-field wpforms-field-text col-12 col-sm-12 col-md-12 form-group" data-field-id="3"><label class="wpforms-field-label" for="wpforms-8085-field_3">Organization <span class="wpforms-required-label">*</span></label><input type="text" id="wpforms-8085-field_3" class="wpforms-field-large wpforms-field-required" name="wpforms[fields][3]" placeholder="Organization" required></div><div id="wpforms-8085-field_11-container" class="wpforms-field wpforms-field-pagebreak" data-field-id="11"><div class="wpforms-clear wpforms-pagebreak-left"></div></div></div></div><div class="wpforms-field wpforms-field-hp"><label for="wpforms-8085-field-hp" class="wpforms-field-label">Comment</label><input type="text" name="wpforms[hp]" id="wpforms-8085-field-hp" class="wpforms-field-medium"></div><input type="hidden" name="wpforms[recaptcha]" value=""><div class="wpforms-submit-container" style="display:none;"><input type="hidden" name="wpforms[id]" value="8085"><input type="hidden" name="wpforms[author]" value="2"><input type="hidden" name="wpforms[post_id]" value="204"><button type="submit" name="wpforms[submit]" class="wpforms-submit om-trigger-conversion mon-btn" id="wpforms-submit-8085" value="wpforms-submit" aria-live="assertive" data-alt-text="Submitting..." data-submit-text="Submit">Submit</button></div></form></div> <!-- .wpforms-container -->
[wpforms id="8051"]
<div class="wpforms-container wpforms-container-full optin-monster-forms" id="wpforms-8051"><form id="wpforms-form-8051" class="wpforms-validate wpforms-form" data-formid="8051" method="post" enctype="multipart/form-data" action="/insights/machine-learning-model-testing/"><noscript class="wpforms-error-noscript">Please enable JavaScript in your browser to complete this form.</noscript><div class="wpforms-page-indicator progress" data-indicator="progress" data-indicator-color="#72b239" data-scroll="1"><span class="wpforms-page-indicator-page-title" ></span><span class="wpforms-page-indicator-page-title-sep" style="display:none;"> - </span><span class="wpforms-page-indicator-steps">Step <span class="wpforms-page-indicator-steps-current">1</span> of 2</span><div class="wpforms-page-indicator-page-progress-wrap"><div class="wpforms-page-indicator-page-progress" style="width:50%;background-color:#72b239;"></div></div></div><div class="wpforms-field-container"><div class="wpforms-page wpforms-page-1 "><div id="wpforms-8051-field_10-container" class="wpforms-field wpforms-field-pagebreak" data-field-id="10"></div><div id="wpforms-8051-field_24-container" class="wpforms-field wpforms-field-html form-popup-header-wrapper" data-field-id="24"><div id="wpforms-8051-field_24"><div class="form-popup-header"> <h2>Build an AI-Driven Enterprise</h2> <p>unlock the potential of AI at scale</p> </div></div></div><div id="wpforms-8051-field_21-container" class="wpforms-field wpforms-field-radio custom-radio-btn-wrapper wpforms-list-3-columns" data-field-id="21"><label class="wpforms-field-label wpforms-label-hide" for="wpforms-8051-field_21">Sevices <span class="wpforms-required-label">*</span></label><ul id="wpforms-8051-field_21" class="wpforms-field-required wpforms-image-choices wpforms-image-choices-modern"><li class="choice-1 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8051-field_21_1" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/privacy-preserving-ai.png" alt="Privacy-Preserving AI" title="Privacy-Preserving AI"></span><input type="radio" id="wpforms-8051-field_21_1" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Privacy-Preserving AI" tabindex="-1" required ><span class="wpforms-image-choices-label">Privacy-Preserving AI</span></label></li><li class="choice-2 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8051-field_21_2" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/conversation-ai-platform.png" alt="Conversation AI Platform" title="Conversation AI Platform"></span><input type="radio" id="wpforms-8051-field_21_2" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Conversation AI Platform" tabindex="-1" required ><span class="wpforms-image-choices-label">Conversation AI Platform</span></label></li><li class="choice-3 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8051-field_21_3" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/enterprise-ai-platform.png" alt="Enterprise AI Platform" title="Enterprise AI Platform"></span><input type="radio" id="wpforms-8051-field_21_3" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Enterprise AI Platform" tabindex="-1" required ><span class="wpforms-image-choices-label">Enterprise AI Platform</span></label></li><li class="choice-4 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8051-field_21_4" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/continuous-delivery-machine-learning.png" alt="Continuous Delivery for Machine learning" title="Continuous Delivery for Machine learning"></span><input type="radio" id="wpforms-8051-field_21_4" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Continuous Delivery for Machine learning" tabindex="-1" required ><span class="wpforms-image-choices-label">Continuous Delivery for Machine learning</span></label></li><li class="choice-5 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8051-field_21_5" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/federated-transfer-learning.png" alt="Federated and Transfer Learning" title="Federated and Transfer Learning"></span><input type="radio" id="wpforms-8051-field_21_5" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Federated and Transfer Learning" tabindex="-1" required ><span class="wpforms-image-choices-label">Federated and Transfer Learning</span></label></li><li class="choice-6 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-8051-field_21_6" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/explainable-artificial-intelligence.png" alt="Explainable Artificial Intelligence (XAI)" title="Explainable Artificial Intelligence (XAI)"></span><input type="radio" id="wpforms-8051-field_21_6" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Explainable Artificial Intelligence (XAI)" tabindex="-1" required ><span class="wpforms-image-choices-label">Explainable Artificial Intelligence (XAI)</span></label></li></ul></div><div id="wpforms-8051-field_23-container" class="wpforms-field wpforms-field-pagebreak" data-field-id="23"><div class="wpforms-clear wpforms-pagebreak-left"><button class="wpforms-page-button wpforms-page-next" data-action="next" data-page="1" data-formid="8051">Next</button></div></div></div><div class="wpforms-page wpforms-page-2 last " style="display:none;"><div id="wpforms-8051-field_25-container" class="wpforms-field wpforms-field-html form-popup-header-wrapper" data-field-id="25"><div id="wpforms-8051-field_25"><div class="form-popup-header"> <h2>How can we get in Touch</h2> <p>Fill the form and we will revert back to you soon.<p> </div></div></div><div id="wpforms-8051-field_20-container" class="wpforms-field wpforms-field-name col-12 col-sm-12 col-md-12 form-group" data-field-id="20"><label class="wpforms-field-label" for="wpforms-8051-field_20">Name <span class="wpforms-required-label">*</span></label><input type="text" id="wpforms-8051-field_20" class="wpforms-field-large wpforms-field-required" name="wpforms[fields][20]" placeholder="Name" required></div><div id="wpforms-8051-field_2-container" class="wpforms-field wpforms-field-email col-12 col-sm-12 col-md-12 form-group" data-field-id="2"><label class="wpforms-field-label" for="wpforms-8051-field_2">Email <span class="wpforms-required-label">*</span></label><input type="email" id="wpforms-8051-field_2" class="wpforms-field-large wpforms-field-required" name="wpforms[fields][2]" placeholder="Email" required></div><div id="wpforms-8051-field_3-container" class="wpforms-field wpforms-field-text col-12 col-sm-12 col-md-12 form-group" data-field-id="3"><label class="wpforms-field-label" for="wpforms-8051-field_3">Organization <span class="wpforms-required-label">*</span></label><input type="text" id="wpforms-8051-field_3" class="wpforms-field-large wpforms-field-required" name="wpforms[fields][3]" placeholder="Organization" required></div><div id="wpforms-8051-field_11-container" class="wpforms-field wpforms-field-pagebreak" data-field-id="11"><div class="wpforms-clear wpforms-pagebreak-left"></div></div></div></div><div class="wpforms-field wpforms-field-hp"><label for="wpforms-8051-field-hp" class="wpforms-field-label">Comment</label><input type="text" name="wpforms[hp]" id="wpforms-8051-field-hp" class="wpforms-field-medium"></div><input type="hidden" name="wpforms[recaptcha]" value=""><div class="wpforms-submit-container" style="display:none;"><input type="hidden" name="wpforms[id]" value="8051"><input type="hidden" name="wpforms[author]" value="2"><input type="hidden" name="wpforms[post_id]" value="204"><button type="submit" name="wpforms[submit]" class="wpforms-submit om-trigger-conversion mon-btn" id="wpforms-submit-8051" value="wpforms-submit" aria-live="assertive" data-alt-text="Submitting..." data-submit-text="Submit">Submit</button></div></form></div> <!-- .wpforms-container -->
[wpforms id="7646"]
<div class="wpforms-container wpforms-container-full optin-monster-forms" id="wpforms-7646"><form id="wpforms-form-7646" class="wpforms-validate wpforms-form" data-formid="7646" method="post" enctype="multipart/form-data" action="/insights/machine-learning-model-testing/"><noscript class="wpforms-error-noscript">Please enable JavaScript in your browser to complete this form.</noscript><div class="wpforms-page-indicator progress" data-indicator="progress" data-indicator-color="#72b239" data-scroll="1"><span class="wpforms-page-indicator-page-title" ></span><span class="wpforms-page-indicator-page-title-sep" style="display:none;"> - </span><span class="wpforms-page-indicator-steps">Step <span class="wpforms-page-indicator-steps-current">1</span> of 2</span><div class="wpforms-page-indicator-page-progress-wrap"><div class="wpforms-page-indicator-page-progress" style="width:50%;background-color:#72b239;"></div></div></div><div class="wpforms-field-container"><div class="wpforms-page wpforms-page-1 "><div id="wpforms-7646-field_10-container" class="wpforms-field wpforms-field-pagebreak" data-field-id="10"></div><div id="wpforms-7646-field_24-container" class="wpforms-field wpforms-field-html form-popup-header-wrapper" data-field-id="24"><div id="wpforms-7646-field_24"><div class="form-popup-header"> <h2>Accelerate Digital Transformation with Intelligent Automation</h2> </div></div></div><div id="wpforms-7646-field_21-container" class="wpforms-field wpforms-field-radio custom-radio-btn-wrapper wpforms-list-2-columns" data-field-id="21"><label class="wpforms-field-label wpforms-label-hide" for="wpforms-7646-field_21">Sevices <span class="wpforms-required-label">*</span></label><ul id="wpforms-7646-field_21" class="wpforms-field-required wpforms-image-choices wpforms-image-choices-modern"><li class="choice-1 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-7646-field_21_1" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/real-time-data-analysis.png" alt="Real Time Data Analytics" title="Real Time Data Analytics"></span><input type="radio" id="wpforms-7646-field_21_1" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Real Time Data Analytics" tabindex="-1" required ><span class="wpforms-image-choices-label">Real Time Data Analytics</span></label></li><li class="choice-2 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-7646-field_21_2" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/data-visualization.png" alt="Interactive Data Visualisation" title="Interactive Data Visualisation"></span><input type="radio" id="wpforms-7646-field_21_2" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Interactive Data Visualisation" tabindex="-1" required ><span class="wpforms-image-choices-label">Interactive Data Visualisation</span></label></li><li class="choice-3 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-7646-field_21_3" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/application-modernisation.png" alt="Application Modernisation" title="Application Modernisation"></span><input type="radio" id="wpforms-7646-field_21_3" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Application Modernisation" tabindex="-1" required ><span class="wpforms-image-choices-label">Application Modernisation</span></label></li><li class="choice-4 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-7646-field_21_4" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/enterprise-ai.png" alt="Enterprise AI" title="Enterprise AI"></span><input type="radio" id="wpforms-7646-field_21_4" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Enterprise AI" tabindex="-1" required ><span class="wpforms-image-choices-label">Enterprise AI</span></label></li><li class="choice-5 depth-1 wpforms-image-choices-item"><label class="wpforms-field-label-inline" for="wpforms-7646-field_21_5" tabindex="0"><span class="wpforms-image-choices-image"><img src="https://www.xenonstack.com/wp-content/uploads/2020/07/intelligent-cognitive-automation.png" alt="Intelligent and Cognitive Automation" title="Intelligent and Cognitive Automation"></span><input type="radio" id="wpforms-7646-field_21_5" class="wpforms-screen-reader-element" name="wpforms[fields][21]" value="Intelligent and Cognitive Automation" tabindex="-1" required ><span class="wpforms-image-choices-label">Intelligent and Cognitive Automation</span></label></li></ul></div><div id="wpforms-7646-field_23-container" class="wpforms-field wpforms-field-pagebreak" data-field-id="23"><div class="wpforms-clear wpforms-pagebreak-left"><button class="wpforms-page-button wpforms-page-next" data-action="next" data-page="1" data-formid="7646">Next</button></div></div></div><div class="wpforms-page wpforms-page-2 last " style="display:none;"><div id="wpforms-7646-field_25-container" class="wpforms-field wpforms-field-html form-popup-header-wrapper" data-field-id="25"><div id="wpforms-7646-field_25"><div class="form-popup-header"> <h2>How can we get in Touch</h2> <p>Fill the form and we will revert back to you soon.<p> </div></div></div><div id="wpforms-7646-field_20-container" class="wpforms-field wpforms-field-name col-12 col-sm-12 col-md-12 form-group" data-field-id="20"><label class="wpforms-field-label" for="wpforms-7646-field_20">Name <span class="wpforms-required-label">*</span></label><input type="text" id="wpforms-7646-field_20" class="wpforms-field-large wpforms-field-required" name="wpforms[fields][20]" placeholder="Name" required></div><div id="wpforms-7646-field_2-container" class="wpforms-field wpforms-field-email col-12 col-sm-12 col-md-12 form-group" data-field-id="2"><label class="wpforms-field-label" for="wpforms-7646-field_2">Email <span class="wpforms-required-label">*</span></label><input type="email" id="wpforms-7646-field_2" class="wpforms-field-large wpforms-field-required" name="wpforms[fields][2]" placeholder="Email" required></div><div id="wpforms-7646-field_3-container" class="wpforms-field wpforms-field-text col-12 col-sm-12 col-md-12 form-group" data-field-id="3"><label class="wpforms-field-label" for="wpforms-7646-field_3">Organization <span class="wpforms-required-label">*</span></label><input type="text" id="wpforms-7646-field_3" class="wpforms-field-large wpforms-field-required" name="wpforms[fields][3]" placeholder="Organization" required></div><div id="wpforms-7646-field_11-container" class="wpforms-field wpforms-field-pagebreak" data-field-id="11"><div class="wpforms-clear wpforms-pagebreak-left"></div></div></div></div><div class="wpforms-field wpforms-field-hp"><label for="wpforms-7646-field-hp" class="wpforms-field-label">Phone</label><input type="text" name="wpforms[hp]" id="wpforms-7646-field-hp" class="wpforms-field-medium"></div><input type="hidden" name="wpforms[recaptcha]" value=""><div class="wpforms-submit-container" style="display:none;"><input type="hidden" name="wpforms[id]" value="7646"><input type="hidden" name="wpforms[author]" value="2"><input type="hidden" name="wpforms[post_id]" value="204"><button type="submit" name="wpforms[submit]" class="wpforms-submit om-trigger-conversion mon-btn" id="wpforms-submit-7646" value="wpforms-submit" aria-live="assertive" data-alt-text="Submitting..." data-submit-text="Submit">Submit</button></div></form></div> <!-- .wpforms-container -->