The Quest for Quality in AI
What happens when a storyteller meets the ‘king of testing’, a cyber security expert, artificial intelligence (AI) strategists and quality assurance leaders? A comedy of errors? No. Lots of musing, that is what happens.
I travelled to Dublin from Washington, DC where I’m the storyteller in chief and founder of IVOW, a start-up building cultural intelligence for AI. The majesty of Croke Park — Ireland’s largest sporting arena, home to Gaelic games, concerts and on this occasion, an event on artificial intelligence (AI) — was simply breathtaking. Software engineers, QA testers, data scientists, and AI specialists from as far as New Zealand,came together to engage around Quest for Quality 2019, organised by Comtrade.
Together with cyber security expert Nishan Chelvachandran, we talked about the future of AI and cultural intelligence. We suggested that identity is the commodity of today’s digital economy and for AI and personalisation to be relevant and effective, we need to solve for identity and design for diversity.
We also soft launched An Algorithm for Stories on Women, a crowdfunding and crowdsourcing campaign to source structured data on global stories. We begin with stories on women in history, culture, science and technology. Currently, there is no comprehensive algorithm to create AI-ready structured data around narratives of women who have inspired humanity across the centuries.
This data collection is necessary to provide critical historical context for the preservation of culture with an emphasis on the role of women. This is the first of 10 global AI and storytelling challenges between 2019-2029 that will introduce global stories to AI and advance Cultural IQ in AI systems.
The two-day summit was as informative as it was inspiring to a storyteller turned entrepreneur like myself. The opening keynote by Tariq King of Ultimate Software was especially profound as he walked us through pioneering work in AI-driven testing and test bots in action.
I asked several of the Quality Assurance leaders and speakers to share their insights on the state of AI today.
Tariq King, head of quality,Ultimate Software (US)
AI is reshaping our thoughts about the types of tasks that machines can do and causing us to evaluate where and how machines fit into our society culturally, ethically, and morally. The idea of developing machines that can simulate our cognition, intuition and/or emotion is exciting yet terrifying. On one side there is great promise and opportunity for technological advancement, and on the other side a fear that we will not be able to control this new technology. It is therefore not surprising to me that both of these sentiments are prevalent today in the quality and testing space.
In the last three years, I have seen an exponential growth in the number of conference presentations, panel discussions, and events around the intersection of AI and software testing. Several vendors and startups are boasting about their AI-powered testing tools and there are even a few open source projects out there. As someone who is very passionate about software testing, I am thrilled to see some much-needed innovation happening in our space. Testing has typically lagged behind development, but now it’s almost as if that problem is being flipped.
However, if AI is to be successful in any industry, it will require solid validation and verification practices to make sure these systems are functional, safe, secure, performant, culturally aware, law-abiding and ethical. This is where a community of testing professionals becomes invaluable.
I firmly believe that the testing community will help to pave the road as AI moves forward. Without them AI will be lost or unable to progress for fear of a takeover. A significant quality threat is that many AI-based systems are dynamically adaptive, and can therefore modify their own behavior after being released to production. As a result, these types of systems will need to be designed with self-testing in mind.
Nishan Chelvachandran, cyber security, cyber warfare, AI expert, RSA fellow (Finland)
We are at the intersection of technological and human evolution, where technology, and more specifically AI, is driving change and influence in humanity; how we think, what we do, and even why we do it. At this critical juncture, at the precipice of extensive proliferation and pervasion, the importance of a hybridised, diverse, equitable, and communal approach to address not only the emerging problems we see today, but also lay the groundwork to prevent the catastrophes of tomorrow is unparalleled.
When we think of cybersecurity, of course, the immediate thoughts are of the deep tech methodologies of the exploitation of vulnerabilities and APTS. We need to go beyond the specific attack vector, and take a more holistic view of the threat landscape.
The paradigm culture shift that we hear in the cyber echo chamber needs to happen across sectors, where diversity of thinking, skills, perspective, and experience must all play their part in design, testing, and implementation of the tools and mechanisms driving our digital society.
The DevOps cycle must evolve to bring testers together with developers, where a true continual V&V process can be implemented into the larger DevOps cycle.
For a truly human-centric digital society, we must encapsulate and empower everyone, and feed the data and models that are missing. This approach is embodied in Cultural IQ. For starters, an algorithm for stories women: the women’s dataset challenge.
Katja Obring, test consultant Infinity Works (UK)
In my view, AI is the most exciting thing happening in software development right now. This does not necessarily mean AI is the silver bullet to solve all our problems, and in fact I believe it is going to create challenges we do not even dream about right now.
Humans tend to invent and implement in an overly optimistic fashion — we find something new, and then we push it. We are a species of boundary pushers. Let us collect all the data to train our models, with little regard to the dark side of it, and less regard to the bias the choices we make can create.
And what if AI does not take off? I was involved in a company in the early days of the VR hype, and that never took off the way we envisioned. It did create what can only be described as a virtual graveyard, where masses of data are available for anybody to access, while the creators of those pixel fragments have moved on, never logged in again or passed away.
The stories they created live on, and I am excited about the idea of being able to tell many sides of the same story, to responsibly represent the ancient and the very modern in entirely new ways, to make the invisible visible and lend a voice to many storytellers that are not often heard in mainstream media.
Yasar Sulaiman, VP, quality assurance, Everest Reinsurance (US)
There is so much hype about AI and naturally, the worry among many professionals that it is here to take over the world. While there is some truth in AI affecting or even causing loss of some of the manual jobs, we are still scratching the surface of what is possible with AI. Despite so much progress made with AI all around us, most of it is still narrow and specific. We are at a stage in AI journey which many researchers call ANI — Artificial Narrow Intelligence — also called ‘weak AI’.
Some of the things that a toddler can learn easily are still very difficult for AI but what is fascinating is what this technology promises and potentially is capable of.
AI’s impact on the global workforce cannot be ignored. As per the world economic forum, AI and machine learning will directly result in loss of about 75 million jobs by 2022 but at the same time, AI will also help create 133 million new jobs.
So, yes we are losing some jobs but at the same time, we are really adding 58 million net new jobs. This will require re-skilling and re-training at a mass scale.
As per an IBM survey, about 130 million people will need re-skilling by 2030 to remain relevant in the age of AI.
James Farrier, CTO, Appsurify (NZ)
I completely agree with Yasar — there has been endless hype about the spread of Artificial Intelligence in society, and more than our share in the software QA testing world.
The point I wanted to emphasise in my own talk was not instead of AI taking jobs away from testers, AI is a tool to improve their jobs.
For years, we have complained that we do not have enough time to test, and yet we need to test more; that is why I think AI shouldn’t be feared and instead should be embraced to move QA earlier in the SDLC.
I have spent the past two years building a tool that uses machine learning to make existing testing processes run more efficiently.
I loved what Jason Jerina said about test effectiveness (how many tests do we need to find a bug) and that we should be striving to get to a 1:1 ratio, which is what we at Appsurify are helping to achieve by using AI to find which tests in the test suite are actually needed for the specific code changes.
AI can make the testing process faster, help QA to shift left, build quality into the requirements, and take advantage of production data to gain a better understanding of the user and what we should be building.
But AI is not without challenges, including built-in biases, which is why I was so happy to hear about the dataset challenge to help women become more involved in the evolution of technology where diversity is essential to success.
Lina Zubyte, QA Consultant, ThoughtWorks (Germany)
It is essential not to get carried away with the hype of AI without reasoning. As humans we tend to get involved into “solutioning” easily: we hope that there will be this one big thing that will solve our problems or even replace us. And, at this stage AI seems to be the one thing we tend to speculate on, but will it actually solve all our problems?
In Q4Q we clearly heard from various professionals that actually it won’t solve everything (some problems that currently we try to solve with AI could be solved in much simpler ways), however, it can help enormously. There are various tools that could assist us to do our work better and faster. However, we definitely need more mature team structures in order to be able to embrace the AI possibilities. We need to get ready for what AI has to offer.
One of the greatest benefits I see from starting the conversation about AI in general is the fact that it encourages learning, curiosity, and, especially, the ethical side of things. It is a topic that tends to resonate with us by awakening empathy when witnessing examples of AI gone wrong: there are so many examples of biased implementations which can be even insulting to certain groups of people. This makes me feel that our development teams must be extremely diverse, we have to make sure our products are inclusive.
And what about testing AI? Apart from super diverse, good-natured, ethically careful teams doing that, what we need are open source datasets, and, in general, a better understanding of humans. In my talk, I shared what challenges I faced testing a machine learning driven chatbot.
The main learning I had was actually the fact that humans are so multi-dimensional, so to create a great conversational flow is extremely challenging. Not even to mention how tricky is measuring the quality of that (especially in automated ways). In order to create high-quality AI products, we first have to understand human communication and culture better.
Jason Jerina, CVP, quality engineering, New York Life Insurance (US)
While the individuals in attendance had varying points of view and perspectives of need, it felt that the ‘community’ came together to acknowledge the acceptance of AI in the Quality Engineering space.
While there was not a one-answer-fits-all message, the collaboration seen made it feel like the community has embraced what this means for it. It likely is that next ladder of innovation for us, just as the car was for transportation or the Internet was for global communication.
Rhealyn Mugri, PMP, IPMA-D, Fiserv Ireland project leader (Ireland)
We are living in an exciting time of the 4th industrial revolution and I think it is the best time to be human. It’s the beginning of the merging of technologies creating almost all science fictions now within reach.
AI has been around for years and we are now starting to realise its untapped potential towards all areas of our life, in our society, within businesses, and to the world. I think AI will change not only the way we live and work. It will also change the way we think and behave as humans. It is going to fill the gaps in the society with opportunities now available for all those who embrace change with no excuses.
Those who can adapt to change quickly will benefit sooner than the rest. AI will not only be for those who are technical, it will also be for those who have great passion towards specific niches and use their strength with the help of AI. I have witnessed this during the Quest for Quality Conference. It made me realise that AI is not only for the few. It is for the many who can embrace change with a clear vision of what they want for the future.
Follow #Q4Q2019 to check out videos and photos from this year’s Quest for Quality summit.
Davar Ardalan is the founder and storyteller in chief of IVOW, building cultural intelligence for AI development.
Subscribers 0
Fans 0
Followers 0
Followers