Large-scale language models and other AI systems reach their full potential only if they are built by a wider range of people.
Kathy Pham Computer scientist and senior advisor at Mozilla. At Workday, he is vice president of AI and machine learning. Part-time lecturer at Harvard University.
In 2023, we learned a lot about AI models. For example, what might be possible, such as the discovery of new classes of antibiotics. What are the uncertainties, such as how exactly companies will integrate chatbots into their operations? We also explain how this technology will be regulated and deployed across government in accordance with the AI Bill of Rights Blueprint, the NIST Risk Management Framework, EU AI law, and the U.S. Executive Order on AI.
But for those in the industry, or those paying close attention, there was another big learning. The vast majority of people building the foundations of the most influential technologies of our time come from very narrow disciplines, such as computer science, software engineering, and more specifically machine learning. That's it.
It's no surprise that people who write code are computer scientists and programmers; that's our profession. But building these models is more than just code. It's important to identify use cases, make design decisions, and anticipate and mitigate potential harm. This means it's important to make sure your technology not only works, but that it's reliable. And this requires a skill set that goes beyond machine learning and traditional computer science.
I know this well from my own experience across academia, nonprofits, government, and industry. The hallmark of truly great technology is its productive application. Companies can only build what's best and right by bringing together experts across disciplines.
Five years ago, I published an essay about the effects of this bias in our industry and how we can fix it. I argued that if we want to change the technology industry — if we want software to be more fair and less harmful — we need to change the classes required for a computer science degree. .
Since then, AI has grown exponentially. With its vast computing power, more Unforeseeable and unintended harm. My first proposition remains true. That means the computer science curriculum needs to expand to include the humanities as well. However, these changes take time. At this time, there is also a need to directly break down the persistent engineering/non-engineering divide at the industry level.
There are no shortage of stories about the harms of AI systems, including misinformed hallucinations, disinformation, sexist and racist output, and harmful training sets. Often these are problems overlooked by engineers, but then faced by others, such as cognitive scientists and media theorists, after the fact. The order of operations is obviously reversed. These harms need to be addressed. in front The system will be deployed. Proactive rather than reactive responses should be the status quo, and that is only possible with more diverse expertise. It's true that some companies are making progress on this front with trust and safety teams. But those teams remain siled and fragmented.
There are some bright spots these companies can emulate. In recent years, Airbnb has been fighting discrimination in its product features, led by legal and civil rights expert Laura Murphy. In 2018, Salesforce established an “Office on Ethical and Humane Use” for the company's front-line employees, executives, and external experts from a wide range of departments to guide decision-making. (I was on the inaugural advisory board.) When AI really entered the zeitgeist, the office was ready to jump into action.
It's not that difficult for engineers to talk to lawyers and social scientists. Simple conversation — “Is it okay to create a Boolean value that allows two gender choices in a model?” — May avoid bias and the need for subsequent audits. I often think of my late father, who was a truck driver. There are countless engineers working to streamline the trucking industry. But how many people spend their time talking to actual truck drivers?
In addition to overt harms like prejudice, there are also harms of omission. A lack of humanities experts means missed opportunities for his AI systems with clear use cases and problem-solving capabilities. Today, many AI systems are first developed and deployed with the intention of finding a purpose later. Consider OpenAI's 128K context window or Anthropic's 200K context window. Although technically impressive, it has no clear goal until it is used in meaningful applications for society. Research and development without clear goals is not necessarily a bad thing, but nurturing these systems costs enormous amounts of money and energy. We also know that there are real goals, such as better cancer detection algorithms for people with darker skin. There are powerful examples of what is possible with this approach, such as Tanzania's Kiazi Bora app. Its creators first identified a problem, the lack of agricultural literacy among women in East Africa, and then built an AI chatbot to help solve it.
To improve the reliability of AI systems, Technology companies need to prioritize broad expertise. Yes, we are seeing encouraging progress on this front. Initiatives like All Tech is Human connect engineering to the social sciences and humanities, programs like Mozilla's Responsible Computing Challenge reimagine computer science curricula to include humanities research, and Workday's Responsible AI A team like this convenes an interdisciplinary board meeting.
What's next? Now we need to transform our industry to align with the accelerated development and deployment of AI.
