Importance of AI Education

June 2nd, 2020 - Lane Arnold

My First Task as Associate VP of Engineering

One of my first tasks as Associate VP of Engineering at BetaFlix was to develop an internal AI program that integrates AI into several domains: traditional software development, proposal writing, and vertical slice prototypes. Upon doing some market research and consulting with our in house AI experts to accomplish this task, I came to a curious conclusion: there is immense misunderstanding and a general lack of education when it comes AI tools and pipelines.

The Inverse Pyramid

During the creation of one of our first programs, we discovered some crucial things: employees that had either little experience with the limitations of their AI tools, or those who were not experts in the field that they were using the tool in suffered greatly in productivity.

This is a painful lesson that the Tech industry is experiencing right now. AI tools and pipelines are still in their infancy, and are not established SaaS tools as some Marketing and Sales types would lead you to believe. As such, employees need to know the AI tools' limitations and tendencies to hallucinate as much as its capabilities to accelerate productivity. Lets look at a hypothetical example. An experienced developer in React but little experience with AI tools. They are tasked with adding a calender component to an account web page that syncs with tasks that are assigned to them. This developer would prompt the agentic pipeline to "add a calender component to web page x, and have it hit /getTasksDates for the data". This simple prompt will cause massive problems. How? The agentic pipeline is missing massive context to effectively create this component in the right way, in the right location, with the right features for the project. This prompt is missing: the component file name and location, what should it's parent and child components be, making sure the component is rendered the same way all the other components are rendered, etc. More often than not, the agent will hallucinate or put files/components in the wrong place, add unnecessary libraries, etc. If the web-developer was trained and educated on the tool's limitations, they would come up with a much more effective prompt: "Create a calendar component on web page x. Have the location of the component file be in the /components folder named calender.js and have it rendered in account.js in folder /features. Have it render the component in account.js using method x in location y." This prompt with the knowledge of the limitations of the tool will result in much more usable code that the agentic pipeline spits out, with a higher likelihood that the agent will only touch the files and methodologies that is specified from the developer.

Additionally, these tools do NOT 1:1 replace decades of industry knowledge and experience of personnel. The though process was simple: replace the expensive, experienced employees with cheaper, younger ones and have these AI tools fill the knowledge gaps. Companies are rapidly finding out that this is a pipedream and results in reduced productivity and increased costs. Ford was one of the first large corporations to learn this painful lesson. They attempted to replace experienced QA engineers with AI assisted automated tools to help speed up QA and cut costs. What resulted was unpredictable and unreliable systems that resulted in them going back to the "old ways" and rehiring more than 300 veteran inspectors.

To be clear, these AI tools are fantastic and can greatly increase productivity across the board. However, you need to have experienced personnel to be trained in the limitations of these tools to be able to use them effectively. AI tools are not (yet) a replacement for anything, but an enhancement to existing personnel and their expertise. However, AI education can go along way in helping both leadership and workers more effectively use AI in the workplace.