An NSF & IES AI Institute for Transforming Education for Children with Speech and Language Processing Challenges
The National AI Institute for Exceptional Education (AI4ExceptionalEd) is a led by the University at Buffalo and funded by the and the of the US Department of Education.
The Institute is a partnership of multidisciplinary faculty from 8 other institutions viz., , , , , , , , and .
Millions of children who require speech and language services under the are at risk of falling behind in their academic and social-emotional development without timely intervention by speech-language pathologists (SLPs). Unfortunately, there is a significant shortage of SLPs making it almost impossible for them to provide individualized services for these children.
The National AI Institute for Exceptional Education aims to close this gap by developing advanced AI technologies to scale SLPs’ availability and services. Towards this end, AI4ExceptionalEd is in the process of developing two novel AI solutions: (1) the AI Screener to enable universal early screening for all children, and (2) the AI Orchestrator to work with SLPs and teachers to provide individualized interventions for children with their formal Individualized Education Program (IEP).
"By advancing AI techniques, human-AI interactions and learning science, this Institute will develop entirely new ways to identify and assist millions of children — and their teachers — across the United States in need of speech and language services. This effort will empower generations of young people with a high-quality education, opening a lifetime of bigger and better opportunities than ever available to them before." said NSF Director Sethuraman Panchanathan.
The UB-led initiative — in partnership with leading research universities, school districts and non-profits — will develop, test and scale AI systems that ensure children with speech and language disorders receive timely, effective assistance.
Photographed NSF Director, Sethuraman Panchanathan speaking at UB's "AI Chat Series," where he shared NSF's vision for the future and AI's role in it.
Photo: Douglas Levere
Solution 1: AI Screener
AI Screener will enable universal early screening for all children.
An edge-based solution, AI Screener will analyze video and audio streams of children’s classroom interactions, derive conventional speech and language measures used by SLPs, as well as assess novel and hard to obtain automaticity measures.
Speech, behavior, handwriting and language are some of the indicators being considered for AI4ExceptionalEd's AI Screener solution. A computer would process this data automatically and identify critical snippets of these interactions that a speech-language pathologist can quickly analyze and screen candidates for further evaluation. The use of multiple modalities such as visual, audio and textual data simultaneously by the AI system to effectively screen for speech and language issues will be a significant feature of the AI Screener.
ADD caption: The digitizing tablet pictured is a prototype of the product “IoT Paper” supplied by Wacom, Inc., an industry partner of AI4ExceptionalEd. Data being collected using the tablet will be used for screening for dyslexia and dysgraphia.
Solution 2: AI Orchestrator
AI Orchestrator will assist SLPs and teachers to provide individualized interventions for children with their formal Individualized Education Program (IEP).
AI Orchestrator will leverage multi-modal data of interactions, AI-assisted progress monitoring and a knowledge-base to deliver ability-based interventions in a scalable model. It will enable SLPs and teachers to collaboratively develop custom-based interventions for each child by:
- Analyzing collected multi-modal data, map each child’s learning targets and challenges by calculating relevant metrics automatically
- Recommending individualized intervention plans with proper parameters out of a myriad of existing and new methods
- Monitoring each child’s progress on a task and recommending the next intervention after the child achieves the desired goal
- Testing novel intervention methods, such as adaptive text rewriting and enrichment, and quantifying the benefits
Our senior leadership team represents accomplished researchers, thought leaders and educators who are committed to using AI for social good. They are the fulcrum of the governing structure of the Institute.
The senior leadership team meets regularly to ensure that AI4ExceptionalEd's numerous research and community activities are well-concerted and aligned with its goals.
Collectively, members of the team: coordinate the Institute’s end-to-end scientific research agenda; are responsible for driving research collaborations, timelines and deliverables; and manage day-to-day business including staffing, resource allocation, communication and reporting. They ensure compliance with research guidelines within the Institute including Institutional Review Board (IRB) and other processes.
Venu Govindaraju, PI
Institute Director
University at Buffalo
Pamela Hadley, Co-PI
University of Illinois Urbana-Champaign
Jinjun Xiong, Co-PI
Scientific Director
University at Buffalo
Julie Kientz, Co-PI
University of Washington
Srirangaraj (Ranga) Setlur
Managing Director
University at Buffalo
David Feil-Seifer, Co-PI
University of Nevada, Reno
