Creatively Using ARTS to Modify Speech and Protect Privacy

June 20, 2023

With the explosion and ubiquity of the Internet of Things in modern life, most people are accustomed to their electronic devices being “always on” and “listening” and asking their mobile phone, tablet, or smart speaker for recipes or directions. However, the trade-off for the convenience smart devices offer is the risk that they may also record one’s voice with or without consent.

While the private sector’s primary motivation in recording or retaining a person’s voice may be monetary, others — malign governments, terrorists, and other bad actors — likely have other motives. Namely, they may seek to identify someone through voice identification software or by other means in order to suppress or target that person.

One’s voice contains a wealth of personal sensitive information that can be exploited by privacy threats, such as biometric recognition, background profiling, and emotional screening. A person’s speech can reveal not only their identity, but also static traits such as dialect, gender, and age, as well as dynamic traits, such as fear, stress, and anger.

Given the privacy concerns that speech may convey, it’s not surprising some people may wish to speak anonymously. For example, someone protesting government oppression may want to avoid censorship or retaliation. Others may want to simply protect their privacy.

For the Intelligence Community (IC), the need to protect the identity of an employee or someone who is affiliated with the IC is often critical to staying safe and effective.

To help with this, the Intelligence Advanced Research Projects Activity (IARPA) developed the Anonymous Real-Time Speech (ARTS) program. The ARTS program aims to develop new speech modification technologies to alter an individual’s speech in real-time so that the modified speech can’t be attributed to the actual speaker.

“Anonymity and discretion are vital to the IC’s mission success, and the ARTS program will give the IC an important new tool to help facilitate mission priorities,” said ARTS Program Manager, Dr. Mark Becker.

The ARTS program will seek to develop capabilities to overcome privacy threats, such as speaker identification software, human evaluation of one’s dialect and age, and artificial intelligence and machine learning assessments of whether a person is nervous or angry. To achieve this, the goal is to create novel software-based systems suitable for conversations, with transformed speech that is understandable, natural, and operable in real-time.

A 36-month effort, the ARTS program will initially focus on the English language before transitioning to Spanish and other commonly spoken languages.

“While we’re just in the beginning stage of the program, I’m very excited about the possibilities ARTS presents,” Dr. Becker said. “If we’re successful, I believe ARTS will provide the IC a significant advantage.”

To help facilitate interest in and provide more information about ARTS and other IARPA programs, a Proposers’ Day is usually held shortly before the release of a new Broad Agency Announcement (BAA) for an IARPA program. During a Proposers’ Day, the IARPA program manager will present his/her vision for the program, outlining the material in the BAA before it is formally issued, enabling potential performers to prepare to respond to the BAA when it is released.

For those interested in the upcoming ARTS Proposers’ Day on June 27, they can find more information and register here. The last day to register is Thursday, June 22 before 5 pm EDT.

Creatively Using ARTS to Modify Speech and Protect Privacy Logo