Building a local voice assistant with open source tools

Voice assistants like Amazon Alexa or Google Assistant allow to control numerous functionalities in the smart home via spoken language. For many people, however, such a cloud-based solution is out of the question due to considerable concerns about security, data protection and privacy, although they would like to use the functionality of voice control.

The implementation of a local «offline» voice assistant (which does not require any internet access) with open source tools is a solution to eliminate these security concerns and to exploit the potential of this interested customer group.

This workshop will show what is currently possible with open source tools. By the end of the workshop the participants should know how to build a local voice assistant.

Tutorial Outline (half-day event)

  • Welcome / Introduction
  • Motivation
  • Introduction to the architecture and components
    • Voice Interface
    • Speech Recognition
    • Natural Language Understanding
    • Dialog Management
    • Speech Synthesis
  • Demo “End-to-end voice assistant with Rasa and Mozilla DeepSpeechMozilla TTS open source tools”
  • Hands-on Session Building your own local voice assistant
  • Outlook & Discussion

Target Audience & Requirements

We target a broad audience interested in the implementation of a local voice assistant. We will provide sample code in python, so for the hands-on part, it would be good to have some experience installing python packages and intermediate Python skills.

Note: Participants must bring a laptop.


iHomeLab, Research Center for Building Intelligence, Lucerne University of Applied Sciences and Arts

Guido Kniesel, Senior Researcher

Education: Computer Science with emphasis on Artificial Intelligence, University of Applied Science Berlin / City College of New York, Higher National Diploma, 1994

Professional Experience: Research & Development in the areas of Smart Energy Management, Active Assisted Living and Internet of Things. Topics: Machine Learning / Deep Learning / Natural Language Processing (NLP) / Non-Intrusive Load Monitoring (NILM) / Project Management

Note: Depending on the number of participants a second instructor will participate.