Category: Cool Projects

Chatbot with Rivescript & p5js

I recently came across a very cool scripting language; Rivescript, which can be used in chatbots or other conversational entities. It’s a plain text keeps simple replies simple; for example:

This will add a reply so that when a human says the words “Hello bot”, the bot would respond with “Hello, human!”

With more advanced RiveScript code we can learn and repeat user variables and use more complicated trigger matching patterns:

This looks like AIML (Artificial Intelligence Markup Language), doesn’t it?

Here’s a comparison of Rivescript vs. AIML

In my latest project; I used p5js and it’s Speech to Text (STT) and Text to Speech (TTS) libraries to create this little chatbot program.

Please try in Chrome and you will need to give permission to microphone.

Please click on me to speak with BlueBot

Please see my previous post to learn more about p5js and several cool Machine Learning projects.

Please click on me to see the blog post

Machine Learning Projects with JavaScript

I’m very excited to have completed 3 fun Machine Learning projects. I will try to explain in this blog how this works and include the resources/links.

My first project is Speech Recognition, which repeats what the user says on a colorful canvas.

2nd fun project is Draw , which recognizes user’s command and draws on the canvas.

And Body Parts via PoseNet processes the image it sees via the camera and recognizes various body parts.

All the points the PoseNet model can identify. Source : https://github.com/tensorflow/tfjs-models/tree/master/posenet

These projects were done by p5js, which is a JavaScript library for creative coding.

You could immediately start coding by using p5js’s online editor. https://editor.p5js.org/

There are tons of fun examples on the website: https://p5js.org/examples/.

I incorporated ml5js libraries for the machine Learning portion of the code. ml5.js is machine learning for the web in your web browser. Through some clever and exciting advancements, the folks building TensorFlow.js figured out that it is possible to use the web browser’s built in graphics processing unit (GPU) to do calculations that would otherwise run very slowly using central processing unit (CPU).

PoseNet is a machine learning model that allows for Real-time Human Pose Estimation.

PoseNet can be used to estimate either a single pose or multiple poses, meaning there is a version of the algorithm that can detect only one person in an image/video and one version that can detect multiple persons in an image/video.

The original PoseNet model was ported to TensorFlow.js by Dan Oved. For background, read Real-time Human Pose Estimation in the Browser with TensorFlow.js.

p5js library can be found on github here: https://github.com/processing/p5.js

ml5js library can be found on github here: https://github.com/ml5js

ml5js libraries currently support Chrome, so please use Chrome to review my sample projects above.

Also, check out Daniel Shiffman’s Coding Train youtube channel for very helpful and entertaining tutorials.

I hope you enjoy these 3 fun projects and hope they inspire you to learn more about these technologies.

Thank you! And Happy Valentine’s Day!

Ebru

Amazon’s Alexa

Unless you’ve been living on another planet, you’ve probably heard about Amazon’s Alexa. It is a pretty cool cloud-based voice service.

What can you do with Alexa?

I’ve been looking into the technology behind Alexa. At the high level, it’s simple, yet elegant.

I believe Alexa uses SSML; Speech Synthesis Markup Language, when converting Text to Speech (TTS); because she sounds very conversational; rather than a robot reading the text word by word.

Here are a few more technical diagrams of how it works at the high level:

ASR Overview

I thought that the developing an Alexa Skill was straight forward and user friendly; especially if you have used any ML/AI tools. Navigation and setup look similar to others.

I created a few new test skills by using existing templates and added new custom intents. It was fun and I can see by a little bit of creativity, some great skills can be added to this fun smart tool!

Please watch my demo down below:

  • First skill is a game template where I added new custom intents.
  • Second skill is calling a fun external API that returns the number of astronauts currently in space and their names. 🙂 http://api.open-notify.org/astros.json

Another interesting video about “Lessons Learned Growing Alexa” and a few fun capabilities/skills that the Amazon Team discusses.

Bonus point: If you’d like to do a deep dive and create Alexa Skills with serverless backend, this youtube video should help and here are some more technical diagrams from the presentation:

Build a Serverless Back End for Your Alexa-Based Voice Interactions

IAM Management Flow

PS: Photos/Graphics are from Amazon.com

Object Detection & Identification

Via YoloV3, TensorFlow, Python

Yolov3 is an algorithm that uses deep convolutional neural networks to perform object detection.

In this quick project, I used the code that converts YoloV3 official already trained weights into TensorFlow models.

Once converted, I run the detect python code to use TensorFlow models to detect and identify the objects. I think it did pretty good.

As you can see in the short video below, it did detect my refrigerator, books on the shelves, bottle, myself as a person, my cell phone and my cup, a teddy bear toy and potted plants. 🙂

Output Video of Object Detection

Very exciting and these are some of the technologies that are and will be utilized in self-driving cars.

Please also check out my Face Detection project, here:

Face Detection and Recognition