Smart Piano Prototype. Play the video, or visit the link, to see how it works.
matrix Keypad.h, which allows me to assign rows and columns to the keypad and checks for the button-press state.
2. Pitches.h library containing all the pitch value for typical notes, and I use it to assign it to each key. For example, when “button 1” on the 4x4 keypad is being pressed, the pitch for a middle C will play through a speaker.
Initial Design with blue dots
Anticipated Problem
With the current design, when a key is played the first time, a blue dot will show on the screen, reacting to the user’s action. However, if the same key is being pressed more than once, users will not see any feedback or indications on the screen, because it will just be the same blue dots piling up on each other at the same location when the same key is pressed more than once.
Testing
I did quick usability testing with two people, by observing them interacting with the prototype. The two participants both expressed interest when first seeing blue dots on the screen and hearing the sound playing out from the speaker as they press the keys. However, when they pressed the same keys again, they were confused because the screen did not provide any feedback, whereas they have expectations for the dot to perform some sort of actions. After all the keys have been filled with blue dots on the screen, they quit playing with the prototype.
Iterations
According to testing participants, users tend to seek immediate feedbacks when interacting with this prototype. I decided to keep track of the number of counts for each key and change the color based on the counts, as shown below.
Design iteration with colored dots
Project Recap
I started off the project by picking the Arduino tool kits I wanted to practice using. I thought the 4x4 keypad would allow the most diverse inputs from users and give me the flexibility to work with different input values and conditions. I first made sure I could print out the input values to the Serial Monitor, which was fairly easy to complete. Then I added functions based on different input values using if statements. One of the most challenging things was to assign distinct meanings and distinct output for each input value and to make the 4x4 keypad useful in a way that I am taking advantage of the 16 buttons.
With those challenges or requirements in mind, I came up with ideas about creating a calculator-like prototype, but I thought it was not meaningful enough, and the interaction would very minimal and clunky, especially with a 4x4 keypad, and there was not much I could visualize on the screen with just numbers unless it was a large set of data input, which would not need the keypad at all. The idea of creating a simple piano keyboard came to mind, I was inspired by talking to my cousin about her getting a new digital piano. My cousin is new to the piano, and she needs to use a piano learning APP, which guides beginner to play each note, and show the location of the note on a keyboard as well. My cousin would keep moving her eyes between the actual keyboard and the screen to make sure she is playing the right key. I thought it would be helpful if we can connect the keyboard with a screen, and the computer can detect which key you are pressing and do the checking for the user. A 4x4 keypad works perfect for creating a simple piano keyboard, and adding one speaker allows me to play the pitch of the note out loud.
Overall, I have been successful in connecting Arduino and writing my own processing codes to achieve the visual outcome. The current visualization was not as polished as I wanted. I tried to create the effect that a key was being pressed, and the entire key area will light up, but I was having difficulties drawing and mapping coordinates, so I ended up with colored dots simulating where players' fingertips would be on a keyboard.