COS 126

Digital Signal Processing		 Programming Assignment

Due: Wednesday, 11:59pm

Write a program to generate sound waves, apply an echo filter to an MP3 file, and plot the waves using Turtle graphics.

A crash course in sound. A music note can be characterized by measuring its frequency over time. For example, the music note A is a sine wave repeated 440 times per second (440 Hz); the note C is a sine wave repeated 523.25 times per second. We amplify the signal to an audible level by multiplying it by a constant, say 8,000. Below are pictures of an A and a C with duration 15 milliseconds and maximum amplitude of 8,000. The picture for A consists of 0.015 × 440 = 6.6 sine waves.

Below is a table containing the frequencies in hertz of all musical notes in the fifth octave.

A A# B C C# D D# E F F# G G#
440.00 466.16 493.88 523.25 554.37 587.33 622.25 659.26 698.46 739.99 783.99 830.61

To lower a note by one octave, divide it's frequency by two; to raise it by one octave, multiply it's frequency by two. For reference, the normal range of human hearing is between 20 and 20,000 Hz.

Digital audio. Digital audio is produced by sampling the instantaneous amplitude of the continuous sound wave many times a second. Each sample is a signed 16-bit integer (a short in Java) that represents the amplitude of the sound wave at a particular instant in time. The sampling rate is the number of samples taken per second. Audio from CDs typically uses a sampling rate of 44,100 and two channels (left and right). Below is a picture of an A using a sampling rate of 44,100 and a duration of approximately 1/440th of a second.

The following arrays contain the short integer representing the height of the wave above, at a sampling rate of 44,100: 

left  = { 0, 501, 1000, 1495, 1985, 2466, 2938, 3399, 3846, 4277, 4693, ..., -113 }
right = { 0, 501, 1000, 1495, 1985, 2466, 2938, 3399, 3846, 4277, 4693, ..., -113 }
Sample i is given by 8000 sin(2 π * 440 * i / 44,100), and we've rounded the numbers towards zero.

Sound synthesis. Your first task is to create a data type Wave to store and manipulate sound wave samples so that the program A.java below plays an A for 2 seconds with maximum amplitude 8,000. Similarly, FurElise.java plays the first nine notes of Fur Elise. 

  public class A {
      public static void main(String[] args) {
          Player.open();
          Wave A = new Wave(440.0, 2.0, 8000);  
          A.play();
          Player.close();
          System.exit(0);
      }
  }

For this part of the assignment, you need to write a data type Wave to initialize and access the data. Implement the samples for the left and right channels using two arrays of type short. First, implement the constructor 

public Wave(double Hz, double seconds, double amplitude)
It should create a new Wave object that represents a sine wave of the specified number of Hz that is sampled 44,100 times per second over the specified number of seconds. Use Math.sin and Math.PI to initialize the left and right channels. Initialize both arrays with the same values so that the note will play in both speakers. Next, implement a method 

public void play()
that sends the Wave data to the sound card. Use the static library method Player.playWave(left, right), which takes as input two short arrays representing the left and the right channel.

Compilation and execution. In order to play music on your computer, you will need to use a specialized library. Download the file player.jar and put in your working directory. A .jar file is an archive like a .zip file, but there is no need to unzip it or peek inside unless you are extremely curious. This library is a modified version of the JavaLayer 0.2 MP3 Player. (As per the GPL license, the jar file contains the original JavaLayer library and our modified source code.) To test the library on your system, use the JavaLayer Player that comes with player.jar by typing the following command: 

java -classpath player.jar javazoom.jl.player.jlp felten.mp3
You can replace felten.mp3 with the name of another MP3 file in the working directory. In order to use the library in your Java program, you must put the following import statement at the beginning: 
import javazoom.jl.player.Player;
You also need to tell Java where to find the library by compiling and executing with the following commands:

  Compile Execute
Unix  javac -classpath .:player.jar A.java  java -classpath .:player.jar A
 Windows   javac -classpath .;player.jar A.java  java -classpath .;player.jar A

Creating a tune. Now that you can play single notes, your next challenge is to play several notes at once, for example, to play a chord. To accomplish this, first add a new constructor 

public Wave (short[] left, short[] right)
to Wave.java that takes two short integer arrays as arguments and initializes a new Wave object with the given data. Now, to create a chord, you can create individual notes and combine them together by writing a method: 
public static Wave add(Wave a, Wave b)
that adds the Waves a and b and returns the resulting sum. To add two Waves, add the corresponding array entries for the left and right channels, element-by-element. Test your data type by using the program StairwayToHeaven.java, which is the beginning of the famous Led Zeppelin tune. Note the fifth wave played is created by the following sequence of statements: 
  Wave B6  = new Wave(493.88 * 2, 0.4, 8000.0);
  Wave Gs4 = new Wave(830.61 / 2, 0.4, 8000.0);  
  Wave GsB = Wave.add(B6, Gs4);

Score to Stairway to Heaven

Playing an MP3 file. The program MP3Player.java decodes the MP3 file specified by the command line input and plays it. Assuming that you implemented all of the Wave.java methods and constructors above, there is no need to write any code for this part (but you should test it and enjoy). Compile and execute the program with the following commands. 

javac -classpath .;player.jar MP3Player.java
java  -classpath .;player.jar MP3Player felten.mp3
The key part of the code is the following loop. It will be useful for the remaining parts of the assignment. 
  Player.open(args[0]);
  while (!Player.isEmpty()) {
      short[] left  = Player.getLeftChannel();
      short[] right = Player.getRightChannel();  
      Wave w = new Wave(left, right);
      w.play();
  }
This program uses three new methods from the MP3 Player library to decode data from an MP3 file. The String argument to the method Player.open specifies which MP3 file to use. The method Player.getLeftChannel returns an array of 1,152 short integers which are the samples of the music intended for the left speaker. The method Player.getRightChannel is analogous.

Echo filter. Now that you can decode and play MP3 files, you're ready to modify the data and change the characteristics of the sound waves. An analog filter accomplishes this by manipulating the electrical signals that represent the sound wave; a digital filter does this by manipulating the digital data that represents that Wave. Your task is to write a program EchoFilter.java that implement a digital echo filter. An echo filter of delay 10 is a filter that adds an echo to the sound by adding the sound wave at time t - 10 to the one at time t. To create this effect, maintain an array of the past 10 Wave objects and add the current Wave to the one that was originally played 10 waves ago. The echo filter is a client program and should be written entirely in EchoFilter.java. To test the filter, you can use pearlharbor.mp3 which contains the speach President Roosevelt delivered after the attack on Pearl Harbor.

Plotting the waves. The final part of the assignment is to write a program MP3Viewer.java that takes the name of an MP3 file as a command line argument and animates both stereo channels. This program is not supposed to play the MP3 file, only to animate the sound waves. Add the following method to Wave.java to plot the left channel on the top half of the screen, and the right channel on the bottom half: 

public void draw()
Use a 576-by-512 window and scale the data to fit snugly in the window. We use 576 because it divides evenly into 1,152, the number of samples in one channel of an MP3 wave. When rescaling, recall that each sample is a short integer between -32,768 and 32,767.

Challenge for the bored. Write a program MP3Visualizer.java that plays the MP3 file and simultaneously displays a cool effect based on the raw data. To keep the music and animation smooth, you may need to tweak a few parameters. For example, adjust the delay in the Turtle.pause method to keep the wave from scrolling by too fast. Also, try plotting every other wave if your computer is too slow.

This assignment was created by Bradley Zankel and Kevin Wayne.