IEEE(c) 1996 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.

By choosing to view this document, you agree to all provisions of the copyright laws protecting it.

Return to Table of Contents

Using Multi-media to Teach the Theory of Digital Multi-media Signals *

Jeffrey B. Schodorf Mark A. Yoder
Electrical and Computer Engineering Electrical and Computer Engineering
Georgia Institute of Technology
Rose-Human Institute of Technology

James H. McClellan Ronald W. Schafer
Electrical and Computer Engineering Electrical and Computer Engineering
Georgia Institute of Technology
Georgia Institute of Technology


DSP, Education, World Wide Web, MATLAB


A new Digital Signal Processing (DSP) course designed to build students' intuition about signals and systems is presented. The course makes extensive use of multi-media demonstrations to relate real-world signals and discrete-time systems to their mathematical descriptions. In addition to the numerous in-class demonstrations, weekly laboratory exercises have been developed to further strengthen the ties between DSP theory and real-world signals and systems. Many of the in-class demonstrations have been captured on the World Wide Web (WWW), so the students can explore them outside of class. In the near future, the WWW-based demonstrations will be packaged on a CD-ROM as an integral part of a new text.

1 Introduction

Although students have been exposed to multi-media for most of their lives, few technical courses take advantage of this exposure to enhance the presentation of signals and system theory. Computers now have multi-media capabilities that are ideal for developing the materials necessary to integrate this aspect of education into DSP courses. In addition, DSP is situated in a special position where the underlying theory is essential for understanding the actual multi-media systems contained in present-day computers.

In 1974 an introductory DSP course that attempted to make use of computer exercises and demonstrations was offered at Princeton [1]. That course was limited by the computer tools of the day---FORTRAN on a batch mainframe. Now, with sophisticated mathematical programs such as MATLAB [5] and Mathematica [6], students can easily simulate signal processing techniques that underlie applications such as audio, voice, image and video processing. Development of a complete processing system may take only a few hours once the system is described in terms of its frequency response and system block diagram. This high-level approach makes it possible to use DSP as the introductory course in an electrical or computer engineering curriculum.

With this philosophy in mind, new materials for demonstrating real-world signals and systems have been developed for computer engineering students at Georgia Tech and Rose-Hulman. Both schools have a relatively new, required course for computer engineering sophomores that teaches the fundamentals of DSP. Because DSP is involved in every aspect of multi-media information (coding, transmission, storage, playback, etc.) the numerous in-class demonstrations and in-depth laboratory exercises developed for the course relate to the synthesis and/or analysis of sounds or images via computer (i.e., multi-media). In other words, the course is actually using multi-media to teach the theory of digital multi-media signals. The rationale for studying applications such as audio and video is to provide a richer, more interesting context and motivation for later courses in engineering theory. The same approach has been taken by Steiglitz in a new book targeted at digital audio and computer music [2].

In the spirit of presenting via the Web, this paper is also a Web document on the accompanying CD-ROM. Words underlined in the printed version are links in the electronic version. Feel free to click on these links to explore this paper in the same way a student would explore our class.

Section 2 presents the goals and structure of the new sophomore DSP course and shows what topics are presented in the class. Section 3 provides an overview of the laboratory exercises, which are an integral part of the course. Section 4 describes some of the in-class demonstrations and how they are made available outside of class via the Web or via CD-ROM. Section 5 conveys our belief that a DSP course such as the one presented here is well suited to be both an introductory EE course (before circuit analysis) and, with perhaps a few minor modifications, a general introductory course for non EE majors. Finally, section 6 contains some concluding remarks.

2 Course Overview

  The objective of this course is to show students the role that mathematical system theory can play in the development of computer applications and products such as multi-media systems. The style of the course is ``hands-on,'' with computers being used as an integral part of the classroom and laboratory environment. Traditional classroom lectures present the underlying theory of signals and systems, but our goal has been to supplement every lecture with a computer demonstration that relates the theory to real-world signals and applications. In addition, weekly laboratory assignments are given to explore the signals in greater depth.

The course outline given below is close to the final definition of the course, but continues to evolve as different instructors teach the material. Since Georgia Tech and Rose-Hulman are both on a quarter schedule, the topic list would need to be expanded for a semester-length course.

  1. Definition of a signal and a system
  2. Intro to MATLAB (assumes programming experience)
  3. Review vector/matrix notation
  4. Complex numbers: represent sinusoids with phasors
  5. Sinusoidal signals: amplitude, phase and frequency
  6. Synthesizing sounds with general classes of sinusoids
  7. Frequency content: harmonics, AM and FM signals
  8. Sampling, aliasing and reconstruction
  9. Linear filtering: the concept of smoothing data
  10. Block level description of systems
  11. FIR filtering: a generalization of running average filters
  12. Recursive filtering: difference equations with feedback
  13. Frequency response of FIR and IIR filters
  14. Simulation of dynamic time response; impulse response
  15. Z-transform analysis: rational transfer functions; omits the inverse transform
For a fifteen-week semester course, the following topics might be appropriate:
  1. Synthesizing sounds with narrowband recursive filters
  2. Fourier Spectrum analysis: spectrograms and windowing
  3. Inverse Z-transform

3 The Computer Laboratory

  While the format of the course still includes traditional lectures, the key part of the course development is the laboratory experience. The laboratory determines the style of the course, because the students must be actively involved in learning how to apply theory through the laboratory exercises. Many useful ideas for labs are appearing in new books dedicated entirely to DSP computer-based labs [3, 4]. For example, sinusoidal synthesis of music brings up the issue of choosing the sampling rate correctly for D/A conversion---a trivial matter to some, but quite confusing at first. In addition, lab projects that involve simple speech recording, analysis and synthesis could be used to have the students experiment with processing of their own voices. In the area of image processing, different types of enhancement filters can be used to modify pictures for bizarre special effects. The mathematical software packages that are now available for signal analysis allow for quick programming of all these different processing methods, so the lab exercises encourage students to explore a variety of signal enhancement systems.

Recently, we have used the graphical user interface (GUI) tools of MATLAB to build simple environments where the students can experiment with just one concept. We have found that sometimes the details of MATLAB programming can distract from the DSP ideas. For example, in image enhancement, our emphasis is on filtering, but the display of processed images is complicated by the fact that the output dynamic range must be eight bits in the range [0,255] while filtering usually produces negative values. This extraneous issue is bad from the point of view that it masks the direct learning of filters for smoothing or edge enhancement; on the other hand, it is good because it teaches the valuable lesson that real signals often have attributes or constraints that must be obeyed independent of the DSP programs.

Another valuable part of the laboratory is the opportunity for some design and creativity. In one lab the students must synthesize a tune by using simple sinusoids to prove their understanding of amplitude, frequency, and phase of a cosine signal. This seemingly trivial task leads to very creative solutions in which the sinusoids can be tailored to optimize the ``musical'' sound. For example, several sines can be added to produce a chord or harmonic overtones; pauses can be inserted to produce better timing; and amplitude envelopes can be used to produce the attack, decay, sustain, release (ADSR) shape used in computer music. Each of these activities requires additional work by the students, but they are often willing to do the extra research and experimentation to produce the best sounds.

3.1 Laboratory Topics

In one quarter, nine laboratories can be completed. The following list gives describes the type of labs we are using to promote the connection to real signals.

3.2 Laboratory Facilities

In designing laboratory exercises for the class, one goal has been to ensure that they are readily transportable between workstations and personal computers. The fundamental barrier to this goal is that many students who own personal computers are reluctant to purchase the full version of MATLAB . Student versions of MATLAB are much cheaper; however, they limit the maximum size of a matrix to 8192 elements. This limitation prohibits the use of student version MATLAB in the image filtering labs and restricts the speech and audio labs to sounds on the order of 1 to 2 seconds. However, all other labs are easily performed on a Macintosh or PC clone using the student version of MATLAB. In fact, personal computers may be the machines of choice since the audio capabilities of many workstations are poor unless expensive add-ons are purchased. On the other hand, Macintoshes and PC clones generally have many options for audio input/output and are starting to have low-cost options for video.

Once multi-media software and hardware are widely available, the audio and video components of a multi-media computer system can be demonstrated and studied. The theory presented in the course ultimately explains the inner workings of A/D conversion boards and the corresponding software packages for recording, processing and playback. These components are now rather inexpensive. Future plans include more extensive use of video, using frame grabbers or video rate A/D converters that digitize from a VCR output. Even if the resolution were rather limited, it would be more important in the lab to have access to the video signals rather than to incur the expense of extremely high-quality I/O.

4 In-Class Demonstrations, the Web and CD-ROM

  The in-class demonstrations are an effective means of presenting an overview of a particular topic. For example, in the early stages of the course, students are introduced to the idea that sinusoids are real signals that occur naturally. To reinforce this idea, the equations that underlie the behavior of a tuning fork are derived. The solution to the differential equation that describes the physics of a tuning fork is, of course, a sinusoid. Next, a demonstration with a tuning fork is presented. The instructor strikes the tuning fork and captures the audio signal on an oscilloscope where the amplitude, period and frequency can be measured. The demonstration is repeated with tuning forks of different size and mass, thus illustrating the validity of the differential equations and reinforcing the idea that sinusoids are naturally occurring signals.

When these demonstrations are presented in class they allow some student interaction. However, the student can only influence the demonstration through the professor. The demonstrations also tend to hide many implementation details. Glossing over the details is good when the student is first exposed to a new topic because the goal of the demonstration is to motivate a connection between a fundamental concept and its physical manifestation. However, the details involved are also an area rich for educational exploitation. For example, the link between a digital signal stored on a computer and the sound perceived when it is played through a speaker is largely a function of the sampling rate of the D/A converter. Allowing students to experiment with this parameter and even make mistakes helps them to build a more intuitive understanding of the concept of sampling.

Putting the demonstrations on the WWW is one way of facilitating increased student interaction [7]. A number of classroom demonstrations have been duplicated or modified slightly in order to facilitate their presentation on the WWW so that students can have access to them outside the classroom.

Another area for computer usage lies in automatic generation of quiz problems for review and drill. The MATLAB GUI can be programmed to generate problems that involve a graphical component such as reading the parameters of sinusoids or doing complex arithmetic calculations by visualizing the graphical (or geometric) vector solution. Below is a brief description of a few of the demonstrations and drill problems used.

Table 1. Examples of demonstrations
Tuning Forks
In class, the equation of the frequency of a tuning fork is derived. This demonstration shows how the mass and type of material affect the pitch of the tuning fork. Some QuickTime® movies of various tuning forks being played allow the student to see the different forks and hear the different pitches. A plot of the sound of one fork is also shown.
Reading Sinusoids
This is a MATLAB program that drills the student on how to read a sinusoid. It uses MATLAB's graphical user interface so the student who hasn't learned MATLAB (yet) won't get lost in the details of plotting, etc..
Rotating Phasors
Is there a better way to learn about a rotating phasor than to watch one rotate? These MATLAB generated QuickTime® movies reinforce the idea of a rotating phasor and illustrate Euler's formula by tracing out the projection of the phasor onto the real axis.
Sampling and Reconstruction
In the sampling and interpolation chapter, the concepts of aliasing, folding and reconstruction through interpolation are all introduced. An alternate view of these concepts is presented in an interesting demo involving a rotating disk and a video camera.
PEZ A MATLAB Graphical User Interface (GUI) (written by course alumnus Craig Ulmer) that allows pole-zero placement, movement and editing. A second window showing the corresponding frequency response and impulse response is updated whenever the poles or zeros are moved.
Mapping from the Z-plane to Frequency Response
A QuickTime® movie that provides the student with a three-dimensional view of an FIR system function. The frequency response is evaluated by tracing out a path along the unit circle.

The above demonstrations and drill problems have been developed by the various instructors and even some students associated with the course. Currently, they reside along with other support material on the ``EE2200 homepage'' on the Georgia Tech Web server. As students continue to provide feedback on the material and as new demonstrations and drill programs are developed, the look and feel of the homepage continues to evolve. Eventually, the material will be packaged in HTML format onto a CD-ROM equipped with a local Web browser. The CD-ROM will accompany a textbook that is currently being developed. As a result, students can navigate through the demos, labs and homework problems with or without a network connection, and the interface will be identical if the same browser is used. Students taking courses similar to this one at other universities also will be able to take advantage of the same support material that has been assembled for the course at Georgia Tech. Finally, one can envision a cooperative effort among several universities on the Web to develop more course demos and labs that add to this initial effort.

It should also be noted that the Internet has been used in other ways to increase student participation in the course. For example, a Georgia Tech ``newsgroup'' devoted entirely to the course has been active for over a year. The newsgroup provides a convenient means for students to interact with one another and exchange ideas, form study groups, etc. In addition, relevant announcements and handouts are typically posted on the course homepage. Future plans include the electronic submission of assignments and allowing students to modify the course homepage by adding links to their own homepages, etc.

4.1 Homework

  The Web and CD-ROM also provide an efficient means of storing the wealth of solved problems that have accumulated over the nine quarters that this course has been taught at Georgia Tech. Rather than overload the printed textbook with too many problems, solved problems from previous quarters have been scanned and stored as digital images which are accessible to the class or interested reader. The format of these files is a gray-scale GIF compressed format at 150 dpi. This format was chosen as a compromise between screen viewing which requires gray scale, and printing quality which needs higher dpi resolution.

4.2 The Textbook

  As mentioned previously, the WWW demonstrations, the lab resources, MATLAB drill programs and solved problems will be packaged together on a CD-ROM that will accompany a textbook for the course soon to be published by Prentice Hall, Inc. The outline of the book follows the course outline presented in section 2, including a chapter on the discrete-time Fourier transform and its application to spectrum analysis. The book is self-contained, but will be relatively short since a bulk of the material is on the companion CD-ROM. References in the text highlight connections to demonstrations and laboratory exercises on the CD-ROM. In addition, an outline of the text appears on the CD-ROM with hypertext links to appropriate supporting materials.

5 DSP Before Circuits

  This introductory course represents a major departure from the traditional EE curriculum where circuit theory has long been the first course. The reasons for starting with circuit theory are: (1) it is a good subject for introducing the use of mathematics in representing and analyzing physical systems and (2) the knowledge gained is useful for most of the subsequent classical EE subjects. Since these points remain true, why change?

One reason is that basic circuit theory focuses on a very low level of engineering systems, and therefore it is difficult to motivate students who ask, ``Why am I studying this subject?'' On the other hand, introductory discrete-time signals and systems is easy to motivate if it goes beyond the mathematics of DSP; e.g., by processing audio and video signals. Furthermore, in this era of digital systems, a basic DSP course is more naturally tailored to the widespread use of computers for instrumentation and processing.

In the coming years, there is likely to be a general recognition that signal processing methods are more important than circuit theory for a wide audience of engineering students outside of EE. For example, mechanical engineers find significant use for signal processing in measurement systems, and computer scientists involved in multi-media applications can benefit greatly from a deeper understanding of discrete-time signals and systems. Thus, an ``introductory DSP'' course would be attractive to other majors because the material would not be tied to a long prerequisite chain as is now the case. Instead, second or third year students could have a glimpse of computer processing methods that might include applications from their own area.

6 Conclusions

  The motivation behind the new sophomore-level course on discrete-time signals and systems was the desire to create an introductory electrical/computer engineering course that would couple modern computer technology to traditional EE topics such as phasors and Fourier analysis and to applications of these concepts. Students in this course can see how the mathematics, physics and computer science learned in freshman courses underlies the multi-media technology of present-day computers. The course makes use of traditional teaching tools such as lectures, problem sets, and written notes. Somewhat less traditional is the extensive use of laboratory exercises and CD-ROM based demonstrations, videos and drill programs carried out using state-of-the-art multi-media computer software. All of this material is available on the World Wide Web, so that students can explore many topics at their own pace.

Perhaps most interesting is the fact that the course focuses on the very heart of the multi-media explosion that is currently underway. Therefore it is eminently reasonable that multi-media technology be applied in the teaching of the course itself. Toward this end, the authors have begun to explore how CD-ROMs and World Wide Web technology can be used effectively in teaching the theory behind multi-media signals and systems. In this paper, examples of some early activities in this emerging area of education have been presented. The success encountered so far bodes well for the notion that modern information technology will enhance education in engineering and many other fields.


The authors wish to thank David Anderson, Robbie Griffin, Joe Monaco and Craig Ulmer for their contributions to the development of demos used in this paper and the course.


K. Steiglitz, An Introduction to Discrete Systems, John Wiley and Sons, New York, 1974.
K. Steiglitz, A DSP Primer: with Applications to Digital Audio and Computer Music, Addison-Wesley, 1996.
C. S. Burrus, J. H. McClellan, A. V. Oppenheim, T. W. Parks, R. W. Schafer, and H. W. Schussler, Computer-Based Exercises for Signal Processing using MATLAB , Prentice-Hall, Englewood Cliffs, NJ, 1994.
V. Stonick and K. Bradley, Labs for Signals and Systems using MATLAB , PWS Publishing Co., 1996.
The Mathworks Inc., MATLAB User's Guide, Natick, MA, 1985--1995.
S. Wolfram, Mathematica: A System for Doing Mathematics by Computer, Addison-Wesley, 1988.
J. H. McClellan, R. W. Schafer, J. B. Schodorf, M. A. Yoder, ``Multi-Media and World Wide Web Resources for Teaching DSP'', IEEE Int. Conf. Acoustics, Speech and Sig. Proc., May 1996.

* Supported in part by the EduTech Institute at Georgia Tech.