Prepared by Jeanette Mulligan and Chris Andrichak

Exploring Multimedia and Computer Networking

Introduction to Computing
Multimedia Sound Processing
Image Processing.
Digital Video
Video Conferencing

To introduce the power multimedia computing and networking.

4 PowerMac Computers (equipped with miscellaneous multimedia software including Quickcam and Adobe Photoshop, SoundEdit, QuickMovie and CU See Me ), scanner and color printer.

Girls are given simplified version of the material below and are invited to explore multimedia following the handout with the help of electrical and computer engineers.

Introduction to Computing

Computers think much differently than people do. While people use their brains to think, computers use a microprocessor.

In the English language, we use 36 different characters to represent things, that is, the letters A-Z and the digits 0-9. Computers, on the other hand, are very limited -- a computer uses only two characters to represent things, 0 and 1.

Despite the tremendous difference between how we process information, and how a computer does, we are still able to communicate with computers. However, before a computer can really understand any information that we give it, it first must digitize the information. That is, a computer must first translate the information into a format it can understand by putting everything in terms of zero's and one's.

As an example of how a computer digitizes information, let's try putting an English word, such as your first name, into a format a computer would understand.

We first must assign each letter of your name a code which contains only zero's and one's. A standard code for doing this already exists. It is called the American Standard Code for Information Interchange (ASCII), andis shown in the table below.

Note that the code of zero's and one's, or bits, which represents each of the characters shown above is unique. No two characters have the same code. Also, in the table above, each of our characters is represented by eight bits. Computers think in terms of eight bits at a time. A group of eight bits is called a byte.

Try writing out your first name in terms of zero's and ones using the codes above. When you rewrite your name in zero's and one's, you are actually writing it in binary code, meaning you are representing your name in terms of only two characters.

Your First Name (in English): _______________________________________

Your First Name (in Binary): ________________________________________

Our method of processing information is much more efficient than a computer's. The computer requires a much longer string of characters to represent your name than we do.

Computers are capable of processing more than just text. They can also process sound, pictures, and even video. In fact, the combination of these three things -- sound, pictures, and video -- in computer applications is called multimedia.


Multimedia is the combination of sound, images, video, and text in computer applications. With multimedia, you can do many things, such as make a movie, combine and edit images, and create interactive presentations.

Introduction to Sound Processing

With a computer, you can record a sound and then save it to a disk, just as if it were a document. You can then edit the sound to distort it, accentuate it, or even mix it with other sounds.

Sounds such as a human voice resemble a waveform like the one shown below.

A Sound Waveform

Before this sound can be stored in a computer, there are two problems which must be solved:

* PROBLEM #1: The sound contains too much information. Computers have a limited amount of storage space in them. When developing programs for computers, engineers and computer scientists often try to minimize the amount of information that must actually be stored in the computer.

To minimize the amount of information that must be stored to represent this sound, the computer first samples, or picks points of the sound waveform to store. The sampled waveform is shown below.

The Sampled Sound Waveform

You can see that much of the information in the waveform above has been cut out, but the wave, for the most part has not lostits shape. If the samples were taken at smaller intervals, the waveform above would look even more like the original waveform. On the other hand, if the waveform were sampled less frequently, the points in the figure above would be spaced further apart, and the waveform would no longer look very much like the original waveform.

* PROBLEM #2: Computers can only think in terms of zero's and one's. Before sound can be stored in a computer, it must be changed into a format that a computer can understand. Therefore, each point in the sampled voice waveform must be assigned a code representing its value or height. The catch is that the code name given to each point can only consist of zero's and one's. If two of the points have the same or nearly the same height, they can be assigned the same code.
Once the sound is sampled and coded, it can be stored as a file in the computer. The sound can then be edited with a computer software program. When a sound is edited, the editing software actually performs many mathematical operations to the zero's and one's which represent the sound. The result is that the edited sound plays differently than the original sound did.

Experimenting with Sound Processing

To experiment with how a computer processes sounds, let's try recording your voice into the computer. However, we will try doing this at two different sampling rates. The first time you record, you will tell the computer to sample your voice at a frequency of 11 kHz (kiloHertz), meaning you want the computer sample your sound waveform 11,000 times every second. During the second trial, you will tell the computer to use a sampling rate of 22.5 kHz -- that is, the computer will sample your voice waveform 22,500 times every second!

1. Open the program Sound Edit 16.
2. Go to the Sound menu, and select Sound Format.
3. Change the rate to 11 kHz. Then click OK.
4. Click Record, and then say something for two or three seconds.
3. Click Stop.
4. Now click Play and listen to what you just recorded.

Now let's record a new sound, but with a faster sampling rate.

1. Go to the File menu, and select New.
2. Go to the Sound menu, and select Sound Format.
3. Change the rate to 22.5 kHz. Then click OK.
4. Click Record, and then say something short (such as two or seconds long).
3. Click Stop.
4. Now click Play and listen to what you just recorded.

Try going back and playing the original sound which was sampled at 11kHz. Which sound is better? Why?

Applications of Sound Processing

Electrical and computer engineers work together to use sound processing to solve real world problems. In many cases, they even work with people in other disciplines, such as the medical field. Some examples of how sound processing is being used include:

* Compact Disks -- Music is stored on compact disks much the same way that it can be stored in a computer. It is first digitized and then stored on the disk. In fact, audio that is stored on a CD is typically sampled at a rate of 44kHz.

* Voice Recognition -- Engineers and computer scientists have developed software which enables computers to understand and react to something being said to them. For example, a user can say "Open File," and the computer obeys the command. Voice recognition technologies may be particularly useful for the handicapped.

* Speech Therapy -- Engineers and computer scientists have developed combination hardware and software packages which enable a computer to store a person's voice, and then graphically display various parameters of it. Patients can compare the pictures of their voice with those of a model voice. This application is especially helpful for the hearing impaired -- it enables them to learn how to speak by using their sight!

Introduction to Image Processing

Pictures enter a computer much the same way that sound does. They are sampled, coded, and then stored as a file. For each dot, or pixel, that occurs in a picture, the computer must store the pixel's color, its brightness, and its position in terms of x and y coordinates. In some cases, the pixel's color alone can require 24 bits of information. It is no surprise, therefore, that computer graphics files can be very large. One of the most popularized uses of image processing is for graphics art design. Engineers and computer scientists have developed many software packages which perform intense mathematical computations on a digitized picture to display a particular effect.

Experimenting with Image Processing

To experiment with image processing, let's scan a picture, and then edit it with a graphics software package:

1. Go to the apple menu in the upper left-hand corner of the screen. Hold down the mouse button and go down to Adobe Photoshop.
2. Under the File menu, select Acquire, then Plug-In Scan.
3. Place a picture on the scanner. The upper left-hand corner of the picture should be placed in the lower left-hand corner of the scanner.
4. Click the Preview button.
5. When the preview is finished scanning, use the selection tool to select the specific area of the picture that you want to actually scan.
6. Click on the Autodensity button. (It is a little circle with a black and white squiggle in it.)
7. Click on the Scan button to scan the image.
8. Now that you have digitized the image, experiment with some of the filters in Adobe Photoshop.

Keep in mind that each time you edit the image, the computer is performing tons of calculations on the bits that represent your image.

Applications of Image Processing

Besides graphics design, image processing is used in many other applications, including:

* Medical Imaging -- Engineers and medical professionals are combining their knowledge of electricity, magnetism, computers, and medicine to devise better medical imaging systems to detect diseases such as cancer.

* Image Reconstruction -- Electrical and computer engineers are developing programs which can reconstruct blurry or distorted pictures. This may be particularly useful in law enforcement to reconstruct blurry surveillance videos or pictures.

* Image Comparison -- Computer techniques can be used to compare images such as pictures, fingerprints, and even human signatures.

Introduction to Digital Video

Computers are also capable of capturing and storing videos. A computer stores a video as a series of still pictures. It then plays each picture back at a very fast rate to display a movie. Each image of a movie is called a frame.

A short video requires a computer to store many pictures. Video files, therefore, are exceptionally large. Typically, multimedia files, such as video, pictures, sound, or combinations of the three, are compressed, or scrunched by the computer to minimize the amount of storage space that they require. That is, once the multimedia information has been coded into zero's and one's, the computer performs computations to minimize the number of zero's and one's needed to accurately store all of the information. For example, the computer might look for patterns or repetitions in the sequence of zero's and one's, and eliminate any redundancies. Multimedia files can then be stored in a more reasonable amount of space.

Experimenting with Digital Video

To get a feel for how much storage space is required to capture video in a computer, let's create a short video clip.

1. Go to the apple menu in the upper left-hand corner of the screen. Hold down the mouse button and go down to QuickMovie.
2. Click Desktop. Now give the movie a name ... how about your name?
3. Click Record. Now film yourselves for a few seconds, then click Stop.
4. In the lower left-hand corner of the movie window, click once on the picture of the video camera and film.
5. Now click go to the File menu, and select Save As. Name the movie your name.out.
6. When you are finished, go to the File menu, and select Quit.

Now record the length of your video:

Video Length (in seconds): _____________________

Now let's find out how many bits the computer used to store the video.

1. Click on File, then Quit.
2. Click once on the file, Our Film.
3. Go to the File menu, and select Get Info.
4. Record the number of bytes used after the Size category.

Number of bytes used to store the video: _________________________

Remember that one byte actually consists of eight bits, so let's multiply this number by eight to see how many bits, or zero's and one's, the computer actually used to store the video.

Number of bits used to store the video: __________________________

Divide the number of bits used to store the video by the video length in seconds. This will tell us how many bits are needed to store one second of video on a computer.

Number of bits needed to store one second of video: ________________________

Even after compression, video requires a tremendous amount of storage space within a computer. Data compression is a common example of the many challenges that engineers face in the computer industry.

Applications of Digital Video:

* Video Conferencing -- Current engineering technologies allow computers to be used as a tool for video conferencing. However, video conferencing is currently slow and expensive. Electrical and computer engineers are working together to make video conferencing more feasible and affordable.

* Education -- Multimedia is currently being used to develop interactive learning systems. Many companies rely on multimedia to train employees. Universities are using multimedia as a learning tool both in and out of the classroom. In fact, some universities are combining multimedia and video conferencing technologies to implement distance education programs.

* Simulations -- Multimedia is being used for simulation-based training in many fields, including medicine, aircraft flying, and even fire fighting. Medical students can explore the inside of a human body and perform virtual surgeries without leaving a desk. Likewise, potential pilots can practice flight control without the expense of being in a plane Firefighters can improve their skills without the risk of personal injury.

Video Conferencing

One common application of multimedia that involves joint efforts of both electrical and computer engineers is video conferencing. Video conferencing is similar to having a phone conversation; only, in addition to hearing the other person's voice, you can also see them!

Video conferencing can be done by transmitting computer information over phone lines. Keep in mind that a computer requires enormous amounts of bits to store video. Video conferencing, therefore, is somewhat slow. Not only must the computer store video information, but it also must send it to another computer. The other computer then has to translate all of the information so that the human user at its end can understand it.

Experimenting with Video Conferencing Try having a video conference with people at another computer by following the instructions below:

1. Go to the apple menu in the upper left-hand corner of the screen. Hold down the mouse button and go down to CU See Me.

2. Click on the Conference menu, and go down to Connect to, and then select Other Computer.

3. Change the Conference ID to 1. Then click Connect.

4. When you want to speak to the other person, click and hold down the audio icon in the audio window. When you are finished speaking, release the mouse button.

5. To end video conferencing, select Conference, Disconnect. Then go to the file menu, and select Quit.

Did you notice any problems while video conferencing? Why do you think these problems exist, and what can electrical and computer engineers do to resolve them?

Return to Progress Report

Return to WiE Home Page