Program visualizations have the potential to convey information about the behavior of the programs they depict. However, program visualizations in use vary widely in their ability to convey the desired information. In this work we conduct a study of the quality of the visualizations, the ability of those visualizations to present information in a way that is both effective and efficient in creating understanding. In order to measure program visualization quality, many traits of the visualization need to be considered: size, shape, color, and location of graphical elements, cueing and motion styles, etc. A testing environment called TestTaker has been developed that allows the experimenter to display visualizations to the user and receive feedback. The feedback is then used to evaluate the quality of the program visualizations based on the metrics specified. These quality metrics will be used to create a standard way of measuring the quality of program visualizations.