COS 432: Information Security |
Design reviews are an important part of security practice in the real world. Bringing in a set of "fresh eyes" to look over a project, and requiring the designers to go through the exercise of justifying their design, can provide invaluable improvement in a design.
Done properly, design review is not an adversarial process. The designers must approach the review as an opportunity to learn; the reviewers must approach the review with an attitude of respect for the designers and for what they have done right. The key is to take a "bugs are good" attitude --- you know the bugs are there, and you're happy to find them so you can exterminate them.
We will make a group's handed-in code and report available to their reviewers. Reviewers should read these carefully, and the reviewers should discuss them briefly among themselves before the review meeting.
The reviewers and designers must pick a mutually agreeable time and place for their face-to-face meeting. When you schedule your meeting, please email the time and location to the instructor. The instructor will attend some of the meetings to observe the design review process at work. The meeting should last at least a half hour, and no more than an hour.
After the meeting, the reviewers must write a report summarizing their conclusions regarding the design. The report should be frank about the design's good and bad points, while being written in a tone respectful of the designers' efforts. Where possible, the review should suggest specific ways to improve the design.
As always, the group of reviewers should write and submit a single, joint report, with all of their names clearly listed.
In order to encourage frankness in the design reviews, we will maintain a "Chinese wall" policy to ensure that the results of design reviews do not influence the grades we give for the original designs. In other words, if assignment N is a design and assignment N+1 is a review of that design, we will make sure that your grade on assignment N is completely determined before we look at what your reviewers wrote in their report for assignment N+1. Because of this rule, you can provide constructive criticism in your design review without worrying that your criticism is undermining anybody's grade.
You have limited time in the face-to-face meeting, so try to use it wisely. Review the documents in advance so you don't spend meeting time learning things that you could have gotten from the documents.
The review team might want to meet briefly before you meet with the designers, to help each other understand what is in the documents, and to discuss what questions you want to ask in the main meeting. This kind of pre-meeting works well if it is held immediately before the main meeting; then everything is fresh in your mind when you need it.
Try to focus your attention on the hard problems and tough design choices that the designers had to make. Doing this will focus your attention on the places where mistakes are most likely, and will give you the best opportunity to find mistakes or to notice something clever that the designers did.