This paper describes a usability study designed to assess ease of use, user satisfaction, and performance of a mobile robot mission specification system. The software under consideration, MissionLab , allows users to specify a robot mission as well as compile it, execute it, and control the robot in real-time. In this work, a new automated mission repair mechanism that aids users in correcting faulty missions was added to the system. This mechanism was compared to an older version in order to better inform the development process, and set a direction for future improvements in usability. In military and other critical applications, it is vital for users with little or no robotic background to quickly and accurately create and execute mobile robot missions, and the importance of empirically evaluating the interfaces that support this task cannot be underestimated [1, 2]. For example, researchers have conducted usability studies in critical robotic applications such as in the area of Search and Rescue [3, 4] and on teleoperation in nuclear environments [5-7], dealing with run-time interaction with the robots.