An Empirical User Study of a Smartphone-Based Access-Control

An Empirical User Study of a Smartphone-Based Access-Control System Kami Vaniea Joint work with Lujo Bauer, Lorrie Cranor, Mike Reiter and Rob Reeder CMU Usable Privacy and Security Laboratory http: //cups. cmu. edu/

Physical access control • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 2

Limitations Must delegate all access tokens in advance Necessary to hide an access token for emergency situations Problems Once getting access tokens back given out key can be copied Requires users to carry additional objects • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 3

Smartphones What about using smartphones for access control? Smartphone capabilities • User interface • Computing ability • Communication Smartphones are increasing in popularity • Computational power of mobile phones also increasing • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 4

Research questions What are the usability challenges in building a smartphone-based access-control system? How well does a deployed smartphonebased access-control system match users’ needs? • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 5

Outline Introduction Grey Overview & Deployment Study 1: System Acceptance Study 2: Policy Creation Related Work Conclusion • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 6

Grey Smartphone based accesscontrol system Used to open doors in the CIC building Allows users to grant access to their doors from anywhere at any time • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 7

Grey example Lorrie Kami • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ Lorrie’s Office 8

Grey advantages Can easily delegate authority In advance of the access At the time of the access Guarantee access is no longer allowed after specified time • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 9

Field trial: environment 30 doors Perimeter doors to a large research area Offices Storage closets Conference room A lab A machine room • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 10

Users Chose participants who work together Wanted groups of users who share resources 29 • • users 9 faculty 11 graduate students 7 technical staff 2 administrative assistants • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 11

Interview procedure Interviewed participants • Security practices • Types of resources managed and needed Gave participants a smartphone with Grey preinstalled and brief instruction on use Interviewed one month later • Changes in security practices • General reactions to Grey Periodically conducted follow-up interviews at approximately one month intervals • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 12

Data Recorded interviews System approximately 30 hours of was actively used • Logged 19, 500 Grey accesses for 29 users • Active users averaged 12 accesses a week • Five users accessed their office almost exclusively with Grey • Users interacted with an average of 7. 4 different doors during the study Study lasted a year • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 13

Outline Introduction Grey Overview & Deployment Study 1: System Acceptance Study 2: Policy Creation Related Work Conclusion • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 14

Research question What are the usability challenges in building a smartphone-based accesscontrol system? • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 15

Design issues Analyzed interview data and identified five different design issues Speed Failures Complex features Non-Grey users New uses • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 16

Issue 1: Perceived speed Users quickly began to complain about speed and convenience of unlocking doors We knew Grey and keys required similar amounts of time to open a door Videotaped a highly trafficked door to better understand how doors are opened differently with Grey and keys • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 17

Issue 1: Videotaping Videotaped participants accessing kitchenette door Videotaped two hours daily after 6 pm for two weeks 18 users taped • 5 Grey participants • 13 additional participants were solicited as they passed through the door • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 18

Issue 1: Average access times Keys 3. 6 sec σ = 3. 1 Getting keys 5. 4 sec σ = 3. 1 5. 7 sec σ = 3. 6 Door Closed Door opened Stop in front of door Total 14. 7 sec σ = 5. 6 Grey 8. 4 sec σ = 2. 8 Getting phone 2. 9 sec σ = 1. 5 Stop in front of door Total 15. 1 sec 3. 8 sec σ = 1. 1 Door opened • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ Door Closed σ = 3. 9 19

Issue 2: Failure Cost of failure is potentially high Rebooting a phone or door was considered very inconvenient Several users stopped using Grey actively after a single inopportune failure • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 20

Issue 2: Delays interpreted as failures Delays can be interpreted as failures even when the system is functioning perfectly • Humans can be slow or unresponsive Providing feedback on the status of the request is very important • Did it arrive? • Is a human currently responding? • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 21

Issue 3: Confusing features Users would rather choose a suboptimal solution that they understand than one with an uncertain outcome Initially tried for concise interface (top) Adopted wizard solution (bottom) • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 22

Issue 4: Non-Grey users Grey is a service that becomes more valuable as more people use it Our participants were selected so that their work network included others with Grey Still had many people who would have benefited if Grey participant could have given access • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 23

Issue 4: Alice’s colleagues Bob No Grey Marie Frank Lillian Alice Sue Jake Mark Joe Have Grey • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 24

Issue 5: Unanticipated uses Unlocking door from inside the office without having to stand Unlocking nearby door for someone else without leaving office • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 25

Study 1: summary 1. Perceived speed and convenience are critical to user acceptance 2. A single failure can strongly discourage adoption 3. Users won’t use features they don’t understand 4. Important to consider occasional users of the system 5. Unanticipated uses can improve acceptance • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 26

Outline Introduction Grey Overview & Deployment Study 1: System Acceptance Study 2: Policy Creation Related Work Conclusion • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 27

Research question How well does a deployed smartphonebased access-control system match users’ needs? Do users make more or less secure access -control decisions when using Grey than when using physical keys? • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 28

Policies A policy is a collection of rules A rule is a tuple containing a user, resource and condition (Bob, Alice’s office, true) Bob True Alice’s Office • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 29

Methodology overview Examined access-control policies created by 8 resource owners • 8 offices • 1 machine room Using interviews we created ideal, key and Grey policies for each of 9 resources Compared ideal and implemented rules • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 30

Ideal policies Ideal Policy – Policy the user would enact if not restricted by technology Based on interview data Looked at not only what was enacted but endeavored to determine why • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 31

Policy synthesis. . . Garry Frank True Rick Larry Logged Joan Mary. . . Lab owner is notified Logged False Charlie’s Lab • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 32

Ideal conditions True (can access anytime) Logged Owner notified Owner gives real-time approval and witness present Trusted person gives real time approval and is present False (no access) • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 33

Policy analysis We compared each of the 244 ideal access rules, with the key and Grey rules and marked them as: • False Accept – User not required to fulfill all conditions required by the ideal policy • False Reject – User must fulfill conditions not required by the ideal policy • Faithfully Implemented – Matched the ideal policy • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 34

Policy analysis example Charlie’s Lab Faithfully False Accept False Reject implemented Alice Ideal Conditions Key Conditions Access anytime Has a key Bob Owner Notified Has a key Sue Logged Doesn’t have a key 35 35

Keys vs. ideal User 28 User 29 Alice Bob Sue User 4 User 27 User 26 User 5 20 Faithful Implementations (Green) User 25 User 6 4 False Accepts (Red) 5 False Rejects (Yellow) User 7 User 24 Charlie’s Lab User 23 User 22 User 8 User 9 User 21 User 10 User 20 User 11 User 19 User 18 User 17 User 16 User 15 User 14 User 13 User 12 36

Conditions Ideal Keys True (can access anytime) True (has a key) Logged Owner notified Ask trusted person with key access Owner gives real-time approval Know location of hidden key Owner gives real-time approval and witness present Ask owner who contacts witness False (no access) ? Trusted person gives real time approval and is present False (no access) • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 37

Rules Key implementation accuracy Ideal Conditions • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 38

Conditions Ideal Grey True (can access anytime) True (has a delegation) Logged Owner notified Ask trusted person with Grey access Owner gives real-time approval Ask owner via Grey Ask owner who contacts witness False (no access) Owner gives real-time approval and witness present Trusted person gives real time approval and is present False (no access) • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 39

Rules Implementation accuracy Ideal Conditions • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 40

Study 2: Contributions Documented the collection of ideal policy data Developed a metric and methodology for quantitatively comparing accuracy of implemented policies Showed that a smarphone access-control system outperformed keys in overall security and effectiveness • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 41

Outline Introduction Grey Overview & Deployment Study 1: System Acceptance Study 2: Policy Creation Related Work Conclusion • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 42

Related work Several Grey-like systems have been proposed but not implemented • Digital Key system [Beaufour and Bonnet] • The Master Key [Zhu, Mutka and Ni] Access-control tokens are not very easy to use and those that are tend to be less secure [Braz and Robert; Piazzalunga et. al. ] • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 43

Related work Usability of access control for file systems • Manipulating access-control lists is difficult for users to do accurately [Cao and Iverson] • Users have difficulty understanding how rules interact to form the effective policy [Maxion and Reeder] Studies of users’ access-control needs • Identified several different approaches to access control management [Ferraiolo et al. ] • Users have dynamic access-control needs that very by task [Whalen et al. ] • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 44

Summary Study 1 • Users have low tolerance for failure and treat Grey like an appliance Study 2 • Policies made using Grey were less permissive than key policies and better matched the ideal policies Related work • Unlike previous work we study an actual working system and examine gathered empirical data • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 45

Future work Explore the tasks policy authors engage in Explore the use of a Grey like system in large organizations Develop technologies that assist in the authoring of policies • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 46

CMU Usable Privacy and Security Laboratory http: //cups. cmu. edu/ 47

Bibliography X. Cao and L. Iverson. Intentional access management: Making access control usable for end-users. In Symposium On Usable Privacy and Security, 2006. A. Beaufour and P. Bonnet. Personal servers as digital keys. In 2 nd IEEE International Conference of Pervasive Computing and Communications, 2004. C. Braz and J. Robert. Security and usability: The case of the user authentication methods. In IHM ’ 06, p 199 -203, 2006. D. F. Ferraiolo, D. M. Gilbert and N. Lynch. An examination of federal and commercial access control policy needs. In 16 th National computer Security Conference, p 107 -116, 1993. • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 48

Bibliography R. A. Maxion and R. W. Reeder. Improving user-interface dependability through mitigation of human error. International Journal of Human-Computer Studies, 63(1 -2), 2005. U. Piazzalunga, P. Salveneschi, and P. Confetti. The usability of security devices. In L. F. Cranor and S. Garfinkel, editors, Security and Usability: Designing Secure Systems that People Can Use, p 221 -241. O’Reilly, 2005. T. Whalen, D. Smetters, and E. F. Churchill. User experiences with sharing and access control. In CHI ’ 06 extended abstracts on Human factors in computing systems, p 1517 -1522, 2006. F. Zhu, M. W. Mutka, and L. M. Ni. The master key: A private authentication approach for pervasive computing environments. In 4 th IEEE Interantional Conference on Pervasive Computering and Communications, p 212 -221, 2006. • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 49

Number of Accesses Grey accesses per week Week • CMU Usable Privacy and Security Laboratory • http: //cups. cmu. edu/ 50