Authentication

Account breaches have significant negative impacts at the individual, organization, and nation-state levels. For example, an account breach caused the recent Colonial Pipeline incident, costing the company millions and leading to fuel shortages and panic across the eastern seaboard. It is well known that a key factor in breaches is weak and reused passwords1. Many authentication systems have been proposed to improve the security of passwords1. However, most of these systems never see real-world usage, and uptake is far from universal when they do. The long-term goal of our research in this area is to (i) identify the impediments to adoption and correct usage of authentication systems and (ii) to design and demonstrate the ability of systems designs to address these impediments, helping users to leverage robust authentication systems to protect their online accounts.

Understanding the Adoption Process

There is a critical knowledge gap regarding the process by which users adopt authentication systems, preventing system designs from addressing this process and thereby decreasing the likelihood of supplanting traditional password-based authentication. To address this gap, our overall object is to model the adoption process for authentication systems, enabling system designs that address users’ needs throughout this process. The rationale for this research is that designing authentication systems to support the adoption process will improve the security and usability of those systems and increase the likelihood that they can supplant passwords.

We began our investigation of this area by conducting a laboratory study investigating the setup process for YubiKey and a empirical study observing users in their first two weeks using a YubiKey2. These studies revealed significant problems with the setup process for hardware security tokens. However, these studies also demonstrated that users generally enjoyed day-to-day usage. We have also conducted observational interviews of password manager users, having them demonstrate and explain how they configure their password manager, create accounts (including credential selection/generation), log into accounts, and update accounts. Results from this study strongly suggest a multi-phase process by which users had adopted their manager, with phases including setup, acclimation, gradual replacement of existing passwords with generated passwords, and steady-state usage.

These studies demonstrated a need for longitudinal studies to flesh out the adoption process model for authentication systems. To this end, we plan to conduct large-scale (200+~participant), long-term (1-year), longitudinal studies, observing users as they adopt an authentication tool into their daily lives. Initially, we will conduct studies for hardware security tokens (e.g., a YubiKey) and password managers, using the collected quantitative and qualitative data to model the adoption process for each type of tool. By comparing these models, we will derive a standard model describing the adoption process of authentication systems or demonstrate that it is not possible to create a unified model. Finally, we will design, prototype, and evaluate adoption process-aware systems, measuring their ability to improve security, utility, and usability during adoption and day-to-day usage. This research plan is creative and original in that it breaks from the status quo of treating adoption as a binary process—users do or do not adopt the tool—instead modeling it as a process consisting of distinct phases, each with their respective behaviors and transitions.

Exploring Security and Usability Designs

Even when adopted, security-critical functionality of new authentication systems is underutilized3, leaving users vulnerable. To address this issue, our overall objective is to identify design principles that can improve the security, utility, and usability of authentication systems. Moreover, we will quantify and explain the ability of competing system designs to encourage correct usage of these systems. The rationale for this research is that it will increase the understanding of underlying issues, identify generalizable design principles, and promote the design of more usable and secure password managers.

We have already evaluated the security of over twenty password managers on desktop4 and mobile systems5, examining the full password manager lifecycle—password generation, storage, and autofill. Our results identified numerous security issues with these browsers. One of the most damaging and persistent is the ability of malicious apps, websites, and browser extensions to steal credentials as they are autofilled. To address this issue, we are currently exploring how to build a trusted pathway into the browser, allowing for credentials to be safe from theft. We plan to extend this research, exploring other ways browser-supported authentication can improve the security and usability of authentication systems.6

We have also conducted usability studies of various web authentication systems7, including an in-depth systematization of the use cases design paradigms for password managers.8 Based on this research, we have identified the following research plan to increase the utility, utilization, and usability of password managers. First, we will quantify the ability of entry-aware password generation—generating passwords that are easy to enter on devices where the password manager is unavailable—to increase users’ willingness to use generated passwords. Second, we will quantify the extent to which prioritizing suggestions from password health checks improves users’ compliance with those suggested actions. Third, we will describe the unique processes, needs, and challenges faced by parents and children using password managers and quantify the extent to which system designs addressing this reality improve usability and security. This research plan is creative and original in that it breaks from the status quo of high-level usability assessments, instead quantifying and explaining the ability of system designs to address specific usability issues and improve the utilization of security features. Furthermore, it will be the first research investigating password health checks and multi-user password manager usage.

  1. Verizon. 2021. Data breach investigations report. https://www.verizon.com/business/resources/reports/dbir/. ↩ ↩2

  2. Reynolds et al. 2018. A tale of two studies: The best and worst of YubiKey usability. In Proceedings of the 38th IEEE Symposium on Security and Privacy. IEEE. ↩

  3. Lyastani et al. 2018. Better Managed Than Memorized? Studying the Impact of Managers on Password Strength and Reuse. In Proceedings of the 28th USENIX Security Symposium. USENIX. ↩

  4. Oesch and Ruoti. 2020. That was then, this is now: A security evaluation of password generation, storage, and autofill in browser-based password managers. In Proceedings of the 30th USENIX Security Symposium. USENIX. ↩

  5. Oesch et al. 2021. The emperor’s new autofill framework: A security analysis of autofill on iOS and Android. In Proceedings of the 37th Annual Computer Security Applications Conference. ACM. ↩

  6. Ruoti and Seamons. 2017. End-to-end passwords. In Proceedings of the 20th New Security Paradigms Workshop. ACM. ↩

  7. Ruoti et al. 2015. Authentication melee: A usability analysis of seven web authentication systems. In Proceedings of the 24th International Conference on World Wide Web. Internet Society. ↩

  8. Simmons et al. 2021. Systematization of password manager use cases and design paradigms. In Proceedings of the 37th Annual Computer Security Applications Conference. ACM. ↩

Publications

Journals and Magazines

SenCAPTCHA: A Mobile-First CAPTCHA Using Orientation Sensors. Yunhe Feng, Qing Cao, Hairong Qi, and Scott Ruoti. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 4, No. 2, June 2020. ACM, 2020. (IMWUT/UbiComp)
Abstract:  CAPTCHAs are used to distinguish between human- and computer-generated (i.e., bot) online traffic. As there is an ever-increasing amount of online traffic from mobile devices, it is necessary to design CAPTCHAs that work well on mobile devices. In this paper, we present SenCAPTCHA, a mobile-first CAPTCHA that leverages the device's orientation sensors. SenCAPTCHA works by showing users an image of an animal and asking them to tilt their device to guide a red ball into the center of that animal's eye. SenCAPTCHA is especially useful for devices with small screen sizes (e.g., smartphones, smartwatches). In this paper, we describe the design of SenCAPTCHA and demonstrate that it is resilient to various machine learning based attacks. We also report on two usability studies of SenCAPTCHA involving a total of 472 participants; our results show that SenCAPTCHA is viewed as an "enjoyable" CAPTCHA and that it is preferred by over half of the participants to other existing CAPTCHA systems.

Conferences

A Comparison of Three Approaches to Assist Users in Memorizing System-Assigned Passwords. Michael Clark, Scott Ruoti, and Michael Mendoza Kent Seamons. In Proceedings of the 13th Symposium on Usable Security. ACM, 2024. (USEC)
Abstract:  Users struggle to select strong passwords. System-assigned passwords address this problem, but they can be difficult for users to memorize. While password managers can help store system-assigned passwords, there will always be passwords that a user needs to memorize, such as their password manager's master password. As such, there is a critical need for research into helping users memorize system-assigned passwords. In this work, we compare three different designs for password memorization aids inspired by the method of loci or memory palace. Design One displays a two-dimensional scene with objects placed inside it in arbitrary (and randomized) positions, with Design Two fixing the objects' position within the scene, and Design Three displays the scene using a navigable, three-dimensional representation. In an A-B study of these designs, we find that, surprisingly, there is no statistically significant difference between the memorability of these three designs, nor that of assigning users a passphrase to memorize, which we used as the control in this study. However, we find that when perfect recall failed, our designs helped users remember a greater portion of the encoded system-assigned password than did a passphrase, a property we refer to as durability. Our results indicate that there could be room for memorization aids that incorporate fuzzy or error-correcting authentication. Similarly, our results suggest that simple (i.e., cheap to develop) designs of this nature may be just as effective as more complicated, high-fidelity (i.e., expensive to develop) designs.
"If I Could Do This, I Feel Anyone Could:" The Design and Evaluation of A Secondary Authentication Factor Manager. Garrett Smith, Tarun Yadav, Jonathan Dutson, Scott Ruoti, and Kent Seamons. In Proceedings of the 32nd USENIX Security Symposium. USENIX, 2023. (USENIX Security, 29% acceptance rate)
Abstract:  Two-factor authentication (2FA) defends against account compromise by protecting an account with both a password—the primary authentication factor—and a device or resource that is hard to steal—the secondary authentication factor (SAF). However, prior research shows that users need help registering their SAFs with websites and successfully enabling 2FA. To address these issues, we propose the concept of a SAF manager that helps users manage SAFs through their entire life cycle: setup, authentication, removal, replacement, and auditing. We design and implement two proof-of-concept prototypes. In a between-subjects user study (N=60), we demonstrate that our design improves users' ability to correctly and quickly setup and remove a SAF on their accounts. Qualitative results show that users responded very positively to the SAF manager and were enthusiastic about its ability to help them rapidly replace a SAF. Furthermore, our SAF manager prevented fatal errors that users experienced when not using the manager.
Improving Password Generation Through The Design of A Password Composition Policy Description Language. Anuj Gautam, Shan Lalani, and Scott Ruoti. In Proceedings of the 18th Symposium on Usable Privacy and Security. USENIX, 2022. (SOUPS, 25% acceptance rate)
Abstract:  Password managers help users more effectively manage their passwords, yet the adoption of password generation is minimal. One explanation for this problem is that websites' password composition policies (PCPs) can reject generated passwords, creating a usability impediment. To address this issue, we design a PCP language that websites use to describe their PCP and that managers use to generate compliant passwords. We develop this language using an iterative process involving an extensive collection of PCPs scraped from the Web. We provide libraries for adopting our PCP language into websites and password managers and build proof-of-concept prototypes to verify the real-world feasibility of our PCP language. Using a 25-person user study, we demonstrate that our language and libraries are easy to pick up and correctly use for novice developers. Finally, we replicate and extend past research evaluating Web PCPs, showing that half of PCPs fail to require passwords that resist offline attacks when considering that users prefer certain character classes when selecting their passwords.
"It Basically Started Using Me:" An Observational Study of Password Manager Usage. Sean Oesch, Anuj Gautam, and Scott Ruoti. In Proceedings of the 40th ACM Conference on Human Factors in Computing Systems. ACM, 2022. (CHI, 24% acceptance rate)
Abstract:  There is limited information regarding how users employ password managers in the wild and why they use them in that manner. To address this knowledge gap, we conduct observational interviews with 32 password manager users. Using grounded theory, we identify four theories describing the processes and rationale behind participants' usage of password managers. We find that many users simultaneously use both a browser-based and a third-party manager, using each as a backup for the other, with this new paradigm having intriguing usability and security implications. Users also eschew generated passwords because these passwords are challenging to enter and remember when the manager is unavailable, necessitating new generators that create easy-to-enter and remember passwords. Additionally, the credential audits provided by most managers overwhelm users, limiting their utility and indicating a need for more proactive and streamlined notification systems. We also discuss mobile usage, adoption and promotion, and other related topics.
Systematization of Password Manager Use Cases and Design Paradigms. James Simmons, Oumar Diallo, Sean Oesch, and Scott Ruoti. In Proceedings of the 37th Annual Computer Security Applications Conference. ACM, 2021. (ACSAC, 24% acceptance rate)
Abstract:  Despite efforts to replace them, passwords remain the primary form of authentication on the web. Password managers seek to address many of the problems with passwords by helping users generate, store, and fill strong and unique passwords. Even though experts frequently recommend password managers, there is limited information regarding their usability. To aid in designing such usability studies, we systematize password manager use cases, identifying ten essential use cases, three recommended use cases, and four extended use cases. We also systematize the system designs employed to satisfy these use cases, designs that should be examined in usability studies to understand their relative strengths and weaknesses. Finally, we describe observations from 136 cognitive walkthroughs exploring the identified essential use cases in eight popular managers. Ultimately, we expect that this work will serve as the foundation for an explosion of new research into the usability of password managers.
The Emperor's New Autofill Framework: A Security Analysis of Autofill on iOS and Android. Sean Oesch, Anuj Gautam, and Scott Ruoti. In Proceedings of the 37th Annual Computer Security Applications Conference. ACM, 2021. (ACSAC, 24% acceptance rate)
Abstract:  Password managers help users more effectively manage their passwords, encouraging them to adopt stronger passwords across their many accounts. In contrast to desktop systems where password managers receive no system-level support, mobile operating systems provide autofill frameworks designed to integrate with password managers to provide secure and usable autofill for browsers and other apps installed on mobile devices. In this paper, we evaluate mobile autofill frameworks on iOS and Android, examining whether they achieve substantive benefits over the ad-hoc desktop environment or become a problematic single point of failure. Our results find that while the frameworks address several common issues, they also enforce insecure behavior and fail to provide password managers sufficient information to override the frameworks' insecure behavior, resulting in mobile managers being less secure than their desktop counterparts overall. We also demonstrate how these frameworks act as a confused deputy in manager-assisted credential phishing attacks. Our results demonstrate the need for significant improvements to mobile autofill frameworks. We conclude the paper with recommendations for the design and implementation of secure autofill frameworks.
That Was Then, This Is Now: A Security Evaluation of Password Generation, Storage, and Autofill in Browser-Based Password Managers. Sean Oesch and Scott Ruoti. In Proceedings of the 30th USENIX Security Symposium. USENIX, 2020. (USENIX Security, 16% acceptance rate)
Abstract:  Password managers have the potential to help users more effectively manage their passwords and address many of the concerns surrounding password-based authentication, however prior research has identified significant vulnerabilities in existing password managers. Since that time, five years has passed, leaving it unclear whether password managers remain vulnerable or whether they are now ready for broad adoption. To answer this question, we evaluate thirteen popular password managers and consider all three stages of the password manager lifecycle—password generation, storage, and autofill. Our evaluation is the first analysis of password generation in password managers, finding several non-random character distributions and identifying instances where generated passwords were vulnerable to online and offline guessing attacks. For password storage and autofill, we replicate past evaluations, demonstrating that while password managers have improved in the half-decade since those prior evaluations, there are still significant issues, particularly with browser-based password managers; these problems include unencrypted metadata, unsafe defaults, and vulnerabilities to clickjacking attacks. Based on our results, we identify password managers to avoid, provide recommendations on how to improve existing password managers, and identify areas of future research.
A Tale of Two Studies: The Best and Worst of YubiKey Usability. Joshua Reynolds, Trevor Smith, Ken Reese, Luck Dickinson, Scott Ruoti, and Kent Seamons. In Proceedings of the 38th IEEE Symposium on Security and Privacy. IEEE, 2018. (S&P, 11% acceptance rate)
Abstract:  Two-factor authentication (2FA) significantly improves the security of password-based authentication. Recently, there has been increased interest in Universal 2nd Factor (U2F) security keys-small hardware devices that require users to press a button on the security key to authenticate. To examine the usability of security keys in non-enterprise usage, we conducted two user studies of the YubiKey, a popular line of U2F security keys. The first study tasked 31 participants with configuring a Windows, Google, and Facebook account to authenticate using a YubiKey. This study revealed problems with setup instructions and workflow including users locking themselves out of their operating system or thinking they had successfully enabled 2FA when they had not. In contrast, the second study had 25 participants use a YubiKey in their daily lives over a period of four weeks, revealing that participants generally enjoyed the experience. Conducting both a laboratory and longitudinal study yielded insights into the usability of security keys that would not have been evident from either study in isolation. Based on our analysis, we recommend standardizing the setup process, enabling verification of success, allowing shared accounts, integrating with operating systems, and preventing lockouts.
Authentication Melee: A Usability Analysis of Seven Web Authentication Systems. Scott Ruoti, Brent Roberts, and Kent Seamons. In Proceedings of the 24th International Conference on World Wide Web. Internet Society, 2015. (WWW, 14% acceptance rate)
Abstract:  Passwords continue to dominate the authentication landscape in spite of numerous proposals to replace them. Even though usability is a key factor in replacing passwords, very few alternatives have been subjected to formal usability studies, and even fewer have been analyzed using a standard metric. We report the results of four within-subjects usability studies for seven web authentication systems. These systems span federated, smartphone, paper tokens, and email-based approaches. Our results indicate that participants prefer single sign-on systems. We report several insightful findings based on participants' qualitative responses: (1) transparency increases usability but also leads to confusion and a lack of trust, (2) participants prefer single sign-on but wish to augment it with site-specific low-entropy passwords, and (3) participants are intrigued by biometrics and phone-based authentication. We utilize the Systems Usability Scale (SUS) as a standard metric for empirical analysis and find that it produces reliable, replicable results. SUS proves to be an accurate measure of baseline usability. We recommend that new authentication systems be formally evaluated for usability using SUS, and should meet a minimum acceptable SUS score before receiving serious consideration.

Workshops

Evaluating Password Composition Policy and Password Meters of Popular Websites. Kyungchan Lim, Joshua H Kang, Matthew Dixson, Hyungjoon Koo, and Doowon Kim. In Proceedings of the 4th SecWeb Workshop. IEEE, 2023. (SecWeb)
Abstract:  Password-based authentication is one of the most commonly adopted mechanisms for online security. Choosing strong passwords is crucial for protecting ones' digital identities and assets, as weak passwords can be readily guessable, resulting in a compromise such as unauthorized access. To promote the use of strong passwords on the Web, the National Institute of Standards and Technology (NIST) provides website administrators with password composition policy (PCP) guidelines. We manually inspect popular websites to check if their password policies conform to NIST's PCP guidelines by generating passwords that meet each criterion and testing the 100 popular websites. Our findings reveal that a considerable number of web sites (on average, 53.5 %) do not comply with the guidelines, which could result in password breaches.
End-to-End Passwords. Scott Ruoti and Kent Seamons. In Proceedings of the 20th New Security Paradigms Workshop. ACM, 2017. (NSPW)
Abstract:  Passwords continue to be an important means for users to authenticate themselves to applications, websites, and backend services. However, password theft continues to be a significant issue, due in large part to the significant attack surface for passwords, including the operating system (e.g., key loggers), application (e.g., phishing websites in browsers), during transmission (e.g., TLS man-in-the-middle proxies), and at password verification services (e.g., theft of passwords stored at a server). Relatedly, even though there is a large body of research on improving passwords, the massive number of application verification services that use passwords stymie the diffusion of improvements—i.e., it does not scale for each improvement to require an update to every application and verification service. To address these problems, we propose a new end-to-end password paradigm that transfers password functionality to two end-points, the operating system (entry, management, storage, and verification) and the password verification service (verification, and verification token storage). In this paradigm, passwords are never shared with applications or transmitted over the network, but are instead verified using zero-knowledge protocols. There are five key benefits of this approach that are not possible with the current password paradigm: (a) a minimal attack surface, (b) protection from password phishing, (c) protection from malware, (d) consistent password policies, and (e) the ability to more rapidly diffuse improvements from password research.
Augmenting Centralized Password Management with Application-Specific Passwords. Trevor Smith, Scott Ruoti, and Kent Seamons. In Proceedings of the 3rd Workshop on Who Are You?! Adventures in Authentication. USENIX, 2017. (WAY)
Abstract:  Password authentication is the most prevalent form of authentication; however, passwords have numerous usability issues. For example, due to the large number and high complexity required of passwords, users frequently reuse and choose weak passwords. One way to address these problems is to centralize password management by using a password manager or single sign-on. While this centralizing approach can improve a user's security, it also magnifies the damage caused by a compromise of the user's master password. In this paper, we describe a new approach to enhance centralized password management using application-specific passwords. This approach prevents the compromise of a master password from immediately compromising all associated applications and instead, requires the attacker to conduct further online attacks against individual applications. We detail five possible system designs for application-specific passwords and describe our plans for user studies to test the acceptance and usability of this approach.
Standard Metrics and Scenarios for Usable Authentication. Scott Ruoti and Kent Seamons. In Proceedings of the 2nd Workshop on Who Are You?! Adventures in Authentication. USENIX, 2016. (WAY)
Abstract:  There is a constant flow of new authentication schemes proposed in the literature. In the past, most proposed schemes were not evaluated empirically, though in recent years there has been an increase in the number of authentication systems that have undergone a user study. Still, most of these user studies employ ad-hoc metrics (e.g., task completion time) and a unique scenario. Bonneau et al. included usability criteria in their heuristic evaluation of various types of web authentication mechanisms.…
Strengthening Passwords-Based Authentication. Scott Ruoti, Jeff Andersen, and Kent Seamons. In Proceedings of the 2nd Workshop on Who Are You?! Adventures in Authentication. USENIX, 2016. (WAY)
Abstract:  Even with years of research into new authentication technologies, passwords still dominate the authentication landscape. This is due primarily to a combination of security, deployability, and usability that has been difficult to match. While password alternatives exist, their lack of widespread adoption indicates that for the foreseeable future passwords are here to stay.…