Scott Ruoti

Director of the USER Lab
Assistant Professor

Email: ruoti@utk.edu
Address: Min H. Kao Building, Room 203
1520 Middle Drive
Knoxville, TN 37996-2250
Phone: 865-974-5449
Fax: 865-974-5483
CV: Download

Areas of Interest

Research

There is a significant gap between the theoretical security and privacy properties discussed in academic circles and those available to individuals in their day-to-day lives. For example, password managers were designed to get users to abandon human-selected passwords with weak security in favor of computer-generated passwords with strong security; however, in practice, due to various usability impediments, users forego generated passwords, using password managers primarily to store their human-selected passwords. In my research, I seek to identify the design principles necessary to bridge the gap between theoretical and practical security and privacy.

My research approach is multi-faceted. First, I conduct empirical user studies of security and privacy systems, whether academic prototypes or software deployed in the wild, measuring what issues (e.g., usability, functionality, key management requirements) impeded the adoption and correct usage of these systems. These efforts are enhanced through (a) collaboration with application-domain experts, (b) surveys and interviews of stakeholders, and (c) measurement studies of existing deployments, helping me to more concretely identify key requirements and design constraints for the systems under study.

Next, I rigorously explore the design space for these systems, looking for designs with the potential to address the issues previously identified. These designs can include novel cryptographic protocols and user interfaces, as well as designs improving other aspects of user experience. I evaluate each of these designs theoretically from both a security (e.g., threat modeling, proofs) and usability (e.g., cognitive walkthroughs, expert review) perspective.

Lastly, for the most promising designs, I develop proof-of-concept prototypes and conduct empirical user studies to ensure that these prototypes satisfy stakeholder requirements and identified design constraints under real-world usage. Critically, these evaluations include both technical evaluations—such as measuring latency or demonstrating the ability to stop an attack—and user-centered evaluations—such as usability testing and longitudinal studies. Based on the results of this work, I strive to extract generalizable design principles that could help improve the usability, security, and utility of other systems. Notably, I am one of only a few researchers who takes such a holistic approach to security research.

Bio

I’m a member of The Church of Jesus Christ of Latter-Day Saints. I’m married to Emily Ruoti and am the father of four children. When I’m not wrangling the children (which takes up most of my free time), I enjoy swimming, biking, and playing video games.

Education
  • Ph.D. in Computer Science, Brigham Young University, 2016
  • M.S. in Computer Science, Brigham Young University, 2015
  • B.S. in Computer Science, Brigham Young University, 2011
  • B.A. in Chinese, Brigham Young University, 2011
Prior Work Experience

Prior to my time at the University of Tennessee, I was a researcher at MIT Lincoln Laboratory. While there, I led a range of efforts, including acting as the chief architect for the Department of Homeland Security’s Cyber.gov program that is tasked with creating a next-generation cybersecurity architecture for all non-DoD federal departments and agencies. I also led a research team exploring non-cryptocurrency usages for Blockchain technology. Prior to my time at MIT Lincoln Laboratory I’ve also worked at Microsoft, Microsoft Research, Google, Blue Coat Systems (Symantec), and Sandia National Laboratories.

Dissertation

As part of my dissertation, I designed email systems that are both secure and easy-to-use, especially for novice users. The final version of our secure email system outperforms other similar systems in terms of usability, ranking in the top 15% among the hundreds of software systems subjected to a standard usability test. My design reduced user errors from 25% to 2%, and increased user understanding and trust in secure email.

Selected Recent Publications

"If I Could Do This, I Feel Anyone Could:" The Design and Evaluation of A Secondary Authentication Factor Manager. Garrett Smith, Tarun Yadav, Jonathan Dutson, Scott Ruoti, and Kent Seamons. In Proceedings of the 32nd USENIX Security Symposium. USENIX, 2023. (USENIX Security, 29% acceptance rate)
Abstract:  Two-factor authentication (2FA) defends against account compromise by protecting an account with both a password—the primary authentication factor—and a device or resource that is hard to steal—the secondary authentication factor (SAF). However, prior research shows that users need help registering their SAFs with websites and successfully enabling 2FA. To address these issues, we propose the concept of a SAF manager that helps users manage SAFs through their entire life cycle: setup, authentication, removal, replacement, and auditing. We design and implement two proof-of-concept prototypes. In a between-subjects user study (N=60), we demonstrate that our design improves users' ability to correctly and quickly setup and remove a SAF on their accounts. Qualitative results show that users responded very positively to the SAF manager and were enthusiastic about its ability to help them rapidly replace a SAF. Furthermore, our SAF manager prevented fatal errors that users experienced when not using the manager.
Improving Password Generation Through The Design of A Password Composition Policy Description Language. Anuj Gautam, Shan Lalani, and Scott Ruoti. In Proceedings of the 18th Symposium on Usable Privacy and Security. USENIX, 2022. (SOUPS, 25% acceptance rate)
Abstract:  Password managers help users more effectively manage their passwords, yet the adoption of password generation is minimal. One explanation for this problem is that websites' password composition policies (PCPs) can reject generated passwords, creating a usability impediment. To address this issue, we design a PCP language that websites use to describe their PCP and that managers use to generate compliant passwords. We develop this language using an iterative process involving an extensive collection of PCPs scraped from the Web. We provide libraries for adopting our PCP language into websites and password managers and build proof-of-concept prototypes to verify the real-world feasibility of our PCP language. Using a 25-person user study, we demonstrate that our language and libraries are easy to pick up and correctly use for novice developers. Finally, we replicate and extend past research evaluating Web PCPs, showing that half of PCPs fail to require passwords that resist offline attacks when considering that users prefer certain character classes when selecting their passwords.
"It Basically Started Using Me:" An Observational Study of Password Manager Usage. Sean Oesch, Anuj Gautam, and Scott Ruoti. In Proceedings of the 40th ACM Conference on Human Factors in Computing Systems. ACM, 2022. (CHI, 24% acceptance rate)
Abstract:  There is limited information regarding how users employ password managers in the wild and why they use them in that manner. To address this knowledge gap, we conduct observational interviews with 32 password manager users. Using grounded theory, we identify four theories describing the processes and rationale behind participants' usage of password managers. We find that many users simultaneously use both a browser-based and a third-party manager, using each as a backup for the other, with this new paradigm having intriguing usability and security implications. Users also eschew generated passwords because these passwords are challenging to enter and remember when the manager is unavailable, necessitating new generators that create easy-to-enter and remember passwords. Additionally, the credential audits provided by most managers overwhelm users, limiting their utility and indicating a need for more proactive and streamlined notification systems. We also discuss mobile usage, adoption and promotion, and other related topics.
The Emperor's New Autofill Framework: A Security Analysis of Autofill on iOS and Android. Sean Oesch, Anuj Gautam, and Scott Ruoti. In Proceedings of the 37th Annual Computer Security Applications Conference. ACM, 2021. (ACSAC, 24% acceptance rate)
Abstract:  Password managers help users more effectively manage their passwords, encouraging them to adopt stronger passwords across their many accounts. In contrast to desktop systems where password managers receive no system-level support, mobile operating systems provide autofill frameworks designed to integrate with password managers to provide secure and usable autofill for browsers and other apps installed on mobile devices. In this paper, we evaluate mobile autofill frameworks on iOS and Android, examining whether they achieve substantive benefits over the ad-hoc desktop environment or become a problematic single point of failure. Our results find that while the frameworks address several common issues, they also enforce insecure behavior and fail to provide password managers sufficient information to override the frameworks' insecure behavior, resulting in mobile managers being less secure than their desktop counterparts overall. We also demonstrate how these frameworks act as a confused deputy in manager-assisted credential phishing attacks. Our results demonstrate the need for significant improvements to mobile autofill frameworks. We conclude the paper with recommendations for the design and implementation of secure autofill frameworks.
SoK: Securing Email—A Stakeholder-Based Analysis. Jeremy Clark, P.C. van Oorschot, Scott Ruoti, Kent Seamons, and Daniel Zappala. In Proceedings of the 25th International Conference on Financial Cryptography and Data Security. Springer, 2021. (FC, 25% acceptance rate)
Abstract:  While email is the most ubiquitous and interoperable form of online communication today, it was not conceived with strong security guarantees, and the ensuing security enhancements are, by contrast, lacking in both ubiquity and interoperability. This situation motivates our research. We begin by identifying a variety of stakeholders who have an interest in the current email system and in efforts to provide secure solutions. We then use the tussle among stakeholders to explain the evolution of fragmented secure email solutions undertaken by industry, academia, and independent developers. We conclude with a fresh look at the state of secure email and discuss open problems in the area. An extended version of our paper includes an evaluation framework for proposed or deployed secure email systems and identify how well they meet properties related to security, utility, deployability, and usability.

Journals and Magazines

User Perceptions of Security and Privacy for Group Chat. Sean Oesch, Ruba Abu-Salma, Oumar Diallo, Juliane Krämer, James Simmons, Justin Wu, and Scott Ruoti. In ACM Digital Threats: Research and Practice. ACM, 2022. (DTRAP)
Abstract:  Secure messaging tools are an integral part of modern society. To understand users’ security and privacy perceptions and requirements for secure group chat, we surveyed 996 respondents in the US and UK. Our results show that group chat presents important security and privacy challenges, some of which are not present in one-to-one chat. For example, users need to be able to manage and monitor group membership, establish trust for new group members, and filter content that they share in different chat contexts. We also find that respondents lack mechanisms for determining which tools are secure and instead rely on non-technical strategies for protecting their privacy—for example, self-filtering and carefully tracking group membership.

To better understand how these results relate to existing tools, we conduct cognitive walkthroughs (a form of expert usability review) for five popular group chat tools. Our results demonstrate that while existing tools address some items identified in our surveys, this support is partial and is insufficient in many cases. As such, there is a need for improved group chat tools that better align with user perceptions and requirements. Based on these findings, we provide recommendations on improving the security and usability of secure group chat.
SenCAPTCHA: A Mobile-First CAPTCHA Using Orientation Sensors. Yunhe Feng, Qing Cao, Hairong Qi, and Scott Ruoti. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 4, No. 2, June 2020. ACM, 2020. (IMWUT/UbiComp)
Abstract:  CAPTCHAs are used to distinguish between human- and computer-generated (i.e., bot) online traffic. As there is an ever-increasing amount of online traffic from mobile devices, it is necessary to design CAPTCHAs that work well on mobile devices. In this paper, we present SenCAPTCHA, a mobile-first CAPTCHA that leverages the device's orientation sensors. SenCAPTCHA works by showing users an image of an animal and asking them to tilt their device to guide a red ball into the center of that animal's eye. SenCAPTCHA is especially useful for devices with small screen sizes (e.g., smartphones, smartwatches). In this paper, we describe the design of SenCAPTCHA and demonstrate that it is resilient to various machine learning based attacks. We also report on two usability studies of SenCAPTCHA involving a total of 472 participants; our results show that SenCAPTCHA is viewed as an "enjoyable" CAPTCHA and that it is preferred by over half of the participants to other existing CAPTCHA systems.
Blockchain Technology: What Is It Good For? Scott Ruoti, Ben Kaiser, Arkady Yerukhimovich, Jeremy Clark, and Robert Cunningham. In Communications of the ACM, Vol. 63, No. 1, pages 46–53. ACM, 2020. (CACM)
Abstract:  Bitcoin's success has led to significant interest in its underlying components, particularly blockchain technology. Over 10 years after Bitcoin's initial release, the community still suffers from a lack of clarity regarding what properties defines blockchain technology, its relationship to similar technologies, and which of its proposed use-cases are tenable and which are little more than hype. In this paper we answer four common questions regarding blockchain technology: (1) what exactly is blockchain technology, (2) what capabilities does it provide, and (3) what are good applications for blockchain technology, and (4) how does it relate to other distributed technologies (e.g., distributed databases). We accomplish this goal by using grounded theory (a structured approach to gathering and analyzing qualitative data) to thoroughly analyze a large corpus of literature on blockchain technology. This method enables us to answer the above questions while limiting researcher bias, separating thought leadership from peddled hype and identifying open research questions related to blockchain technology. The audience for this paper is broad as it aims to help researchers in a variety of areas come to a better understanding of blockchain technology and identify whether it may be of use in their own research.
Blockchain Technology: What Is It Good For? Scott Ruoti, Ben Kaiser, Arkady Yerukhimovich, Jeremy Clark, and Robert Cunningham. In ACM Queue, Vol. 17, No. 5, pages 60–87. ACM, 2019. (ACM Queue)
Abstract:  Bitcoin's success has led to significant interest in its underlying components, particularly blockchain technology. Over 10 years after Bitcoin's initial release, the community still suffers from a lack of clarity regarding what properties defines blockchain technology, its relationship to similar technologies, and which of its proposed use-cases are tenable and which are little more than hype. In this paper we answer four common questions regarding blockchain technology: (1) what exactly is blockchain technology, (2) what capabilities does it provide, and (3) what are good applications for blockchain technology, and (4) how does it relate to other distributed technologies (e.g., distributed databases). We accomplish this goal by using grounded theory (a structured approach to gathering and analyzing qualitative data) to thoroughly analyze a large corpus of literature on blockchain technology. This method enables us to answer the above questions while limiting researcher bias, separating thought leadership from peddled hype and identifying open research questions related to blockchain technology. The audience for this paper is broad as it aims to help researchers in a variety of areas come to a better understanding of blockchain technology and identify whether it may be of use in their own research.
Johnny's Journey Toward Usable Secure Email. Scott Ruoti and Kent Seamons. In IEEE Security & Privacy, Vol. 17, No. 6, pages 72–76, November/December 2019. IEEE, 2019. (S&P)
Abstract:  Since the publication of Why Johnny Can't Encrypt there has been interest in creating usable, secure email that is adoptable by the general public. In this article, we summarize research from the usable-security community on this topic, identify open problems, and call for more research on usable key management.
A Usability Study of Four Secure Email Tools Using Paired Participants. Scott Ruoti, Jeff Andersen, Luke Dickinson, Scott Heidbrink, Tyler Monson, Mark O'Neill, Ken Reese, Brad Spendlove, Elham Vaziripour, Justin Wu, Daniel Zappala, and Kent Seamons. In ACM Transactions on Privacy and Security, Vol. 22, No. 2, pages 22–29. ACM, 2019. (TOPS)
Abstract:  Secure email is increasingly being touted as usable by novice users, with a push for adoption based on recent concerns about government surveillance. To determine whether secure email is ready for grassroots adoption, we employ a laboratory user study that recruits pairs of novice users to install and use several of the latest systems to exchange secure messages. We present both quantitative and qualitative results from 28 pairs of novices as they use Private WebMail (Pwm), Tutanota, and Virtru and 10 pairs of novices as they use Mailvelope. Participants report being more at ease with this type of study and better able to cope with mistakes since both participants are “on the same page.” We find that users prefer integrated solutions over depot-based solutions and that tutorials are important in helping first-time users. Finally, our results demonstrate that Pretty Good Privacy using manual key management is still unusable for novice users, with 9 of 10 participant pairs failing to complete the study.
TLS Proxies: How Often and Who Cares? Mark O'Neill, Scott Ruoti, Kent Seamons, and Daniel Zappala. In IEEE Internet Computing, Vol. 21, No. 3, pages 22–29, May/June 2017. IEEE, 2017.
Abstract:  TLS inspection—inline decryption, inspection, and re-encryption of TLS traffic—is a controversial practice used for both benevolent and malicious purposes. This article describes measurements of how often TLS inspection occurs and reports on a survey of the general public regarding the practice of TLS inspection. This helps inform security researchers and policymakers regarding current practices and user preferences.

Conferences

A Comparison of Three Approaches to Assist Users in Memorizing System-Assigned Passwords. Michael Clark, Scott Ruoti, and Michael Mendoza Kent Seamons. In Proceedings of the 13th Symposium on Usable Security. ACM, 2024. (USEC)
Abstract:  Users struggle to select strong passwords. System-assigned passwords address this problem, but they can be difficult for users to memorize. While password managers can help store system-assigned passwords, there will always be passwords that a user needs to memorize, such as their password manager's master password. As such, there is a critical need for research into helping users memorize system-assigned passwords. In this work, we compare three different designs for password memorization aids inspired by the method of loci or memory palace. Design One displays a two-dimensional scene with objects placed inside it in arbitrary (and randomized) positions, with Design Two fixing the objects' position within the scene, and Design Three displays the scene using a navigable, three-dimensional representation. In an A-B study of these designs, we find that, surprisingly, there is no statistically significant difference between the memorability of these three designs, nor that of assigning users a passphrase to memorize, which we used as the control in this study. However, we find that when perfect recall failed, our designs helped users remember a greater portion of the encoded system-assigned password than did a passphrase, a property we refer to as durability. Our results indicate that there could be room for memorization aids that incorporate fuzzy or error-correcting authentication. Similarly, our results suggest that simple (i.e., cheap to develop) designs of this nature may be just as effective as more complicated, high-fidelity (i.e., expensive to develop) designs.
"If I Could Do This, I Feel Anyone Could:" The Design and Evaluation of A Secondary Authentication Factor Manager. Garrett Smith, Tarun Yadav, Jonathan Dutson, Scott Ruoti, and Kent Seamons. In Proceedings of the 32nd USENIX Security Symposium. USENIX, 2023. (USENIX Security, 29% acceptance rate)
Abstract:  Two-factor authentication (2FA) defends against account compromise by protecting an account with both a password—the primary authentication factor—and a device or resource that is hard to steal—the secondary authentication factor (SAF). However, prior research shows that users need help registering their SAFs with websites and successfully enabling 2FA. To address these issues, we propose the concept of a SAF manager that helps users manage SAFs through their entire life cycle: setup, authentication, removal, replacement, and auditing. We design and implement two proof-of-concept prototypes. In a between-subjects user study (N=60), we demonstrate that our design improves users' ability to correctly and quickly setup and remove a SAF on their accounts. Qualitative results show that users responded very positively to the SAF manager and were enthusiastic about its ability to help them rapidly replace a SAF. Furthermore, our SAF manager prevented fatal errors that users experienced when not using the manager.
Improving Password Generation Through The Design of A Password Composition Policy Description Language. Anuj Gautam, Shan Lalani, and Scott Ruoti. In Proceedings of the 18th Symposium on Usable Privacy and Security. USENIX, 2022. (SOUPS, 25% acceptance rate)
Abstract:  Password managers help users more effectively manage their passwords, yet the adoption of password generation is minimal. One explanation for this problem is that websites' password composition policies (PCPs) can reject generated passwords, creating a usability impediment. To address this issue, we design a PCP language that websites use to describe their PCP and that managers use to generate compliant passwords. We develop this language using an iterative process involving an extensive collection of PCPs scraped from the Web. We provide libraries for adopting our PCP language into websites and password managers and build proof-of-concept prototypes to verify the real-world feasibility of our PCP language. Using a 25-person user study, we demonstrate that our language and libraries are easy to pick up and correctly use for novice developers. Finally, we replicate and extend past research evaluating Web PCPs, showing that half of PCPs fail to require passwords that resist offline attacks when considering that users prefer certain character classes when selecting their passwords.
"It Basically Started Using Me:" An Observational Study of Password Manager Usage. Sean Oesch, Anuj Gautam, and Scott Ruoti. In Proceedings of the 40th ACM Conference on Human Factors in Computing Systems. ACM, 2022. (CHI, 24% acceptance rate)
Abstract:  There is limited information regarding how users employ password managers in the wild and why they use them in that manner. To address this knowledge gap, we conduct observational interviews with 32 password manager users. Using grounded theory, we identify four theories describing the processes and rationale behind participants' usage of password managers. We find that many users simultaneously use both a browser-based and a third-party manager, using each as a backup for the other, with this new paradigm having intriguing usability and security implications. Users also eschew generated passwords because these passwords are challenging to enter and remember when the manager is unavailable, necessitating new generators that create easy-to-enter and remember passwords. Additionally, the credential audits provided by most managers overwhelm users, limiting their utility and indicating a need for more proactive and streamlined notification systems. We also discuss mobile usage, adoption and promotion, and other related topics.
Systematization of Password Manager Use Cases and Design Paradigms. James Simmons, Oumar Diallo, Sean Oesch, and Scott Ruoti. In Proceedings of the 37th Annual Computer Security Applications Conference. ACM, 2021. (ACSAC, 24% acceptance rate)
Abstract:  Despite efforts to replace them, passwords remain the primary form of authentication on the web. Password managers seek to address many of the problems with passwords by helping users generate, store, and fill strong and unique passwords. Even though experts frequently recommend password managers, there is limited information regarding their usability. To aid in designing such usability studies, we systematize password manager use cases, identifying ten essential use cases, three recommended use cases, and four extended use cases. We also systematize the system designs employed to satisfy these use cases, designs that should be examined in usability studies to understand their relative strengths and weaknesses. Finally, we describe observations from 136 cognitive walkthroughs exploring the identified essential use cases in eight popular managers. Ultimately, we expect that this work will serve as the foundation for an explosion of new research into the usability of password managers.
The Emperor's New Autofill Framework: A Security Analysis of Autofill on iOS and Android. Sean Oesch, Anuj Gautam, and Scott Ruoti. In Proceedings of the 37th Annual Computer Security Applications Conference. ACM, 2021. (ACSAC, 24% acceptance rate)
Abstract:  Password managers help users more effectively manage their passwords, encouraging them to adopt stronger passwords across their many accounts. In contrast to desktop systems where password managers receive no system-level support, mobile operating systems provide autofill frameworks designed to integrate with password managers to provide secure and usable autofill for browsers and other apps installed on mobile devices. In this paper, we evaluate mobile autofill frameworks on iOS and Android, examining whether they achieve substantive benefits over the ad-hoc desktop environment or become a problematic single point of failure. Our results find that while the frameworks address several common issues, they also enforce insecure behavior and fail to provide password managers sufficient information to override the frameworks' insecure behavior, resulting in mobile managers being less secure than their desktop counterparts overall. We also demonstrate how these frameworks act as a confused deputy in manager-assisted credential phishing attacks. Our results demonstrate the need for significant improvements to mobile autofill frameworks. We conclude the paper with recommendations for the design and implementation of secure autofill frameworks.
SoK: Securing Email—A Stakeholder-Based Analysis. Jeremy Clark, P.C. van Oorschot, Scott Ruoti, Kent Seamons, and Daniel Zappala. In Proceedings of the 25th International Conference on Financial Cryptography and Data Security. Springer, 2021. (FC, 25% acceptance rate)
Abstract:  While email is the most ubiquitous and interoperable form of online communication today, it was not conceived with strong security guarantees, and the ensuing security enhancements are, by contrast, lacking in both ubiquity and interoperability. This situation motivates our research. We begin by identifying a variety of stakeholders who have an interest in the current email system and in efforts to provide secure solutions. We then use the tussle among stakeholders to explain the evolution of fragmented secure email solutions undertaken by industry, academia, and independent developers. We conclude with a fresh look at the state of secure email and discuss open problems in the area. An extended version of our paper includes an evaluation framework for proposed or deployed secure email systems and identify how well they meet properties related to security, utility, deployability, and usability.
Understanding User Perceptions of Security and Privacy for Group Chat: A Survey of Users in The US and UK. Sean Oesch, Ruba Abu-Salma, Oumar Diallo, Juliane Krämer, James Simmons, Justin Wu, and Scott Ruoti. In Proceedings of the 36th Annual Computer Security Applications Conference. ACM, 2020. (ACSAC, 23% acceptance rate)
Abstract:  Secure messaging tools are an integral part of modern society. While there is a significant body of secure messaging research generally, there is a lack of information regarding users' security and privacy perceptions and requirements for secure group chat. To address this gap, we conducted a survey of 996 participants in the US and UK. The results of our study show that group chat presents important security and privacy challenges, some of which are not present in one-to-one chat. For example, users need to be able to manage and monitor group membership, establish trust for new group members, and filter content that they share in different chat contexts. Similarly, we find that the sheer volume of notifications that occur in group chat makes it extremely likely that users ignore important security- or privacy- notifications. We also find that participants lack mechanisms for determining which tools are secure and instead rely on non-technical strategies for protecting their privacy—for example, self-filtering what they post and carefully tracking group membership. Based on these findings we provide recommendations on how to improve the security and usability of secure group chat.
That Was Then, This Is Now: A Security Evaluation of Password Generation, Storage, and Autofill in Browser-Based Password Managers. Sean Oesch and Scott Ruoti. In Proceedings of the 30th USENIX Security Symposium. USENIX, 2020. (USENIX Security, 16% acceptance rate)
Abstract:  Password managers have the potential to help users more effectively manage their passwords and address many of the concerns surrounding password-based authentication, however prior research has identified significant vulnerabilities in existing password managers. Since that time, five years has passed, leaving it unclear whether password managers remain vulnerable or whether they are now ready for broad adoption. To answer this question, we evaluate thirteen popular password managers and consider all three stages of the password manager lifecycle—password generation, storage, and autofill. Our evaluation is the first analysis of password generation in password managers, finding several non-random character distributions and identifying instances where generated passwords were vulnerable to online and offline guessing attacks. For password storage and autofill, we replicate past evaluations, demonstrating that while password managers have improved in the half-decade since those prior evaluations, there are still significant issues, particularly with browser-based password managers; these problems include unencrypted metadata, unsafe defaults, and vulnerabilities to clickjacking attacks. Based on our results, we identify password managers to avoid, provide recommendations on how to improve existing password managers, and identify areas of future research.
A Comparative Usability Study of Key Management in Secure Email. Scott Ruoti, Jeff Andersen, Tyler Monson, Daniel Zappala, and Kent Seamons. In Proceedings of the 14th Symposium on Usable Privacy and Security. USENIX, 2018. (SOUPS, 23% acceptance rate)
Abstract:  We conducted a user study that compares three secure email tools that share a common user interface and differ only by key management scheme: passwords, public key directory (PKD), and identity-based encryption (IBE). Our work is the first comparative (i.e., A/B) usability evaluation of three different key management schemes and utilizes a standard quantitative metric for cross-system comparisons. We also share qualitative feedback from participants that provides valuable insights into user attitudes regarding each key management approach and secure email generally. The study serves as a model for future secure email research with A/B studies, standard metrics, and the two-person study methodology.
A Tale of Two Studies: The Best and Worst of YubiKey Usability. Joshua Reynolds, Trevor Smith, Ken Reese, Luck Dickinson, Scott Ruoti, and Kent Seamons. In Proceedings of the 38th IEEE Symposium on Security and Privacy. IEEE, 2018. (S&P, 11% acceptance rate)
Abstract:  Two-factor authentication (2FA) significantly improves the security of password-based authentication. Recently, there has been increased interest in Universal 2nd Factor (U2F) security keys-small hardware devices that require users to press a button on the security key to authenticate. To examine the usability of security keys in non-enterprise usage, we conducted two user studies of the YubiKey, a popular line of U2F security keys. The first study tasked 31 participants with configuring a Windows, Google, and Facebook account to authenticate using a YubiKey. This study revealed problems with setup instructions and workflow including users locking themselves out of their operating system or thinking they had successfully enabled 2FA when they had not. In contrast, the second study had 25 participants use a YubiKey in their daily lives over a period of four weeks, revealing that participants generally enjoyed the experience. Conducting both a laboratory and longitudinal study yielded insights into the usability of security keys that would not have been evident from either study in isolation. Based on our analysis, we recommend standardizing the setup process, enabling verification of success, allowing shared accounts, integrating with operating systems, and preventing lockouts.
Intrusion Detection with Unsupervised Heterogeneous Ensembles Using Cluster-Based Normalization. Scott Ruoti, Scott Heidbrink, Mark O'Neill, Eric Gustafson, and Yung Ryn Choe. In Proceedings of the 24th IEEE International Conference on Web Services. IEEE, 2017. (ICWS, 22% acceptance rate)
Abstract:  Outlier detection has been shown to be a promising machine learning technique for a diverse array of fields and problem areas. However, traditional, supervised outlier detection is not well suited for problems such as network intrusion detection, where proper labelled data is scarce. This has created a focus on extending these approaches to be unsupervised, removing the need for explicit labels, but at a cost of poorer performance compared to their supervised counterparts. Recent work has explored ways of making up for this, such as creating ensembles of diverse models, or even diverse learning algorithms, to jointly classify data. While using unsupervised, heterogeneous ensembles of learning algorithms has been proposed as a viable next step for research, the implications of how these ensembles are built and used has not been explored.
PACE: Proactively-Secure Accumulo with Cryptographic Enforcement. Scott Ruoti, Ben Kaiser, Ariel Hamlin, Cassandra Sparks, and Robert Cunningham. In Proceedings of the 21st IEEE High Performance Extreme Computing Conference. IEEE, 2017. (HPEC)
Abstract:  Cloud-hosted databases have many compelling benefits, including high availability, flexible resource allocation, and resiliency to attack, but it requires that cloud tenants cede control of their data to the cloud provider. In this paper, we describe Proactively-secure Accumulo with Cryptographic Enforcement (PACE), a client-side library that cryptographically protects a tenant's data, returning control of that data to the tenant. PACE is a drop-in replacement for Accumulo's APIs and works with Accumulo's row-level security model. We evaluate the performance of PACE, discussing the impact of encryption and signatures on operation throughput.
TrustBase: An Architecture to Repair and Strengthen Certificate-Based Authentication. Mark O'Neill, Scott Heidbrink, Scott Ruoti, Jordan Whitehead, Dan Bunker, Luke Dickinson, Travis Hendershot, Joshua Reynolds, Kent Seamons, and Daniel Zappala. In Proceedings of the 27th USENIX Security Symposium. USENIX, 2017. (USENIX Security, 16% acceptance rate)
Abstract:  The current state of certificate-based authentication is messy, with broken authentication in applications and proxies, along with serious flaws in the CA system. To solve these problems, we design TrustBase, an architecture that provides certificate-based authentication as an operating system service, with system administrator control over authentication policy. TrustBase transparently enforces best practices for certificate validation on all applications, while also providing a variety of authentication services to strengthen the CA system. We describe a research prototype of TrustBase for Linux, which uses a loadable kernel module to intercept traffic in the socket layer, then consults a user-space policy engine to evaluate certificate validity using a variety of plugins. We evaluate the security of TrustBase, including a threat analysis, application coverage, and hardening of the Linux prototype. We also describe prototypes of TrustBase for Android and Windows, illustrating the generality of our approach. We show that TrustBase has negligible overhead and universal compatibility with applications. We demonstrate its utility by describing eight authentication services that extend CA hardening to all applications.
Layering Security at Global Control Points to Secure Unmodified Software. Scott Ruoti, Kent Seamons, and Daniel Zappala. In Proceedings of the 2nd IEEE Secure Development Conference. IEEE, 2017. Best Paper Award. (SecDev, 32% acceptance rate)
Abstract:  Developing secure software is inherently difficult, and is further hampered by a rush to market, the lack of cybersecurity-trained architects and developers, and the difficulty of identifying flaws and deploying mitigations. To address these problems, we advocate for an alternative paradigm-layering security onto applications from global control points, such as the browser, operating system, or network. This approach adds security to existing applications, relieving developers of this burden. The benefits of this paradigm are three-fold-(1) increased correctness in the implementation of security features, (2) coverage for all software, even non-maintained legacy software, and (3) more rapid and consistent deployment of threat mitigations and new security features. To demonstrate these benefits, we describe three concrete instantiations of this paradigm- MessageGuard, a system that layers end-to-end encryption in the browser; TrustBase, a system that layers authentication in the operating system; and software-defined perimeter, which layers access control at network middleboxes.
Weighing Context and Tradeoffs: How Suburban Adults Selected Their Online Security Posture. Scott Ruoti, Tyler Monson, Justin Wu, Kent Seamons, and Daniel Zappala. In Proceedings of the 13th Symposium on Usable Privacy and Security. USENIX, 2017. (SOUPS, 27% acceptance rate)
Abstract:  Understanding how people behave when faced with complex security situations is essential to designing usable security tools. To better understand users' perceptions of their digital lives and how they managed their online security posture, we conducted a series of 23 semi-structured interviews with mostly middle-aged parents from suburban Washington state. Using a grounded theory methodology, we analyzed the interview data and found that participants chose their security posture based on the immense value the Internet provides and their belief that no combination of technology could make them perfectly safe. Within this context, users have a four-stage process for determining which security measures to adopt: learning, evaluation of risks, estimation of impact, and weighing trade-offs to various coping strategies. Our results also revealed that a majority of participants understand the basic principles of symmetric encryption. We found that participants' misconceptions related to browser-based TLS indicators lead to insecure behavior, and it is the permanence of encrypted email that causes participants to doubt that it is secure. We conclude with a discussion of possible responses to this research and avenues for future research.
TLS Proxies: Friend or Foe? Mark O'Neill, Scott Ruoti, Kent Seamons, and Daniel Zappala. In Proceedings of the 17th ACM Internet Measurement Conference. ACM, 2016. (IMC, 25% acceptance rate)
Abstract:  We measure the prevalence and uses of TLS proxies using a Flash tool deployed with a Google AdWords campaign. We generate 2.9 million certificate tests and find that 1 in 250 TLS connections are TLS-proxied. The majority of these proxies appear to be benevolent, however we identify over 1,000 cases where three malware products are using this technology nefariously. We also find numerous instances of negligent, duplicitous, and suspicious behavior, some of which degrade security for users without their knowledge. Distinguishing these types of practices is challenging in practice, indicating a need for transparency and user awareness.
Private Webmail 2.0: Simple and Easy-to-Use Secure Email. Scott Ruoti, Jeff Andersen, Travis Hendershot, Kent Seamons, and Daniel Zappala. In Proceedings of the 29th ACM Symposium on User Interface Software and Technology. ACM, 2016. (UIST, 20% acceptance rate)
Abstract:  Private Webmail 2.0 (Pwm 2.0) improves upon the current state of the art by increasing the usability and practical security of secure email for ordinary users. More users are able to send and receive encrypted emails without mistakenly revealing sensitive information. In this paper we describe four user interface traits that positively affect the usability and security of Pwm 2.0. In a user study involving 51 participants we validate that these interface modifications result in high usability, few mistakes, and a strong understanding of the protection provided to secure email messages. We also show that the use of manual encryption has no effect on usability or security.
User Attitudes Toward The Inspection of Encrypted Traffic. Scott Ruoti, Mark O'Neill, Kent Seamons, and Daniel Zappala. In Proceedings of the 12th Symposium on Usable Privacy and Security. USENIX, 2016. (SOUPS, 28% acceptance rate)
Abstract:  This paper reports the results of a survey of 1,976 individuals regarding their opinions on TLS inspection, a controversial technique that can be used for both benevolent and malicious purposes. Responses indicate that participants hold nuanced opinions on security and privacy trade-offs, with most recognizing legitimate uses for the practice, but also concerned about threats from hackers or government surveillance. There is strong support for notification and consent when a system is intercepting their encrypted traffic, although this support varies depending on the situation. A significant concern about malicious uses of TLS inspection is identity theft, and many would react negatively and some would change their behavior if they discovered inspection occurring without their knowledge. We also find that a small but significant number of participants are jaded by the current state of affairs and have lost any expectation of privacy.
"We're on the same page": A Usability Study of Secure Email Using Pairs of Novice Users. Scott Ruoti, Jeff Andersen, Scott Heidbrink, Mark O'Neill, Elham Vaziripour, Justin Wu, Daniel Zappala, and Kent Seamons. In Proceedings of the 34th ACM Conference on Human Factors in Computing Systems. ACM, 2016. Honorable Mention for Best Paper. (CHI, 23% acceptance rate)
Abstract:  Secure email is increasingly being touted as usable by novice users, with a push for adoption based on recent concerns about government surveillance. To determine whether secure email is ready for grassroots adoption, we employ a laboratory user study that recruits pairs of novice users to install and use several of the latest systems to exchange secure messages. We present both quantitative and qualitative results from 25 pairs of novice users as they use Pwm, Tutanota, and Virtru. Participants report being more at ease with this type of study and better able to cope with mistakes since both participants are "on the same page". We find that users prefer integrated solutions over depot-based solutions, and that tutorials are important in helping first-time users. Hiding the details of how a secure email system provides security can lead to a lack of trust in the system. Participants expressed a desire to use secure email, but few wanted to use it regularly and most were unsure of when they might use it.
Authentication Melee: A Usability Analysis of Seven Web Authentication Systems. Scott Ruoti, Brent Roberts, and Kent Seamons. In Proceedings of the 24th International Conference on World Wide Web. Internet Society, 2015. (WWW, 14% acceptance rate)
Abstract:  Passwords continue to dominate the authentication landscape in spite of numerous proposals to replace them. Even though usability is a key factor in replacing passwords, very few alternatives have been subjected to formal usability studies, and even fewer have been analyzed using a standard metric. We report the results of four within-subjects usability studies for seven web authentication systems. These systems span federated, smartphone, paper tokens, and email-based approaches. Our results indicate that participants prefer single sign-on systems. We report several insightful findings based on participants' qualitative responses: (1) transparency increases usability but also leads to confusion and a lack of trust, (2) participants prefer single sign-on but wish to augment it with site-specific low-entropy passwords, and (3) participants are intrigued by biometrics and phone-based authentication. We utilize the Systems Usability Scale (SUS) as a standard metric for empirical analysis and find that it produces reliable, replicable results. SUS proves to be an accurate measure of baseline usability. We recommend that new authentication systems be formally evaluated for usability using SUS, and should meet a minimum acceptable SUS score before receiving serious consideration.
Confused Johnny: When Automatic Encryption Leads to Confusion and Mistakes. Scott Ruoti, Nathan Kim, Ben Burgon, Timothy W. van der Horst, and Kent Seamons. In Proceedings of the 9th Symposium on Usable Privacy and Security. ACM, 2013. (SOUPS, 29% acceptance rate)
Abstract:  A common approach to designing usable security is to hide as many security details as possible from the user to reduce the amount of information and actions a user must encounter. This paper gives an overview of Pwm (Private Webmail), our secure webmail system that uses security overlays to integrate tightly with existing webmail services like Gmail. Pwm's security is mostly transparent, including automatic key management and automatic encryption. We describe a series of Pwm user studies indicating that while nearly all users can use the system without any prior training, the security details are so transparent that a small percentage of users mistakenly sent out unencrypted messages and some users are unsure whether they should trust Pwm. We then conducted user studies with an alternative prototype to Pwm that uses manual encryption. Surprisingly users were accepting of the extra steps of cutting and pasting ciphertext themselves. They avoided mistakes and had more trust in the system with manual encryption. Our results suggest that designers may want to reconsider manual encryption as a way to reduce transparency and foster greater trust.
Private Facebook Chat. Chris Robison, Scott Ruoti, Timothy W. van der Horst, and Kent Seamons. In Proceedings of the 2012 International Conference on Privacy, Security, Risk, and Trust and 2012 International Conference on Social Computing. IEEE, 2012. (PASSAT/SocialCom 2012, 22% acceptance rate)
Abstract:  The number of instant messages sent per year now exceeds that of email. Recently users have been moving away from traditional instant messaging applications and instead using social networks as their primary communications platform. To discover attitudes related to instant messaging and its security, we have conducted a user survey. This paper also presents the design of PFC (Private Facebook Chat), a system providing convenient, secure instant messaging within Facebook Chat. PFC offers end-to-end encryption in order to thwart any eavesdropper, including Facebook itself. Finally, we have conducted a usability study of a PFC prototype.

Workshops

A Usability Study of Secure Email Deletion. Tyler Monson, Joshua Reynolds, Trevor Smith, Scott Ruoti, Daniel Zappala, and Kent Seamons. In Proceedings of the 3rd European Workshop on Usable Security. Internet Society, 2018. (EuroUSec)
Abstract:  Messaging applications like SnapChat illustrate that users are concerned about the permanence of information. We find that this concern extends to email. In this paper we present a usability study of an end-to-end secure email tool with the option to securely delete messages. This tool uses ephemeral keys, one per message thread, and default expiration times, with a user prompt to renew or delete keys. Deleting keys causes the messages in the thread to be unreadable for that user. We compare the usability of this tool to a nearly identical tool that uses long term keys and lacks a feature to expire keys. We also interview participants about their email use patterns and attitudes towards information permanence. We find that participants are especially interested in the ability to control the lifetime of an email message. Participants also report trusting the tool that allowed them to make their email messages ephemeral more than the tool that just encrypted their email.
End-to-End Passwords. Scott Ruoti and Kent Seamons. In Proceedings of the 20th New Security Paradigms Workshop. ACM, 2017. (NSPW)
Abstract:  Passwords continue to be an important means for users to authenticate themselves to applications, websites, and backend services. However, password theft continues to be a significant issue, due in large part to the significant attack surface for passwords, including the operating system (e.g., key loggers), application (e.g., phishing websites in browsers), during transmission (e.g., TLS man-in-the-middle proxies), and at password verification services (e.g., theft of passwords stored at a server). Relatedly, even though there is a large body of research on improving passwords, the massive number of application verification services that use passwords stymie the diffusion of improvements—i.e., it does not scale for each improvement to require an update to every application and verification service. To address these problems, we propose a new end-to-end password paradigm that transfers password functionality to two end-points, the operating system (entry, management, storage, and verification) and the password verification service (verification, and verification token storage). In this paradigm, passwords are never shared with applications or transmitted over the network, but are instead verified using zero-knowledge protocols. There are five key benefits of this approach that are not possible with the current password paradigm: (a) a minimal attack surface, (b) protection from password phishing, (c) protection from malware, (d) consistent password policies, and (e) the ability to more rapidly diffuse improvements from password research.
Augmenting Centralized Password Management with Application-Specific Passwords. Trevor Smith, Scott Ruoti, and Kent Seamons. In Proceedings of the 3rd Workshop on Who Are You?! Adventures in Authentication. USENIX, 2017. (WAY)
Abstract:  Password authentication is the most prevalent form of authentication; however, passwords have numerous usability issues. For example, due to the large number and high complexity required of passwords, users frequently reuse and choose weak passwords. One way to address these problems is to centralize password management by using a password manager or single sign-on. While this centralizing approach can improve a user's security, it also magnifies the damage caused by a compromise of the user's master password. In this paper, we describe a new approach to enhance centralized password management using application-specific passwords. This approach prevents the compromise of a master password from immediately compromising all associated applications and instead, requires the attacker to conduct further online attacks against individual applications. We detail five possible system designs for application-specific passwords and describe our plans for user studies to test the acceptance and usability of this approach.
Content-Based Security for The Web. Alexander Afanasyev, J. Alex Halderman, Scott Ruoti, Kent Seamons, Yingdi Yu, Daniel Zappala, and Lixia Zhang. In Proceedings of the 19th New Security Paradigms Workshop. ACM, 2016. (NSPW)
Abstract:  The World Wide Web has become the most common platform for building applications and delivering content. Yet despite years of research, the web continues to face severe security challenges related to data integrity and confidentiality. Rather than continuing the exploit-and-patch cycle, we propose addressing these challenges at an architectural level, by supplementing the web's existing connection-based and server-based security models with a new approach: content-based security. With this approach, content is directly signed and encrypted at rest, enabling it to be delivered via any path and then validated by the browser. We explore how this new architectural approach can be applied to the web and analyze its security benefits. We then discuss a broad research agenda to realize this vision and the challenges that must be overcome.
Standard Metrics and Scenarios for Usable Authentication. Scott Ruoti and Kent Seamons. In Proceedings of the 2nd Workshop on Who Are You?! Adventures in Authentication. USENIX, 2016. (WAY)
Abstract:  There is a constant flow of new authentication schemes proposed in the literature. In the past, most proposed schemes were not evaluated empirically, though in recent years there has been an increase in the number of authentication systems that have undergone a user study. Still, most of these user studies employ ad-hoc metrics (e.g., task completion time) and a unique scenario. Bonneau et al. included usability criteria in their heuristic evaluation of various types of web authentication mechanisms.…
Strengthening Passwords-Based Authentication. Scott Ruoti, Jeff Andersen, and Kent Seamons. In Proceedings of the 2nd Workshop on Who Are You?! Adventures in Authentication. USENIX, 2016. (WAY)
Abstract:  Even with years of research into new authentication technologies, passwords still dominate the authentication landscape. This is due primarily to a combination of security, deployability, and usability that has been difficult to match. While password alternatives exist, their lack of widespread adoption indicates that for the foreseeable future passwords are here to stay.…

Technical Reports

SoK: Blockchain Technology and Its Potential Use Cases. Scott Ruoti, Ben Kaiser, Arkady Yerukhimovich, Jeremy Clark, and Robert Cunningham. In arXiv, arXiv:1909.12454. 2019.
Abstract:  Bitcoin's success has led to significant interest in its underlying components, particularly blockchain technology. Over 10 years after Bitcoin's initial release, the community still suffers from a lack of clarity regarding what properties defines blockchain technology, its relationship to similar technologies, and which of its proposed use-cases are tenable and which are little more than hype. In this paper we answer four common questions regarding blockchain technology: (1) what exactly is blockchain technology, (2) what capabilities does it provide, and (3) what are good applications for blockchain technology, and (4) how does it relate to other distributed technologies (e.g., distributed databases). We accomplish this goal by using grounded theory (a structured approach to gathering and analyzing qualitative data) to thoroughly analyze a large corpus of literature on blockchain technology. This method enables us to answer the above questions while limiting researcher bias, separating thought leadership from peddled hype and identifying open research questions related to blockchain technology. The audience for this paper is broad as it aims to help researchers in a variety of areas come to a better understanding of blockchain technology and identify whether it may be of use in their own research.