Scott Ruoti

Director of the USER Lab
Assistant Professor

Email: ruoti@utk.edu
Address: Min H. Kao Building, Room 344
1520 Middle Drive
Knoxville, TN 37996-2250
Phone: 865-974-5449
Fax: 865-974-5483
CV: Download

Areas of Interest

Education

  • Ph.D. in Computer Science, Brigham Young University, 2016
  • M.S. in Computer Science, Brigham Young University, 2015
  • B.S. in Computer Science, Brigham Young University, 2011
  • B.A. in Chinese, Brigham Young University, 2011

Bio

My primary research focus is computer security and privacy. I am particularly interested in researching how to create software systems that ensure correct and secure operation when deployed to actual users. As such, I conduct holistic, interdisciplinary research using methods from the measurement, security and privacy, and human-computer interaction (HCI) communities. My research process starts by (1) measuring the real-world impact of a given problem, then (2) designing systems that address those issues, and finally (3) conducting usability studies to evaluate whether the developed systems achieves its goals under realistic usage conditions when operated by actual users.

My current interests include authentication (improving password managers and multi-factor authentication), usable encryption (usable secure email and messaging), helping software developers create secure software, and building systems with decentralized trust (blockchain technology).

Prior to my time at the University of Tennessee, I was a researcher at MIT Lincoln Laboratory. While there, I led a range of efforts, including acting as the chief architect for the Department of Homeland Security’s Cyber.gov program that is tasked with creating a next-generation cybersecurity architecture for all non-DoD federal departments and agencies. I also led a research team exploring non-cryptocurrency usages for Blockchain technology. Prior to my time at MIT Lincoln Laboratory I’ve also worked at Microsoft, Microsoft Research, Google, Blue Coat Systems (Symantec), and Sandia National Laboratories.

As part of my dissertation, I designed email systems that are both secure and easy-to-use, especially for novice users. The final version of our secure email system outperforms other similar systems in terms of usability, ranking in the top 15% among the hundreds of software systems subjected to a standard usability test. My design reduced user errors from 25% to 2%, and increased user understanding and trust in secure email.

Selected Recent Publications

SenCAPTCHA: A Mobile-First CAPTCHA Using Orientation Sensors. Yunhe Feng, Qing Cao, Hairong Qi, and Scott Ruoti. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 4, No. 2, June 2020. ACM, 2020. ACM, 2020. (IMWUT/UbiComp)
Abstract: CAPTCHAs are used to distinguish between human- and computer-generated (i.e., bot) online traffic. As there is an ever-increasing amount of online traffic from mobile devices, it is necessary to design CAPTCHAs that work well on mobile devices. In this paper, we present SenCAPTCHA, a mobile-first CAPTCHA that leverages the device's orientation sensors. SenCAPTCHA works by showing users an image of an animal and asking them to tilt their device to guide a red ball into the center of that animal's eye. SenCAPTCHA is especially useful for devices with small screen sizes (e.g., smartphones, smartwatches). In this paper, we describe the design of SenCAPTCHA and demonstrate that it is resilient to various machine learning based attacks. We also report on two usability studies of SenCAPTCHA involving a total of 472 participants; our results show that SenCAPTCHA is viewed as an "enjoyable" CAPTCHA and that it is preferred by over half of the participants to other existing CAPTCHA systems.
That Was Then, This Is Now: A Security Evaluation of Password Generation, Storage, and Autofill in Browser-Based Password Managers. Sean Oesch and Scott Ruoti. In Proceedings of the 30th USENIX Security Symposium. USENIX, 2020. (USENIX Security, 16% acceptance rate)
Abstract: Password managers have the potential to help users more effectively manage their passwords and address many of the concerns surrounding password-based authentication, however prior research has identified significant vulnerabilities in existing password managers. Since that time, five years has passed, leaving it unclear whether password managers remain vulnerable or whether they are now ready for broad adoption. To answer this question, we evaluate thirteen popular password managers and consider all three stages of the password manager lifecycle—password generation, storage, and autofill. Our evaluation is the first analysis of password generation in password managers, finding several non-random character distributions and identifying instances where generated passwords were vulnerable to online and offline guessing attacks. For password storage and autofill, we replicate past evaluations, demonstrating that while password managers have improved in the half-decade since those prior evaluations, there are still significant issues, particularly with browser-based password managers; these problems include unencrypted metadata, unsafe defaults, and vulnerabilities to clickjacking attacks. Based on our results, we identify password managers to avoid, provide recommendations on how to improve existing password managers, and identify areas of future research.
Johnny's Journey Toward Usable Secure Email. Scott Ruoti and Kent Seamons. In IEEE Security & Privacy, Vol. 17, No. 6, pages 72–76, November/December 2019. IEEE, 2019. (S&P)
Abstract: Since the publication of Why Johnny Can't Encrypt there has been interest in creating usable, secure email that is adoptable by the general public. In this article, we summarize research from the usable-security community on this topic, identify open problems, and call for more research on usable key management.
A Usability Study of Four Secure Email Tools Using Paired Participants. Scott Ruoti, Jeff Andersen, Luke Dickinson, Scott Heidbrink, Tyler Monson, Mark O'Neill, Ken Reese, Brad Spendlove, Elham Vaziripour, Justin Wu, Daniel Zappala, and Kent Seamons. In ACM Transactions on Privacy and Security, Vol. 22, No. 2, pages 22–29. ACM, 2019. (TOPS)
Abstract: Secure email is increasingly being touted as usable by novice users, with a push for adoption based on recent concerns about government surveillance. To determine whether secure email is ready for grassroots adoption, we employ a laboratory user study that recruits pairs of novice users to install and use several of the latest systems to exchange secure messages. We present both quantitative and qualitative results from 28 pairs of novices as they use Private WebMail (Pwm), Tutanota, and Virtru and 10 pairs of novices as they use Mailvelope. Participants report being more at ease with this type of study and better able to cope with mistakes since both participants are “on the same page.” We find that users prefer integrated solutions over depot-based solutions and that tutorials are important in helping first-time users. Finally, our results demonstrate that Pretty Good Privacy using manual key management is still unusable for novice users, with 9 of 10 participant pairs failing to complete the study.
A Tale of Two Studies: The Best and Worst of YubiKey Usability. Joshua Reynolds, Trevor Smith, Ken Reese, Luck Dickinson, Scott Ruoti, and Kent Seamons. In Proceedings of the 38th IEEE Symposium on Security and Privacy. IEEE, 2018. (S&P, 11% acceptance rate)
Abstract: Two-factor authentication (2FA) significantly improves the security of password-based authentication. Recently, there has been increased interest in Universal 2nd Factor (U2F) security keys-small hardware devices that require users to press a button on the security key to authenticate. To examine the usability of security keys in non-enterprise usage, we conducted two user studies of the YubiKey, a popular line of U2F security keys. The first study tasked 31 participants with configuring a Windows, Google, and Facebook account to authenticate using a YubiKey. This study revealed problems with setup instructions and workflow including users locking themselves out of their operating system or thinking they had successfully enabled 2FA when they had not. In contrast, the second study had 25 participants use a YubiKey in their daily lives over a period of four weeks, revealing that participants generally enjoyed the experience. Conducting both a laboratory and longitudinal study yielded insights into the usability of security keys that would not have been evident from either study in isolation. Based on our analysis, we recommend standardizing the setup process, enabling verification of success, allowing shared accounts, integrating with operating systems, and preventing lockouts.

Journals and Magazines

SenCAPTCHA: A Mobile-First CAPTCHA Using Orientation Sensors. Yunhe Feng, Qing Cao, Hairong Qi, and Scott Ruoti. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 4, No. 2, June 2020. ACM, 2020. ACM, 2020. (IMWUT/UbiComp)
Abstract: CAPTCHAs are used to distinguish between human- and computer-generated (i.e., bot) online traffic. As there is an ever-increasing amount of online traffic from mobile devices, it is necessary to design CAPTCHAs that work well on mobile devices. In this paper, we present SenCAPTCHA, a mobile-first CAPTCHA that leverages the device's orientation sensors. SenCAPTCHA works by showing users an image of an animal and asking them to tilt their device to guide a red ball into the center of that animal's eye. SenCAPTCHA is especially useful for devices with small screen sizes (e.g., smartphones, smartwatches). In this paper, we describe the design of SenCAPTCHA and demonstrate that it is resilient to various machine learning based attacks. We also report on two usability studies of SenCAPTCHA involving a total of 472 participants; our results show that SenCAPTCHA is viewed as an "enjoyable" CAPTCHA and that it is preferred by over half of the participants to other existing CAPTCHA systems.
Blockchain Technology: What Is It Good For? Scott Ruoti, Ben Kaiser, Arkady Yerukhimovich, Jeremy Clark, and Robert Cunningham. In Communications of the ACM, Vol. 63, No. 1, pages 46–53. ACM, 2020. (CACM)
Abstract: Bitcoin's success has led to significant interest in its underlying components, particularly blockchain technology. Over 10 years after Bitcoin's initial release, the community still suffers from a lack of clarity regarding what properties defines blockchain technology, its relationship to similar technologies, and which of its proposed use-cases are tenable and which are little more than hype. In this paper we answer four common questions regarding blockchain technology: (1) what exactly is blockchain technology, (2) what capabilities does it provide, and (3) what are good applications for blockchain technology, and (4) how does it relate to other distributed technologies (e.g., distributed databases). We accomplish this goal by using grounded theory (a structured approach to gathering and analyzing qualitative data) to thoroughly analyze a large corpus of literature on blockchain technology. This method enables us to answer the above questions while limiting researcher bias, separating thought leadership from peddled hype and identifying open research questions related to blockchain technology. The audience for this paper is broad as it aims to help researchers in a variety of areas come to a better understanding of blockchain technology and identify whether it may be of use in their own research.
Blockchain Technology: What Is It Good For? Scott Ruoti, Ben Kaiser, Arkady Yerukhimovich, Jeremy Clark, and Robert Cunningham. In ACM Queue, Vol. 17, No. 5, pages 60–87. ACM, 2019. (ACM Queue)
Abstract: Bitcoin's success has led to significant interest in its underlying components, particularly blockchain technology. Over 10 years after Bitcoin's initial release, the community still suffers from a lack of clarity regarding what properties defines blockchain technology, its relationship to similar technologies, and which of its proposed use-cases are tenable and which are little more than hype. In this paper we answer four common questions regarding blockchain technology: (1) what exactly is blockchain technology, (2) what capabilities does it provide, and (3) what are good applications for blockchain technology, and (4) how does it relate to other distributed technologies (e.g., distributed databases). We accomplish this goal by using grounded theory (a structured approach to gathering and analyzing qualitative data) to thoroughly analyze a large corpus of literature on blockchain technology. This method enables us to answer the above questions while limiting researcher bias, separating thought leadership from peddled hype and identifying open research questions related to blockchain technology. The audience for this paper is broad as it aims to help researchers in a variety of areas come to a better understanding of blockchain technology and identify whether it may be of use in their own research.
Johnny's Journey Toward Usable Secure Email. Scott Ruoti and Kent Seamons. In IEEE Security & Privacy, Vol. 17, No. 6, pages 72–76, November/December 2019. IEEE, 2019. (S&P)
Abstract: Since the publication of Why Johnny Can't Encrypt there has been interest in creating usable, secure email that is adoptable by the general public. In this article, we summarize research from the usable-security community on this topic, identify open problems, and call for more research on usable key management.
A Usability Study of Four Secure Email Tools Using Paired Participants. Scott Ruoti, Jeff Andersen, Luke Dickinson, Scott Heidbrink, Tyler Monson, Mark O'Neill, Ken Reese, Brad Spendlove, Elham Vaziripour, Justin Wu, Daniel Zappala, and Kent Seamons. In ACM Transactions on Privacy and Security, Vol. 22, No. 2, pages 22–29. ACM, 2019. (TOPS)
Abstract: Secure email is increasingly being touted as usable by novice users, with a push for adoption based on recent concerns about government surveillance. To determine whether secure email is ready for grassroots adoption, we employ a laboratory user study that recruits pairs of novice users to install and use several of the latest systems to exchange secure messages. We present both quantitative and qualitative results from 28 pairs of novices as they use Private WebMail (Pwm), Tutanota, and Virtru and 10 pairs of novices as they use Mailvelope. Participants report being more at ease with this type of study and better able to cope with mistakes since both participants are “on the same page.” We find that users prefer integrated solutions over depot-based solutions and that tutorials are important in helping first-time users. Finally, our results demonstrate that Pretty Good Privacy using manual key management is still unusable for novice users, with 9 of 10 participant pairs failing to complete the study.
TLS Proxies: How Often and Who Cares? Mark O'Neill, Scott Ruoti, Kent Seamons, and Daniel Zappala. In IEEE Internet Computing, Vol. 21, No. 3, pages 22–29, May/June 2017. IEEE, 2017.
Abstract: TLS inspection—inline decryption, inspection, and re-encryption of TLS traffic—is a controversial practice used for both benevolent and malicious purposes. This article describes measurements of how often TLS inspection occurs and reports on a survey of the general public regarding the practice of TLS inspection. This helps inform security researchers and policymakers regarding current practices and user preferences.

Conferences

Understanding User Perceptions of Security and Privacy for Group Chat: A Survey of Users in The US and UK. Sean Oesch, Ruba Abu-Salma, Oumar Diallo, Juliane Krämer, James Simmons, Justin Wu, and Scott Ruoti. In Proceedings of the 36th Annual Computer Security Applications Conference. ACM, 2020. (ACSAC)
Abstract: Secure messaging tools are an integral part of modern society. While there is a significant body of secure messaging research generally, there is a lack of information regarding users' security and privacy perceptions and requirements for secure group chat. To address this gap, we conducted a survey of 996 participants in the US and UK. The results of our study show that group chat presents important security and privacy challenges, some of which are not present in one-to-one chat. For example, users need to be able to manage and monitor group membership, establish trust for new group members, and filter content that they share in different chat contexts. Similarly, we find that the sheer volume of notifications that occur in group chat makes it extremely likely that users ignore important security- or privacy- notifications. We also find that participants lack mechanisms for determining which tools are secure and instead rely on non-technical strategies for protecting their privacy—for example, self-filtering what they post and carefully tracking group membership. Based on these findings we provide recommendations on how to improve the security and usability of secure group chat.
That Was Then, This Is Now: A Security Evaluation of Password Generation, Storage, and Autofill in Browser-Based Password Managers. Sean Oesch and Scott Ruoti. In Proceedings of the 30th USENIX Security Symposium. USENIX, 2020. (USENIX Security, 16% acceptance rate)
Abstract: Password managers have the potential to help users more effectively manage their passwords and address many of the concerns surrounding password-based authentication, however prior research has identified significant vulnerabilities in existing password managers. Since that time, five years has passed, leaving it unclear whether password managers remain vulnerable or whether they are now ready for broad adoption. To answer this question, we evaluate thirteen popular password managers and consider all three stages of the password manager lifecycle—password generation, storage, and autofill. Our evaluation is the first analysis of password generation in password managers, finding several non-random character distributions and identifying instances where generated passwords were vulnerable to online and offline guessing attacks. For password storage and autofill, we replicate past evaluations, demonstrating that while password managers have improved in the half-decade since those prior evaluations, there are still significant issues, particularly with browser-based password managers; these problems include unencrypted metadata, unsafe defaults, and vulnerabilities to clickjacking attacks. Based on our results, we identify password managers to avoid, provide recommendations on how to improve existing password managers, and identify areas of future research.
A Comparative Usability Study of Key Management in Secure Email. Scott Ruoti, Jeff Andersen, Tyler Monson, Daniel Zappala, and Kent Seamons. In Proceedings of the 14th Symposium on Usable Privacy and Security. USENIX, 2018. (SOUPS, 23% acceptance rate)
Abstract: We conducted a user study that compares three secure email tools that share a common user interface and differ only by key management scheme: passwords, public key directory (PKD), and identity-based encryption (IBE). Our work is the first comparative (i.e., A/B) usability evaluation of three different key management schemes and utilizes a standard quantitative metric for cross-system comparisons. We also share qualitative feedback from participants that provides valuable insights into user attitudes regarding each key management approach and secure email generally. The study serves as a model for future secure email research with A/B studies, standard metrics, and the two-person study methodology.
A Tale of Two Studies: The Best and Worst of YubiKey Usability. Joshua Reynolds, Trevor Smith, Ken Reese, Luck Dickinson, Scott Ruoti, and Kent Seamons. In Proceedings of the 38th IEEE Symposium on Security and Privacy. IEEE, 2018. (S&P, 11% acceptance rate)
Abstract: Two-factor authentication (2FA) significantly improves the security of password-based authentication. Recently, there has been increased interest in Universal 2nd Factor (U2F) security keys-small hardware devices that require users to press a button on the security key to authenticate. To examine the usability of security keys in non-enterprise usage, we conducted two user studies of the YubiKey, a popular line of U2F security keys. The first study tasked 31 participants with configuring a Windows, Google, and Facebook account to authenticate using a YubiKey. This study revealed problems with setup instructions and workflow including users locking themselves out of their operating system or thinking they had successfully enabled 2FA when they had not. In contrast, the second study had 25 participants use a YubiKey in their daily lives over a period of four weeks, revealing that participants generally enjoyed the experience. Conducting both a laboratory and longitudinal study yielded insights into the usability of security keys that would not have been evident from either study in isolation. Based on our analysis, we recommend standardizing the setup process, enabling verification of success, allowing shared accounts, integrating with operating systems, and preventing lockouts.
Intrusion Detection with Unsupervised Heterogeneous Ensembles Using Cluster-Based Normalization. Scott Ruoti, Scott Heidbrink, Mark O'Neill, Eric Gustafson, and Yung Ryn Choe. In Proceedings of the 24th IEEE International Conference on Web Services. IEEE, 2017. (ICWS, 22% acceptance rate)
Abstract: Outlier detection has been shown to be a promising machine learning technique for a diverse array of fields and problem areas. However, traditional, supervised outlier detection is not well suited for problems such as network intrusion detection, where proper labelled data is scarce. This has created a focus on extending these approaches to be unsupervised, removing the need for explicit labels, but at a cost of poorer performance compared to their supervised counterparts. Recent work has explored ways of making up for this, such as creating ensembles of diverse models, or even diverse learning algorithms, to jointly classify data. While using unsupervised, heterogeneous ensembles of learning algorithms has been proposed as a viable next step for research, the implications of how these ensembles are built and used has not been explored.
PACE: Proactively-Secure Accumulo with Cryptographic Enforcement. Scott Ruoti, Ben Kaiser, Ariel Hamlin, Cassandra Sparks, and Robert Cunningham. In Proceedings of the 21st IEEE High Performance Extreme Computing Conference. IEEE, 2017. (HPEC)
Abstract: Cloud-hosted databases have many compelling benefits, including high availability, flexible resource allocation, and resiliency to attack, but it requires that cloud tenants cede control of their data to the cloud provider. In this paper, we describe Proactively-secure Accumulo with Cryptographic Enforcement (PACE), a client-side library that cryptographically protects a tenant’s data, returning control of that data to the tenant. PACE is a drop-in replacement for Accumulo’s APIs and works with Accumulo’s row-level security model. We evaluate the performance of PACE, discussing the impact of encryption and signatures on operation throughput.
TrustBase: An Architecture to Repair and Strengthen Certificate-Based Authentication. Mark O'Neill, Scott Heidbrink, Scott Ruoti, Jordan Whitehead, Dan Bunker, Luke Dickinson, Travis Hendershot, Joshua Reynolds, Kent Seamons, and Daniel Zappala. In Proceedings of the 27th USENIX Security Symposium. USENIX, 2017. (USENIX Security, 16% acceptance rate)
Abstract: The current state of certificate-based authentication is messy, with broken authentication in applications and proxies, along with serious flaws in the CA system. To solve these problems, we design TrustBase, an architecture that provides certificate-based authentication as an operating system service, with system administrator control over authentication policy. TrustBase transparently enforces best practices for certificate validation on all applications, while also providing a variety of authentication services to strengthen the CA system. We describe a research prototype of TrustBase for Linux, which uses a loadable kernel module to intercept traffic in the socket layer, then consults a user-space policy engine to evaluate certificate validity using a variety of plugins. We evaluate the security of TrustBase, including a threat analysis, application coverage, and hardening of the Linux prototype. We also describe prototypes of TrustBase for Android and Windows, illustrating the generality of our approach. We show that TrustBase has negligible overhead and universal compatibility with applications. We demonstrate its utility by describing eight authentication services that extend CA hardening to all applications.
Layering Security at Global Control Points to Secure Unmodified Software. Scott Ruoti, Kent Seamons, and Daniel Zappala. In Proceedings of the 2nd IEEE Secure Development Conference. IEEE, 2017. Best Paper Award. (SecDev, 32% acceptance rate)
Abstract: Developing secure software is inherently difficult, and is further hampered by a rush to market, the lack of cybersecurity-trained architects and developers, and the difficulty of identifying flaws and deploying mitigations. To address these problems, we advocate for an alternative paradigm-layering security onto applications from global control points, such as the browser, operating system, or network. This approach adds security to existing applications, relieving developers of this burden. The benefits of this paradigm are three-fold-(1) increased correctness in the implementation of security features, (2) coverage for all software, even non-maintained legacy software, and (3) more rapid and consistent deployment of threat mitigations and new security features. To demonstrate these benefits, we describe three concrete instantiations of this paradigm- MessageGuard, a system that layers end-to-end encryption in the browser; TrustBase, a system that layers authentication in the operating system; and software-defined perimeter, which layers access control at network middleboxes.
Weighing Context and Tradeoffs: How Suburban Adults Selected Their Online Security Posture. Scott Ruoti, Tyler Monson, Justin Wu, Kent Seamons, and Daniel Zappala. In Proceedings of the 13th Symposium on Usable Privacy and Security. USENIX, 2017. (SOUPS, 27% acceptance rate)
Abstract: Understanding how people behave when faced with complex security situations is essential to designing usable security tools. To better understand users' perceptions of their digital lives and how they managed their online security posture, we conducted a series of 23 semi-structured interviews with mostly middle-aged parents from suburban Washington state. Using a grounded theory methodology, we analyzed the interview data and found that participants chose their security posture based on the immense value the Internet provides and their belief that no combination of technology could make them perfectly safe. Within this context, users have a four-stage process for determining which security measures to adopt: learning, evaluation of risks, estimation of impact, and weighing trade-offs to various coping strategies. Our results also revealed that a majority of participants understand the basic principles of symmetric encryption. We found that participants' misconceptions related to browser-based TLS indicators lead to insecure behavior, and it is the permanence of encrypted email that causes participants to doubt that it is secure. We conclude with a discussion of possible responses to this research and avenues for future research.
TLS Proxies: Friend or Foe? Mark O'Neill, Scott Ruoti, Kent Seamons, and Daniel Zappala. In Proceedings of the 17th ACM Internet Measurement Conference. ACM, 2016. (IMC, 25% acceptance rate)
Abstract: We measure the prevalence and uses of TLS proxies using a Flash tool deployed with a Google AdWords campaign. We generate 2.9 million certificate tests and find that 1 in 250 TLS connections are TLS-proxied. The majority of these proxies appear to be benevolent, however we identify over 1,000 cases where three malware products are using this technology nefariously. We also find numerous instances of negligent, duplicitous, and suspicious behavior, some of which degrade security for users without their knowledge. Distinguishing these types of practices is challenging in practice, indicating a need for transparency and user awareness.
Private Webmail 2.0: Simple and Easy-to-Use Secure Email. Scott Ruoti, Jeff Andersen, Travis Hendershot, Kent Seamons, and Daniel Zappala. In Proceedings of the 29th ACM Symposium on User Interface Software and Technology. ACM, 2016. (UIST, 20% acceptance rate)
Abstract: Private Webmail 2.0 (Pwm 2.0) improves upon the current state of the art by increasing the usability and practical security of secure email for ordinary users. More users are able to send and receive encrypted emails without mistakenly revealing sensitive information. In this paper we describe four user interface traits that positively affect the usability and security of Pwm 2.0. In a user study involving 51 participants we validate that these interface modifications result in high usability, few mistakes, and a strong understanding of the protection provided to secure email messages. We also show that the use of manual encryption has no effect on usability or security.
User Attitudes Toward The Inspection of Encrypted Traffic. Scott Ruoti, Mark O'Neill, Kent Seamons, and Daniel Zappala. In Proceedings of the 12th Symposium on Usable Privacy and Security. USENIX, 2016. (SOUPS, 28% acceptance rate)
Abstract: This paper reports the results of a survey of 1,976 individuals regarding their opinions on TLS inspection, a controversial technique that can be used for both benevolent and malicious purposes. Responses indicate that participants hold nuanced opinions on security and privacy trade-offs, with most recognizing legitimate uses for the practice, but also concerned about threats from hackers or government surveillance. There is strong support for notification and consent when a system is intercepting their encrypted traffic, although this support varies depending on the situation. A significant concern about malicious uses of TLS inspection is identity theft, and many would react negatively and some would change their behavior if they discovered inspection occurring without their knowledge. We also find that a small but significant number of participants are jaded by the current state of affairs and have lost any expectation of privacy.
"We're on the same page": A Usability Study of Secure Email Using Pairs of Novice Users. Scott Ruoti, Jeff Andersen, Scott Heidbrink, Mark O'Neill, Elham Vaziripour, Justin Wu, Daniel Zappala, and Kent Seamons. In Proceedings of the 34th ACM Conference on Human Factors in Computing Systems. ACM, 2016. Honorable Mention for Best Paper. (CHI, 23% acceptance rate)
Abstract: Secure email is increasingly being touted as usable by novice users, with a push for adoption based on recent concerns about government surveillance. To determine whether secure email is ready for grassroots adoption, we employ a laboratory user study that recruits pairs of novice users to install and use several of the latest systems to exchange secure messages. We present both quantitative and qualitative results from 25 pairs of novice users as they use Pwm, Tutanota, and Virtru. Participants report being more at ease with this type of study and better able to cope with mistakes since both participants are "on the same page". We find that users prefer integrated solutions over depot-based solutions, and that tutorials are important in helping first-time users. Hiding the details of how a secure email system provides security can lead to a lack of trust in the system. Participants expressed a desire to use secure email, but few wanted to use it regularly and most were unsure of when they might use it.
Authentication Melee: A Usability Analysis of Seven Web Authentication Systems. Scott Ruoti, Brent Roberts, and Kent Seamons. In Proceedings of the 24th International Conference on World Wide Web. Internet Society, 2015. (WWW, 14% acceptance rate)
Abstract: Passwords continue to dominate the authentication landscape in spite of numerous proposals to replace them. Even though usability is a key factor in replacing passwords, very few alternatives have been subjected to formal usability studies, and even fewer have been analyzed using a standard metric. We report the results of four within-subjects usability studies for seven web authentication systems. These systems span federated, smartphone, paper tokens, and email-based approaches. Our results indicate that participants prefer single sign-on systems. We report several insightful findings based on participants' qualitative responses: (1) transparency increases usability but also leads to confusion and a lack of trust, (2) participants prefer single sign-on but wish to augment it with site-specific low-entropy passwords, and (3) participants are intrigued by biometrics and phone-based authentication. We utilize the Systems Usability Scale (SUS) as a standard metric for empirical analysis and find that it produces reliable, replicable results. SUS proves to be an accurate measure of baseline usability. We recommend that new authentication systems be formally evaluated for usability using SUS, and should meet a minimum acceptable SUS score before receiving serious consideration.
Confused Johnny: When Automatic Encryption Leads to Confusion and Mistakes. Scott Ruoti, Nathan Kim, Ben Burgon, Timothy W. van der Horst, and Kent Seamons. In Proceedings of the 9th Symposium on Usable Privacy and Security. ACM, 2013. (SOUPS, 29% acceptance rate)
Abstract: A common approach to designing usable security is to hide as many security details as possible from the user to reduce the amount of information and actions a user must encounter. This paper gives an overview of Pwm (Private Webmail), our secure webmail system that uses security overlays to integrate tightly with existing webmail services like Gmail. Pwm's security is mostly transparent, including automatic key management and automatic encryption. We describe a series of Pwm user studies indicating that while nearly all users can use the system without any prior training, the security details are so transparent that a small percentage of users mistakenly sent out unencrypted messages and some users are unsure whether they should trust Pwm. We then conducted user studies with an alternative prototype to Pwm that uses manual encryption. Surprisingly users were accepting of the extra steps of cutting and pasting ciphertext themselves. They avoided mistakes and had more trust in the system with manual encryption. Our results suggest that designers may want to reconsider manual encryption as a way to reduce transparency and foster greater trust.
Private Facebook Chat. Chris Robison, Scott Ruoti, Timothy W. van der Horst, and Kent Seamons. In Proceedings of the 2012 International Conference on Privacy, Security, Risk, and Trust and 2012 International Conference on Social Computing. IEEE, 2012. (PASSAT/SocialCom 2012, 22% acceptance rate)
Abstract: The number of instant messages sent per year now exceeds that of email. Recently users have been moving away from traditional instant messaging applications and instead using social networks as their primary communications platform. To discover attitudes related to instant messaging and its security, we have conducted a user survey. This paper also presents the design of PFC (Private Facebook Chat), a system providing convenient, secure instant messaging within Facebook Chat. PFC offers end-to-end encryption in order to thwart any eavesdropper, including Facebook itself. Finally, we have conducted a usability study of a PFC prototype.

Workshops

A Usability Study of Secure Email Deletion. Tyler Monson, Joshua Reynolds, Trevor Smith, Scott Ruoti, Daniel Zappala, and Kent Seamons. In Proceedings of the 3rd European Workshop on Usable Security. Internet Society, 2018. (EuroUSec)
Abstract: Messaging applications like SnapChat illustrate that users are concerned about the permanence of information. We find that this concern extends to email. In this paper we present a usability study of an end-to-end secure email tool with the option to securely delete messages. This tool uses ephemeral keys, one per message thread, and default expiration times, with a user prompt to renew or delete keys. Deleting keys causes the messages in the thread to be unreadable for that user. We compare the usability of this tool to a nearly identical tool that uses long term keys and lacks a feature to expire keys. We also interview participants about their email use patterns and attitudes towards information permanence. We find that participants are especially interested in the ability to control the lifetime of an email message. Participants also report trusting the tool that allowed them to make their email messages ephemeral more than the tool that just encrypted their email.
End-to-End Passwords. Scott Ruoti and Kent Seamons. In Proceedings of the 20th New Security Paradigms Workshop. ACM, 2017. (NSPW)
Abstract: Passwords continue to be an important means for users to authenticate themselves to applications, websites, and backend services. However, password theft continues to be a significant issue, due in large part to the significant attack surface for passwords, including the operating system (e.g., key loggers), application (e.g., phishing websites in browsers), during transmission (e.g., TLS man-in-the-middle proxies), and at password verification services (e.g., theft of passwords stored at a server). Relatedly, even though there is a large body of research on improving passwords, the massive number of application verification services that use passwords stymie the diffusion of improvements—i.e., it does not scale for each improvement to require an update to every application and verification service. To address these problems, we propose a new end-to-end password paradigm that transfers password functionality to two end-points, the operating system (entry, management, storage, and verification) and the password verification service (verification, and verification token storage). In this paradigm, passwords are never shared with applications or transmitted over the network, but are instead verified using zero-knowledge protocols. There are five key benefits of this approach that are not possible with the current password paradigm: (a) a minimal attack surface, (b) protection from password phishing, (c) protection from malware, (d) consistent password policies, and (e) the ability to more rapidly diffuse improvements from password research.
Augmenting Centralized Password Management with Application-Specific Passwords. Trevor Smith, Scott Ruoti, and Kent Seamons. In Proceedings of the 3rd Workshop on Who Are You?! Adventures in Authentication. USENIX, 2017. (WAY)
Abstract: Password authentication is the most prevalent form of authentication; however, passwords have numerous usability issues. For example, due to the large number and high complexity required of passwords, users frequently reuse and choose weak passwords. One way to address these problems is to centralize password management by using a password manager or single sign-on. While this centralizing approach can improve a user's security, it also magnifies the damage caused by a compromise of the user's master password. In this paper, we describe a new approach to enhance centralized password management using application-specific passwords. This approach prevents the compromise of a master password from immediately compromising all associated applications and instead, requires the attacker to conduct further online attacks against individual applications. We detail five possible system designs for application-specific passwords and describe our plans for user studies to test the acceptance and usability of this approach.
Content-Based Security for The Web. Alexander Afanasyev, J. Alex Halderman, Scott Ruoti, Kent Seamons, Yingdi Yu, Daniel Zappala, and Lixia Zhang. In Proceedings of the 19th New Security Paradigms Workshop. ACM, 2016. (NSPW)
Abstract: The World Wide Web has become the most common platform for building applications and delivering content. Yet despite years of research, the web continues to face severe security challenges related to data integrity and confidentiality. Rather than continuing the exploit-and-patch cycle, we propose addressing these challenges at an architectural level, by supplementing the web's existing connection-based and server-based security models with a new approach: content-based security. With this approach, content is directly signed and encrypted at rest, enabling it to be delivered via any path and then validated by the browser. We explore how this new architectural approach can be applied to the web and analyze its security benefits. We then discuss a broad research agenda to realize this vision and the challenges that must be overcome.
Standard Metrics and Scenarios for Usable Authentication. Scott Ruoti and Kent Seamons. In Proceedings of the 2nd Workshop on Who Are You?! Adventures in Authentication. USENIX, 2016. (WAY)
Abstract: There is a constant flow of new authentication schemes proposed in the literature. In the past, most proposed schemes were not evaluated empirically, though in recent years there has been an increase in the number of authentication systems that have undergone a user study. Still, most of these user studies employ ad-hoc metrics (e.g., task completion time) and a unique scenario. Bonneau et al. included usability criteria in their heuristic evaluation of various types of web authentication mechanisms.…
Strengthening Passwords-Based Authentication. Scott Ruoti, Jeff Andersen, and Kent Seamons. In Proceedings of the 2nd Workshop on Who Are You?! Adventures in Authentication. USENIX, 2016. (WAY)
Abstract: Even with years of research into new authentication technologies, passwords still dominate the authentication landscape. This is due primarily to a combination of security, deployability, and usability that has been difficult to match. While password alternatives exist, their lack of widespread adoption indicates that for the foreseeable future passwords are here to stay.…

Preprints

SoK: Blockchain Technology and Its Potential Use Cases. Scott Ruoti, Ben Kaiser, Arkady Yerukhimovich, Jeremy Clark, and Robert Cunningham. In arXiv, arXiv:1909.12454. 2019.
Abstract: Bitcoin's success has led to significant interest in its underlying components, particularly blockchain technology. Over 10 years after Bitcoin's initial release, the community still suffers from a lack of clarity regarding what properties defines blockchain technology, its relationship to similar technologies, and which of its proposed use-cases are tenable and which are little more than hype. In this paper we answer four common questions regarding blockchain technology: (1) what exactly is blockchain technology, (2) what capabilities does it provide, and (3) what are good applications for blockchain technology, and (4) how does it relate to other distributed technologies (e.g., distributed databases). We accomplish this goal by using grounded theory (a structured approach to gathering and analyzing qualitative data) to thoroughly analyze a large corpus of literature on blockchain technology. This method enables us to answer the above questions while limiting researcher bias, separating thought leadership from peddled hype and identifying open research questions related to blockchain technology. The audience for this paper is broad as it aims to help researchers in a variety of areas come to a better understanding of blockchain technology and identify whether it may be of use in their own research.
SoK: Securing Email—A Stakeholder-Based Analysis. Jeremy Clark, P.C. van Oorschot, Scott Ruoti, Kent Seamons, and Daniel Zappala. In arXiv, arXiv:1804.07706. 2019.
Abstract: While email is the most ubiquitous and interoperable form of online communication today, it was not conceived with strong security guarantees, and the ensuing security enhancements are, by contrast, lacking in both ubiquity and interoperability. This situation motivates our research. We begin by identifying a variety of stakeholders who have an interest in the current email system and in efforts to provide secure solutions. We then use the tussle among stakeholders to explain the evolution of fragmented secure email solutions undertaken by industry, academia, and independent developers. We develop an evaluation framework for proposed or deployed secure email systems and identify how well they meet properties related to security, utility, deployability, and usability. We conclude with a fresh look at the state of secure email and discuss open problems in the area.