Practical and Rich User Digitization

Karan Ahuja

2024-03-01

Abstract:A long-standing vision in computer science has been to evolve computing devices into proactive assistants that enhance our productivity, health and wellness, and many other facets of our lives. User digitization is crucial in achieving this vision as it allows computers to intimately understand their users, capturing activity, pose, routine, and behavior. Today's consumer devices - like smartphones and smartwatches provide a glimpse of this potential, offering coarse digital representations of users with metrics such as step count, heart rate, and a handful of human activities like running and biking. Even these very low-dimensional representations are already bringing value to millions of people's lives, but there is significant potential for improvement. On the other end, professional, high-fidelity comprehensive user digitization systems exist. For example, motion capture suits and multi-camera rigs that digitize our full body and appearance, and scanning machines such as MRI capture our detailed anatomy. However, these carry significant user practicality burdens, such as financial, privacy, ergonomic, aesthetic, and instrumentation considerations, that preclude consumer use. In general, the higher the fidelity of capture, the lower the user's practicality. Most conventional approaches strike a balance between user practicality and digitization fidelity.

Human-Computer Interaction,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper mainly discusses the issue of user digitization, aiming to enhance the richness of user digitization while maintaining or enhancing its utility. User digitization refers to creating personal digital representations, including various aspects of identity, attributes, behavior, and interaction. Current consumer devices, such as smartphones and smartwatches, provide low-dimensional user representations such as step counting and heart rate monitoring. However, high-fidelity comprehensive user digitization systems, such as motion capture and multi-camera devices, have high accuracy but poor utility due to issues such as cost, privacy, ergonomics, and aesthetics.

The goal of researcher Karan Ahuja is to break this trade-off and develop a sensing system that can increase the accuracy of user digitization while improving its utility and accessibility. This will enable future devices to achieve long-term health tracking, improved work efficiency, full-body virtual reality, and remote telepresence experiences. The research also focuses on privacy-aware user digitization, protecting user privacy through passive and long-term sensing technologies, and proposes a system called IMUPoser, which utilizes various mobile devices carried by users (such as smartphones, smartwatches, and headphones) to estimate full-body poses.

The paper is divided into two parts. The first part focuses on improving the richness of digitization while maintaining utility and investigates the technologies of activity recognition and full-body pose capture. The second part focuses on maintaining richness while improving user utility, with a particular emphasis on privacy protection and continuous perception. Through this work, the paper aims to drive the development of user digitization technology, providing more detailed personal data without sacrificing user convenience.