Enablers

Several concepts and functionality are key enablers for pervasive computing. One example is indoor and outdoor location tracking and positioning. Another is real-world modeling and mapping to physical spaces—what we call self-sensing spaces. Design tools are also essential enablers. For instance, simulating human interactions in a smart space could provide for virtual and early prototyping and could save significant time and avoid costly mistakes for a given, planned real deployment. The Lab is engaged in the pervasive computing enabler projects listed below under “Active Projects”.

Active Projects:

• Self-Sensing Spaces
• Location Tracking/Positioning Systems in Pervasive Spaces
• PerSim: A Simulator of Human Activities in Pervasive Spaces
• Context-Driven Simulation of Pervasive Spaces

Self-Sensing Spaces

Goal

In this project we explore (1) emerging sensor network technology, (2) real-world modeling techniques, (3) Computer Vision, and (4) self-organizing system principles, to realize the vision of self-sensing spaces. Our vision is to create new capabilities in which smart spaces, such as homes, sense themselves and their residents; create their respective real-world models; and enact an accurate mapping between the real world model elements and the physical living space. If realized, this vision will enable many applications that rely on remote monitoring/intervention, such as for the elderly and disabled.

Objectives

Our objective is to create a technology and systems that allow spaces to automatically detect their main aspects (e.g., floor plans, type of rooms, existence of doors, windows, power outlets, etc.), and for objects (e.g., furniture pieces, appliances, etc.) to also be identified automatically. This enables the on-the-fly and incremental creation and update of real-world models of the smart space. Identification and self-sensing both encompass sensing of the software components needed to interact, within the model, with the model elements. This renders an automatically created, interactive real-world model of the physical space. The model can be used in many applications, most notably in remote monitoring and intervention in the case of older people and those with disabilities. In this application, intervention occurs by activating actuators, which are mapped in the real-world model to actual objects in an elderly person’s home. Intervention is limited to objects equipped with actuators. For instance, it is possible to monitor a chair and its exact and current location in a house, but it will not be possible to remotely move the chair from one place to another. It will be possible, however, to monitor the location and status (on/off) of a set of stove burners, as well as to intervene to alter their status.

People

Dr. Sumi Helal
Hicham Zabadani
Dr. Mark Schmaltz
Jung Wook Park

Publications

• J. Park, C. Chen, H Elzabadani and A. Helal, "SmartPlug : Creating Self-Sensing SPaces using the Atlas Middleware," A demonstration and a short paper in the supplementary proceedings of the 11th ACM International Conference on Ubiquitous Computing (Ubicomp), Orlando, FLorida, 2009 (pdf).

• H. El-Zabadani, A. Helal, W. Mann and M. Schmaltz, "PerVision: An integrated Pervasive Computing/Computer Vision Approach to Tracking Objects in a Self-Sensing Space," the 4th IEEE International Conference on Pervasive Computing and Communications (PerCom), Pisa, Italy, 2006. (pdf).

• H. El-Zabadani, A. Helal, B. Abudlrazak, and Erwin Jansen, “Self-Sensing Spaces: Smart Plugs for Smart Environments,” Proceedings of the third International Conference On Smart homes and health Telematic (ICOST), Sherbrooke, Québec, Canada, July 2005. (pdf)
  

Location Tracking/Positioning Systems in Pervasive Spaces

Goal

Our goal is to investigate location tracking technologies and to create passive and active sensors that would enable application-level development of location-based services in a pervasive space. The goal is to achieve a high precision location and tracking of presence, position, and orientation. Another goal of the project is to achieve cost-effectiveness, reliability, and to isolate the specific sensor technology used (or any newer sensor technology replacement) from the application and the application development process.

Objectives

Our objective is to experiment with several sensing technologies including ultrasonic, force (pressure) sensors, and MEMS accelerometers, and to develop system prototypes to evaluate their performance. We explore the design requirements and the trade-offs in designing unencumbered location tracking and positioning systems. We use the flooring as a platform for sensing location and to create the smart floor concept and service to the smart home. To better understand the performance model of such systems, we formally model smart floors capturing sensor grid topology, sensor characteristics, foot shape, walking models, and overall cost factors. We will analyze the formal model using the Monte Carlo simulation to answer accuracy/cost trade-off questions and design predictive algorithms to enable smart floor-based location tracking systems to function fairly accurately despite multiple failures of sensor nodes. We have implemented and evaluated the smart floor concept in the Gator Tech Smart House using force sensors, which requires heavy installation overhead (all sensors must be wired and connected). We are currently designing a wireless, on-walls-only, MEMS-based vibration sensor which requires much less installation overhead and which offers more capabilities, such as user identification and ability to estimate caloric expenditure based on signal identification and analysis.

People

Dr. Sumi Helal
Youssef Kaddoura
Ahmad El-Kouch
Bryan Winkler
Dr. Heyoung Lee
Raja Bose
Jung Wook Park

Publications

• H. Lee, J. Park and A. Helal, “Estimation of Indoor Physical Activity Level Based on Footstep Vibration Signal Measured by MEMS Accelerometer for Personal Health Care Under Smart Home Environments,” in Proceedings of the Second International Workshop on Mobile Entity Localization and Tracking in GPS-less Environments (MELT) held in conjunction with Ubicomp 2009, Orlando, Florida. (pdf)

• R. Bose, A. Helal, "Observing Walking Behavior of Humans using Distributed Phenomenon Detection and Tracking Mechanisms," Proceedings of 2nd International Workshop on Practical Applications of Sensor Networks, held in conjunction with the International Symposium on Applications and the Internet (SAINT), Turku (Finland), July 2008. (pdf)

• Y. Kaddourah, J. King and A. Helal, “Cost-Precision Tradeoffs in Unencumbered Floor-Based Indoor Location Tracking,” Proceedings of the third International Conference On Smart homes and health Telematic (ICOST), Sherbrooke, Québec, Canada, July 2005. (pdf)

• S. Helal, B. Winkler, C. Lee, Y. Kaddourah, L. Ran, C. Giraldo and W. Mann, "Enabling Location-Aware Pervasive Computing Applications for the Elderly" Proceedings of the First IEEE Pervasive Computing Conference, June 2003 in Fort Worth, Texas. (pdf)

PerSim: A Simulator of Human Activities in Pervasive Spaces

Goal

Activity recognition research relies heavily on test data to verify the modeling technique and the performance of the activity recognition algorithm. But data from real deployments are scarce because they are expensive and time consuming to produce and collect. And even if cost is not an issue, regulatory limitations on the use of human subjects prohibit the collection of extensive datasets that can test all scenarios, under all circumstances. A powerful and verifiable simulation tool is needed to accelerate research on human activity recognition. The Mobile and Pervasive Computing Lab has developed an open source project we call PerSim (for pervasive simulation), an event driven simulator of human activities in pervasive spaces. PerSim is capable of capturing elements of space, sensors, behaviors (activities), and their inter-relationships. PerSim allows for verifiable synthesis of datasets. It encourages and facilitates reuse and extension of existing, non-PerSim, as well as PerSim generated data.

Objectives

PerSim is a tool set whose objective is to empower researchers to generate needed datasets without having to have the luxury of an expensive smart house or other smart spaces. Converting datasets (any datasets) into the Lab’s own SDDL standard format is one the basic features of PerSim. Creating a PerSim simulation project is the main feature of PerSim. This involves graphically creating a smart space, instrumenting the desired sensors, and injecting activities in that space. A simulation project is typically created over a period of time via multiple sessions. Once a simulation project is completed, generating a synthesized dataset requires only a click of a button. To facilitate sharing of the generated datasets, PerSim uses a community resource repository hosted at Google. PerSim offers its users another powerful feature, which we call pervasive space “stem-celling”. By accessing and extending an existing dataset, a partial version of the PerSim simulation project that represents the space/activities of the modified dataset can be properly constructed. The user is then able to complete the automatically constructed partial project to generate the corresponding dataset. Validation of a generated dataset against a real or a reference dataset can be done in one of two methods, as we demonstrated and published. Such validation is not yet integrated into the PerSim tool set, though this is the plan moving forward on this project.

The current version of PerSim (1.0) is effective for simple spaces and activities. Higher realism of the space and of the sensor models are required for more complex interactions, activities, and spaces. We are currently working on PerSim 3D, which is a higher realism version of PerSim that exploits smart autonomous characters and is based on the Unity 3G rendering engine. PerSim 3D is based on a novel simulation technique that we are currently developing—the context-driven simulation technique to be contrasted to the classic, event-driven simulation method.

People

Dr. Sumi Helal
Dr. Hani Hagras
Shantonu Hossain
Jae Woong Lee
Eunju Kim

PerSim Webpage

opensource/persim.pdf

Publications

• J. W. Lee, S. Helal, Y. Sung, and K. Cho, “A Context-driven Approach to Scalable Human Activity Simulation? Accepted in ACM SIGSIM Conference on Principles of Advanced Discrete Simulation (PADS), May 19-22, 2013, Montreal, Canada.

• A. Helal, K. Cho, W. Lee, Y. Sung, JW Lee, and E. Kim, "3D Modeling and Simulation of Human Activities in Smart Spaces", In Proceedings of the 9th International Conference on Ubiquitous Intelligence and Computing (UIC), Sep 4-7 2012, Fukuoka, Japan. (pdf)

• E. Kim, A. Helal, C. Nugent, and J.W. Lee, "Assurance-Oriented Activity Recognition", In Proceedings of the International Workshop on Situation, Activity and Goal Awareness (SAGAware), in conjunction with ACM UbiComp Conference, Sep 17-21, 2011, Beijing, China. (pdf)

• A. Helal, J.W. Lee, S. Hossain, E. Kim, H. Hagras, D. cook, "Persim- Simulator for Human Activities in Pervasive Spaces," Proceedings of the 7th International Conference on Intelligent Environments (IE 11), Jul 25-28, 2011, Nottingham, United Kindom. (pdf)

• E. Kim, A. Helal, J.W. Lee, H. Shantonu, "The Making of a Dataset for Smart Spaces," Proceedings of the 7th International Conference on Ubiquitous Intelligence and computing (UIC), Oct 28-29, 2010, Xian, China. (pdf)

• A. Elfaham, H. Hagras, S. Helal, H. Shantonu, J.W. Lee, D. Cook, "A Fuzzy Based Verification Agent for the PerSim Human Activity Simulator in Ambient Intelligent Environments," Proceedings of the IEEE World Congress on Computational Intelligence(WCCI), Jul 18-23, 2010, Barcelona, Spain. (pdf)

• S. Hossain, A. Helal, JW Lee and H. Hagras, "PerSim: A Simulator of Human Activities in Pervasive Spaces," Supplementary Proceedings of Pervasive 2010 Poster Program, May 17-20, 2010, Helsinki, Finland. (pdf)

Context-Driven Simulation of Pervasive Spaces

Goal

The goal of this project is to take PerSim into higher levels of fidelity and capability in capturing and modeling elaborate physical spaces and complex activities. A main consequence and concern of high realism is scalability of the simulation. Event-driven techniques will not scale for what we wish to accomplish. Hence, we are looking at a new simulation methodology that smartly sifts through the most important combination of events, which are basically contexts, and drive the simulation loop on this basis. This work is underway and more details will be posted as we reach key findings.

People

Dr. Sumi Helal
Dr. Kyungeun Cho, Dongguk University, S. Korea
Dr. Yunsick Sung, Dongguk University, S. Korea
Jae Woong Lee
Wonsick Lee
Daxing Jin

Publications

• J. W. Lee, S. Helal, Y. Sung, and K. Cho, “A Context-driven Approach to Scalable Human Activity Simulation? Accepted in ACM SIGSIM Conference on Principles of Advanced Discrete Simulation (PADS), May, 19-22, 2013, Montreal, Canada.