PerFail 2022

First International Workshop on Negative Results in Pervasive Computing

March 25, 2022, Pisa, Italy Virtual

Co-located with IEEE PerCom 2022

“Learn from the mistakes of others. You can’t live long enough to make them all yourself.” - Eleanor Roosevelt


Not all research leads to fruitful results, trying new ways or methods may surpass the state of the art, but sometimes the hypothesis is not proven or the improvement is insignificant. But failure to succeed is not failure to progress and this workshop aims to create a platform for sharing insights, experiences, and lessons learned when conducting research in the area of pervasive computing.

While the direct outcome of negative results might not contribute much to the field, the wisdom of hindsight could be a contribution itself, such that other researchers could avoid falling into similar pitfalls. We consider negative results to be studies that are run correctly (in the light of the current state of the art) and in good practice, but fail in terms of proving of the hypothesis or come up with no significance. The “badness” of the work can also come out as a properly but unfittingly designed data collection, or (non-trivial) lapses of hindsight especially in measurement studies.


The papers of this workshop should highlight lessons learned from the negative results. The main outcome of the workshop is to share experiences so that others avoid the pitfalls that the community generally overlooks in the final accepted publications. All areas of pervasive computing, networking and systems research are considered. While we take a very broad view of “negative results”, submissions based on opinions and non-fundamental circumstances (e.g. coding errors and “bugs”) are not in scope of the workshop as they do not indicate if the approach (or hypothesis) was bad.

The main topics of interests include (but are not limited to):

  1. Studies with unconvincing results which could not be verified (e.g. due to lack of datasets)
  2. Underperforming experiments due to oversights in system design, inadequate/misconfigured infrastructure, etc.
  3. Research studies with setbacks resulting in lessons learnt and acquired hindsights (e.g. hypothesis with too limiting or too broad assumptions)
  4. Unconventional, abnormal, or controversial results that contradict expectations of the community
  5. Unexpected problems affecting publications, e.g. ethical concerns, institutional policy breaches, etc.
  6. “Non-publishable” or “hard-to-publish” side-outcomes of the study, e.g . mis-trials of experiment methodology/design, preparations for proof-of-correctness of results, etc.

We also welcome submissions from experienced researchers that recounts post-mortem of experiments or research directions they have failed in the past (e.g. in a story-based format). With this workshop, our aim is to normalize the negative outcomes and inherent failures while conducting research in pervasive computing, systems and networking, and provide a complementary view to all the success stories in these fields.

Important Dates

Paper submission: November 14, 2021 November 28, 2021
Author notification: January 5, 2022
Camera-ready due: February 5, 2022
Workshop date: March 25, 2022


Regular papers should present novel perspectives within the scope of the workshop: negative results, lessons learned, and other fruitful “failure” stories. Papers must be in PDF format and contain 6 pages maximum (including references), but also shorter submissions are welcome. Papers should contain names and affiliations of the authors (not blinded). All papers must be typeset in double-column IEEE format using 10pt fonts on US letter paper, with all fonts embedded. Submissions must be made via EDAS. The IEEE LaTeX and Microsoft Word templates, as well as related information, can be found at the IEEE Computer Society website.

PerFail will be held in conjunction with IEEE Percom 2022. All accepted papers will be included in the Percom workshops proceedings and included and indexed in the IEEE digital library Xplore. At least one author will be required to have a full registration in the Percom 2022 conference and present the paper during the workshop (either remotely or in location). There will be no workshop-only registration.

Submission link: here


Each accepted workshop paper requires a full PerCom registration (no registration is available for workshops only). Otherwise, the paper will be withdrawn from publication. The authors of all accepted papers must guarantee that their paper will be presented at the workshop. Papers not presented at the workshop will be considered as a "no-show" and it will not be included in the proceedings.

Registration link: here


Welcome and Opening Remarks: 13:00 - 13:15

Ella Peltonen (University of Oulu)

Keynote: 13:15 - 14:15

Title: Experience Comes From Bad Judgment: Learning from Mistakes in Research



Jussi Kangasharju
University of Helsinki

Jussi Kangasharju received his MSc from Helsinki University of Technology in 1998. He received his Diplome d’Etudes Approfondies (DEA) from the Ecole Superieure des Sciences Informatiques (ESSI) in Sophia Antipolis in 1998. In 2002 he received his PhD from University of Nice Sophia Antipolis/Institut Eurecom. In 2002 he joined Darmstadt University of Technology (TUD), first as post-doctoral researcher, and from 2004 onwards as assistant professor. Since June 2007 Jussi is a professor at the department of computer science at University of Helsinki and is currently the head of the department. Between 2009 and 2012 he was the director of the Future Internet research program at Helsinki Institute for Information Technology (HIIT). Jussi’s research interests are information-centric networks, edge and cloud computing, content distribution, opportunistic networks, and green ICT. He is a member of IEEE and ACM.


14:15 - 14:30

Technical Session 1: 14:30 - 15:30

Session Chair: Peter Zdankin (University of Duisburg-Essen)

Insights on Mini-Batch Alignment for WiFi-CSI Data Domain Factor Independent Feature Extraction
Authors: Bram van Berlo, Tanir Ozcelebi, Nirvana Meratnia
Unobtrusive sensing has the ambition to embed sensing into our daily lives. A way to achieve it is through repurposing technology that we are already used to having in our environments. Wireless Fidelity (WiFi) sensing which makes use of Channel State Information (CSI) measurement data seems to be a perfect fit for this, since WiFi networks are already omnipresent. A big challenge is that CSI data is very sensitive to 'domain factors' such as position and orientation, making its interpretation in different domains very difficult. We present a domain factor independent feature extraction pipeline called 'mini-batch alignment'. Its goal is to develop models with domain factor independent latent representations. Unfortunately, based on extensive evaluations on a benchmark dataset, the proposed mini-batch alignment pipeline did not lead to better inference performance. We discuss the pitfalls that may have led to this result, as well as future research directions.
Enabling On-Device Smartphone GPU based Training: Lessons Learned Authors: Anish Das, Young D. Kwon, Jagmohan Chauhan, Cecilia Mascolo
Deep Learning (DL) has shown impressive performance in many mobile applications. Most existing works have focused on reducing the computational and resource overheads of running Deep Neural Networks (DNN) inference on resource-constrained mobile devices. However, the other aspect of DNN operations, i.e. training (forward and backward passes) on smartphone GPUs, has received little attention thus far. To this end, we conduct an initial analysis to examine the feasibility of on-device training on smartphones using mobile GPUs. We first employ the open-source mobile DL framework (MNN) and its OpenCL backend for running compute kernels on GPUs. Next, we observed that training on CPUs is much faster than on GPUs and identified two possible bottlenecks related to this observation: (i) computation and (ii) memory bottlenecks. To solve the computation bottleneck, we optimize the OpenCL backend's kernels, showing 2x improvements (40-70 GFLOPs) over CPUs (15-30 GFLOPs) on the Snapdragon 8 series processors. However, we find that the full DNN training is still much slower on GPUs than on CPUs, indicating that memory bottleneck plays a significant role in the lower performance of GPU over CPU. The data movement takes almost 91% of training time due to the low bandwidth. Lastly, based on the findings and failures during our investigation, we present limitations and practical guidelines for future directions.


15:30 - 15:45

Panel Discussion: 15:45 - 16:45

Topic: Learnings and Failures in Research

Moderator: Tanya Shreedhar (IIIT-Delhi)



Jon Crowcroft
University of Cambridge

Jon Crowcroft has been the Marconi Professor of Communications Systems in the Computer Laboratory since October 2001. He has worked in the area of Internet support for multimedia communications for over 30 years. Three main topics of interest have been scalable multicast routing, practical approaches to traffic management, and the design of deployable end-to-end protocols. Current active research areas are Opportunistic Communications, Social Networks, Privacy Preserving Analytics, and techniques and algorithms to scale infrastructure-free mobile systems. He leans towards a "build and learn" paradigm for research. From 2016-2018, he was Programm Chair at the Turing, the UK's national Data Science and AI Institute, and is now researcher-at-large there. He graduated in Physics from Trinity College, University of Cambridge in 1979, gained an MSc in Computing in 1981 and PhD in 1993, both from UCL. He is a Fellow the Royal Society, a Fellow of the ACM, a Fellow of the British Computer Society, a Fellow of the IET and the Royal Academy of Engineering and a Fellow of the IEEE.


Victor Bahl
Microsoft Research

Victor Bahl is a Technical Fellow and Chief Technology Officer of Azure for Operators at Microsoft Research. He is known for his research contributions to edge computing, white space radio data networks, wireless network virtualization, and for bringing wireless links into the datacenter. He is also known for his leadership of the mobile computing community as the co-founder of the ACM Special Interest Group on Mobility of Systems, Users, Data, and Computing (SIGMOBILE). He is the founder of international conference on Mobile Systems, Applications, and Services Conference (MobiSys), and the founder of ACM Mobile Computing and Communications Review. Bahl has received important awards; delivered dozens of keynotes and plenary talks at conferences and workshops; delivered over six dozen distinguished seminars at universities; written over hundred papers with more than 25,000 citations and awarded over 100 US and international patents. He is a Fellow of the Association for Computing Machinery, IEEE, and American Association for the Advancement of Science.


Daniela Nicklas
University of Bamberg

Daniela Nicklas is full professor at the University of Bamberg, Germany since 2014 and holds the Chair of Computer Science, in particular Mobile Software Systems / Mobility. Before that, she was a junior professor for database and internet technologies at the Universität Oldenburg and member of the Member of Executive Board in the Transportation division at the OFFIS institute for computer science. Her research interests are computer systems that bridge the gap between the physical world and the digital world. She focuses on the continuous management of data from sensors and other active data sources and their incorporation in so-called context-aware applications. In 2009, she received the IBM Exploratory Stream Analytics Innovation Award for „Data Stream Technology for Future Energy Grid Control“. Together with Prof. Dr. Marc Redepenning, she manages the Smart City Research Lab. She is a member of many programme committees and organizing committees of pervasive computing and database conferences and workshops and a member of the editorial board of the Datenbankspektrum (German Journal on Databases).


Keith Winstein
Stanford University

Keith Winstein is an assistant professor of computer science and, by courtesy, of electrical engineering at Stanford University. His research group creates new kinds of networked systems by rethinking abstractions around communication, compression, and computing. Some of his group’s research has found broader use, including the Mosh tool, the Puffer video-streaming site, the Lepton compression tool, the Mahimahi network emulators, and the gg lambda-computing framework. He has received the SIGCOMM Rising Star Award, the Sloan Research Fellowship, the NSF CAREER Award, the Usenix NSDI Community Award (2020, 2017), the Usenix ATC Best Paper Award, a Google Faculty Research Award (2017, 2015), a Facebook Faculty Award, the Applied Networking Research Prize, the SIGCOMM Doctoral Dissertation Award, and a Sprowls award for best doctoral thesis in computer science at MIT. Winstein previously served as a staff reporter at The Wall Street Journal and worked at Ksplice, a startup company (now part of Oracle) where he was the vice president of product management and business development and also cleaned the bathroom. He did his undergraduate and graduate work at MIT.


Suzan Bayhan
University of Twente

Suzan Bayhan is an assistant professor at the University of Twente (UT) in the Netherlands. Before joining the UT, she worked as a researcher at TU Berlin, University of Helsinki, and Bogazici University. She co-received the best paper award at ACM ICN 2015 and IEEE WoWMoM 2020 as well as the best demo award at IEEE INFOCOM 2020. She acted as a mentoring co-chair in N2Women board. Her current research interests include coexistence in unlicensed spectrum, edge computing, and energy efficient communications.


16:45 - 17:00

Technical Session 2: 17:00 - 18:00

Session Chair: Nitinder Mohan (Technical University of Munich)

Lessons Learned from an Eye Tracking Study for Targeted Advertising in the Wild Extraction
Authors: Melanie Heck, Janick Edinger, Christian Becker
With the ambitious objective in mind to assess the validity of gaze-based interest prediction and test its application for targeted advertising on a self-order terminal, we set out to conduct a user study at a fast food restaurant. We recorded the customers' eye movements while they were placing their order, and used the collected data to infer which information about the meals they were interested in. We then wanted to test whether the predictions can be used to display targeted advertisements, and influence the customers' purchase intention for a complementary product. The results did not corroborate the hypothesized relationship between visual attention and interest. However, due to the multiplicity of underlying assumptions, we were unable to identify whether a link does indeed not exist, or whether the result was caused by the experimental setup. In this paper, we describe the design and results of our user study, and discuss our lessons learned. Our observations shall guide other researchers to avoid these common pitfalls.
Hindsight is 20/20: Retrospective Lessons for Conducting Longitudinal Wearable Sensing Studies
Authors: Salaar Liaqat, Daniyal Liaqat, Tatiana Son, Andrea Gershon, Moshe Gabel, Robert Wu, Eyal de Lara
Pervasive sensing using wearables for health monitoring presents a promising and unique opportunity to widely manage illnesses and conditions. To better understand the capabilities and limitations of using wearable devices for health monitoring, systems need to be developed and studies conducted. We conducted one such study for monitoring patients with Chronic Obstructive Pulmonary Disease (COPD), in which we aim to understand the disease and predict patient outcomes. However, despite a carefully well-planned and well-conducted study that resulted in a very large dataset, some non-obvious design oversights meant the data was much less useful. We analyze the shortcomings of our study to construct lessons and concrete actions to avoid these pitfalls. We ratify these lessons by briefly discussing a second iteration of our study, in which we apply these lessons and obtain much better outcomes. Real-world sensing studies are time consuming and expensive investments, for a promising research area. By sharing our failure and proposing actionable lessons, we hope to minimize the risk for others aiming to run such studies.

Best Paper Award and Closing Remarks: 18:00 - 18:10

Nitinder Mohan (Technical University of Munich)


Organizing Committee


Nitinder Mohan Technical University of Munich


Ella Peltonen University of Oulu


Peter Zdankin University of Duisburg-Essen

Panel Organizer


Tanya Shreedhar IIIT-Delhi

Student Volunteers


Tri Nguyen University of Oulu


Mehmet Mert Bese Technical University of Munich

Technical Program Committee


Aaron Ding TU Delft


Apoorv Shukla Huawei Munich Research Center


Atakan Aral University of Vienna


Daniela Nicklas University of Bamberg


Eirini Eleni Tsiropoulou University of New Mexico


Gregor Schiele University of Duisburg-Essen


Gürkan Solmaz NEC Labs Europe


Hamed Haddadi Imperial College London


Javier Berrocal Universidad de Extremadura


Jörg Ott Technical University of Munich


Jussi Kangasharju University of Helsinki


Lauri Lovén University of Oulu


Lik-Hang Lee KAIST, South Korea


Oliver Gasser MPI-Informatics


Petteri Nurmi University of Helsinki


Roman Kolcun University of Cambridge


Stephan Sigg Aalto University


Torben Weis University of Duisburg-Essen