Third International Workshop on Negative Results in Pervasive Computing
Co-located with IEEE PerCom 2024, Biarritz, France
Not all research leads to fruitful results, trying new ways or methods may surpass the state of the art, but sometimes the hypothesis is not proven or the improvement is insignificant. But failure to succeed is not failure to progress and this workshop aims to create a platform for sharing insights, experiences, and lessons learned when conducting research in the area of pervasive computing.
While the direct outcome of negative results might not contribute much to the field, the wisdom of hindsight could be a contribution itself, such that other researchers could avoid falling into similar pitfalls. We consider negative results to be studies that are run correctly (in the light of the current state of the art) and in good practice, but fail in terms of proving of the hypothesis or come up with no significance. The “badness” of the work can also come out as a properly but unfittingly designed data collection, or (non-trivial) lapses of hindsight especially in measurement studies.
We took the insights and discussion from last year and wrote a paper about the collected information. You can read the published manuscript in IEEE Pervasive Computitng here.
The papers of this workshop should highlight lessons learned from the negative results. The main outcome of the workshop is to share experiences so that others avoid the pitfalls that the community generally overlooks in the final accepted publications. All areas of pervasive computing, networking and systems research are considered. While we take a very broad view of “negative results”, submissions based on opinions and non-fundamental circumstances (e.g. coding errors and “bugs”) are not in scope of the workshop as they do not indicate if the approach (or hypothesis) was bad.
The main topics of interests include (but are not limited to):
We also welcome submissions from experienced researchers that recounts post-mortem of experiments or research directions they have failed in the past (e.g. in a story-based format). With this workshop, our aim is to normalize the negative outcomes and inherent failures while conducting research in pervasive computing, systems and networking, and provide a complementary view to all the success stories in these fields.
Paper submission: November 17th, 2023 (extended to December 1st, 2023)
Author notification: January 8th, 2024
Camera-ready due: February 2nd, 2024
Workshop date: March 15th, 2024
Regular papers should present novel perspectives within the scope of the workshop: negative results, lessons learned, and other fruitful “failure” stories. Papers must be in PDF format and contain 6 pages maximum (including references). Papers should contain names and affiliations of the authors (not blinded). All papers must be typeset in double-column IEEE format using 10pt fonts on US letter paper, with all fonts embedded. Submissions must be made via EDAS. The IEEE LaTeX and Microsoft Word templates, as well as related information, can be found at the IEEE Computer Society website.
PerFail will be held in conjunction with IEEE Percom 2024. All accepted papers will be included in the Percom workshops proceedings and included and indexed in the IEEE digital library Xplore. At least one author will be required to have a full registration in the Percom 2024 conference and present the paper during the workshop (either remotely or in location). There will be no workshop-only registration.
Submission link: HERE
Each accepted workshop paper requires a full PerCom registration (no registration is available for workshops only). Otherwise, the paper will be withdrawn from publication. The authors of all accepted papers must guarantee that their paper will be presented at the workshop. Papers not presented at the workshop will be considered as a "no-show" and it will not be included in the proceedings.
Registration link: here
|
Context Matters: Lessons Learned from Emulated and Simulated TSN Environments
Authors: Filip Rezabek, Marcin Bosk, Leander Seidlitz, Jörg Ott, Georg Carle
Read Abstract
Clock synchronization precision and security are imperative in modern
Time Sensitive Networking (TSN) systems and applications building on top
of them. Their deployment validation is challenging due to many,
sometimes incompatible or inaccessible, implementations of TSN and
related standards. Such aspects make ensuring their proper functionality
and performance crucial and require thorough evaluation and validation
to ensure system's robustness. For that reason, we consider various
platforms using hardware or simulations as an evaluation tool.
This work shows the lessons learned during real-world deployments of the
Precision Time Protocol (PTP) and MACsec. As a part of that, we identify
challenges of combining various sensors and how to possibly overcome
them using open-source solutions. To protect sensitive traffic of PTP
and other TSN traffic, we evaluate synergies between MACsec and its
performance implications. The findings point to requirements for future
deployments of TSN systems. Last, we discuss simulation as a supporting
tool for TSN experimentation and its limitations. Nevertheless, we
highlight how different evaluation approaches can provide a complete
view of the system under test.
|
|
Designing and Executing a Large-scale Real-life Affective Study
Authors: Joanna Komoszyńska, Dominika Kunc, Bartosz Perz, Adam Hebko, Przemysław Kazienko and Stanislaw Saganowski
Read Abstract
Despite advancements in emotion recognition, the suitability of existing
AI models for daily use remains uncertain. Creating more accurate
solutions requires amounts of data uncollectible in a laboratory
setting, thus forcing researchers to focus on an information-rich
environment: everyday life. However, the constraints of in-the-field
studies are different than those of in-the-laboratory ones, which are
familiar to most researchers. In this work, we delve into the challenges
in designing and conducting a large-scale real-life affective study,
employing wearable devices and smartphones for data collection. We
highlight encountered issues, made decisions, and mistakes, making our
experiences valuable for researchers aiming to conduct studies in a
real-life setting.
|
|
Too Good To Be True: accuracy overestimation in (re)current practices for Human
Activity Recognition
Authors: Andrés Tello, Victoria Degeler, Alexander Lazovik
Read Abstract
Today, there are standard and well established procedures within the
Human Activity Recognition (HAR) pipeline.
However, some of these conventional approaches lead to accuracy
overestimation.
In particular, sliding windows for data segmentation followed by
standard random k-fold cross validation, produce biased results.
An analysis of previous literature and present-day studies,
surprisingly, shows that these are common approaches in state-of-the-art
studies on HAR.
It is important to raise awareness in the scientific community about
this problem, whose negative effects are being overlooked.
Otherwise, publications of biased results lead to papers that report
lower accuracies, with correct unbiased methods, harder to publish.
Several experiments with different types of datasets and different types
of classification models allow us to exhibit the problem and show it
persists independently of the method or dataset.
|
|
Can we generate real faces from rPPG signals? Probably not
Authors: Li Honghan, Nhi Nguyen, Constantino Alvarez Casado, Xiaoting Wu, Miguel Bordallo Lopez
Read Abstract
The potential of generating authentic human facial images from remote
photoplethysmography (rPPG) signals is a compelling idea, with
significant implications for biometric authentication and human-computer
interaction. This study explores it by using a large-scale dataset to
train a diffusion-based generative model, leveraging rPPG signals
extracted from facial videos. The initial training phase yields
promising results, with the model demonstrating a capacity to synthesize
facial likenesses that closely match the corresponding subjects in the
training dataset. However, the performance notably falters during
validation with an independent dataset, where a marked divergence
between generated and actual faces becomes apparent. A subsequent human
perception study corroborates this discrepancy. These observations
suggest that rPPG signals alone may not be reliable for accurately
generating realistic facial imagery.
|
|
Do Weak Brain Signals Get Amplified When Strong Brain Signals are Evoked?
Authors: Ekansh Gupta, Cheng-Yeh Chen and Raghupathy Sivakumar
Read Abstract
Brain-computer interfaces (BCIs) facilitate an unprecedented fusion
between the human mind and pervasive computing systems, enabling users
to engage with connected devices in their environment through neural
signaling. Despite their potential, BCIs face certain challenges that
hinder their widespread proliferation, such as low SNR and high noise
levels in brain signals recorded via non-invasive techniques like EEG,
high variability in signals among users that hinders generalization,
usability challenges, etc. While brain signals like the error potential
(ErrP) showcase low SNR and have lower detection accuracy, there are
other kinds of signals that showcase high detection rates and resilience
to noise. Motivated by this disparity, we ask ourselves if the abstract
cognitive states involved in the evocation of such resilient signals be
leveraged to amplify or augment the weaker signals, and thus provide
them a performance boost. We investigate this hypothesis by designing an
experiment to interface these two kinds of signals and collect EEG data
in our lab through human trials. We evaluate our hypothesis and contrast
our results with other datasets of isolated signals using spatial
filtering and deep learning models. We obtain negative results and
reflect on the insights and the lessons learned based on them and talk
about plausible explanations and future work while also reassessing our
initial hypothesis.
|
|
Echocardiographic Epicardial Adipose Tissue Quantification: Challenges and
Insights
Authors: Payel Patra, Andrea Bianchi, Daniele Di Pompeo, Antinisca Di Marco
Read Abstract
e-Health applications, as a cornerstone of modern distributed systems,
must synergize with advanced analysis methodologies, incorporating image
processing, statistical, and predictive techniques to expedite diagnosis
and optimize therapeutic strategies. Cardiovascular disease (CVD)
presents a formidable health challenge, claiming 18 million lives
annually, with projections set to worsen due to population aging, the
rise of metabolic diseases, and gaps in effective prevention and precise
risk stratification. A pivotal indicator of cardiovascular health, the
epicardial adipose tissue (EAT) thickness, is traditionally estimated by
medical professionals without a standardized and precise procedure. This
paper chronicles our endeavor to automate the delineation of EAT from
echocardiogram videos, a fundamental precursor to its thickness
quantification. We confronted the intricate task of interpreting
echocardiographic data and trialed a variety of image processing methods
aimed at clarifying the EAT's representation amidst the heart's dynamic
activity and inherent imaging noise. Our study's narrative contributes
to the pervasive computing domain, envisaging the deployment of such
medical applications as on-demand cloud services for medical experts and
institutions, thus fostering collaborative, efficient, and accurate
cardiovascular health real-time assessment. Unfortunately, our study
failed and in this paper we analyse the reasons and we report the lesson
learned.
|
Since 2021, he is an assistant professor of distributed operating systems at the University of Hamburg, Germany. He studied at the University of Mannheim, Germany, where he also received his PhD in the group of Christian Becker. He also studied at Taiwan National University and the University of Alberta, Canada, and completed research stays at the University of British Columbia, Canada, Hong Kong Polytechnical University, and Georgia State University, Atlanta, USA. He has published in proceedings of international conferences such as MobiQuitous, PerCom, IPDPS, MSWiM, ICCCN, IUI, COMPSAC, and CHIIR, and received the PerCom 2021 Mark Weiser Best Paper Award. He was a shadow PC member for EuroSys 2021 and his research interests include computation offloading, edge computing, and assistive technologies.
