Publications related to Data engineering

Diversity and neocolonialism in Big Data research: Avoiding extractivism while struggling with paternalism

The extractive logic of Big Data-driven technology and knowledge production has raised serious concerns. While most criticism initially focused on the impacts on Western societies, attention is now increasingly turning to the consequences for communities i ...

Sage Publications Inc2023

Interactive-time Exploration, Querying, and Analysis of Large High-dimensional Datasets

Sachin Basil John

In the current era of big data, aggregation queries on high-dimensional datasets are frequently utilized to uncover hidden patterns, trends, and correlations critical for effective business decision-making. Data cubes facilitate such queries by employing p ...

EPFL2023

Blue Brain Nexus: An open, secure, scalable system for knowledge graph management and data-driven science

Henry Markram, Sean Lewis Hill, Samuel Claude Kerrien, Carolina Johanna Elisabeth Lindqvist, Alejandra Garcia Rojas Martinez, Huanxiang Lu, Mohameth François Sy, Anna-Kristin Kaufmann, Jonathan Raël Lurie, Henry Genet, Pierre-Alexandre Fonta, Alexander Désiré Ulbrich, Michaël Fernand Paul Dupont, Silvia Rosario Jimenez Tejeda, Bogdan Roman, Ian Lavriushev, Didac Montero Mendez, Wojciech Adam Wajerowicz, Kenneth William Pirman, Julien Antonin Machon, Dhanesh Neela Mana, Natalia Stafeeva

Modern data-driven science often consists of iterative cycles of data discovery, acquisition, preparation, analysis, model building and validation leading to knowledge discovery as well as dissemination at scale. The unique challenges of building and simul ...

2023

Bridging the gap between model-driven and data-driven methods in the era of Big Data

Gael Lederrey

Data-driven and model-driven methodologies can be regarded as competitive fields since they tackle similar problems such as prediction. However, these two fields can learn from each other to improve themselves. Indeed, data-driven methodologies have been d ...

EPFL2022

Columnar Storage Optimization and Caching for Data Lakes

Haoqiong Bian

As a unified data repository, data lake plays a vital role in enterprise data management and analysis. It composes the raw files into tables that are processed in-situ by various computation engines and applications. Therefore, the read performance of the ...

2022

Deep Learning Enables Individual Xenograft Cell Classification in Histological Images by Analysis of Contextual Features

Michaël Unser, Cathrin Brisken, Daniel Sage, Olivier Burri, Quentin Juppet, Fabio De Martino, Martin Weigert

Patient-Derived Xenografts (PDXs) are the preclinical models which best recapitulate inter- and intra-patient complexity of human breast malignancies, and are also emerging as useful tools to study the normal breast epithelium. However, data analysis gener ...

2021

Historical City Maps Semantic Segmentation Dataset

Rémi Guillaume Petitpierre

This dataset includes a total of 635 annotated image patches from historical city maps. It is designed for the semantic segmentation of the maps into 5 semantic classes (building blocks, non-built, water, road network, background frame). 330 patches are ta ...

Zenodo2021

Countering Bias in Personalized Rankings From Data Engineering to Algorithm Development

Mirko Marras

This tutorial presents recent advances on the assessment and mitigation of data and algorithmic bias in personalized rankings. We first introduce fundamental concepts and definitions associated with bias issues, covering the state of the art and describing ...

IEEE COMPUTER SOC2021

Decentralized Industrial IoT Data Management Based on Blockchain and IPFS

Dimitrios Kyritsis, Jinzhi Lu, Xiaochen Zheng

The wide application of Internet of Things (IoT) has fostered the development of Industry 4.0. In manufacturing domain, Industrial IoT (IIoT) are key components of the Factories of the Future (FoF). The big IIoT data are the foundation of implementing data ...

SPRINGER INTERNATIONAL PUBLISHING AG2020

DELF: Safeguarding deletion correctness in Online Social Networks

Georgios Damaskinos

Deletion is a core facet of Online Social Networks (OSNs). For users, deletion is a tool to remove what they have shared and control their data. For OSNs, robust deletion is both an obligation to their users and a risk when developer mistakes inevitably oc ...

USENIX ASSOC2020

A system design for elastically scaling transaction processing engines in virtualized servers

Anastasia Ailamaki, Raja Appuswamy, Angelos Christos Anadiotis, Hillel Avni

Online Transaction Processing (OLTP) deployments are migrating from on-premise to cloud settings in order to exploit the elasticity of cloud infrastructure which allows them to adapt to workload variations. However, cloud adaptation comes at the cost of re ...

ASSOC COMPUTING MACHINERY2020

The Sixth Visual Object Tracking VOT2018 Challenge Results

Erhan Gündogdu, Tianzong Zhang, Wei Wu, Qiang Wang, Zhiye Wang, Li Wang, Bo Li, Yunpeng Li, Zeyu Zhang, Bo Chen, Jiri Matas, Namhoon Lee, Tianyang Xu, Jing Li, Ning Wang, Hongwei Li, Feng Li, Jue Wang, Peixia Li, Ran Tao, Yi Wu, Yunhua Zhang, Yuxuan Sun, Qing Guo, Xiaofan Zhang, Wenjun Zeng, Luca Bertinetto, Siwen Wang, Chong Sun, Litu Rout

The Visual Object Tracking challenge VOT2018 is the sixth annual tracker benchmarking activity organized by the VOT initiative. Results of over eighty trackers are presented; many are state-of-the-art trackers published at major computer vision conferences ...

SPRINGER INTERNATIONAL PUBLISHING AG2019

Crowdbreaks: Tracking Health Trends Using Public Social Media Data and Crowdsourcing

Marcel Salathé, Martin Guillermo Mueller

In the past decade, tracking health trends using social media data has shown great promise, due to a powerful combination of massive adoption of social media around the world, and increasingly potent hardware and software that enables us to work with these ...

2019

Accelerating Data Analysis in Simulation Neuroscience with Big Data Technologies

Felix Schürmann, Judit Planas Carbonell, Fabien Jonathan Delalondre

Important progress in computational sciences has been made possible recently thanks to the increasing computing power of high performance systems. Following this trend, larger scientific studies, like brain tissue simulations, will continue to grow in the ...

Springer International Publishing2018

Awareness Towards Industry 4.0: Key Enablers and Applications for Internet of Things and Big Data

Christopher Tucci, Myrna Fatima Flores Pineda

The fourth Industrial revolution promises to increase productivity, flexibility and automation of internal business processes integrating value chains and supporting companies to design and offer novel services based on the availability of data enabled by ...

SPRINGER INTERNATIONAL PUBLISHING AG2018

Semantic Interoperability and Open IoT APIs for Smart Cities Applications

Dimitrios Kyritsis, Prodromos Kolyvakis

As the percentage of the total population living in urban regions is increasing, new challenges for the cities of the future arise. Smart Cities emerged as a solution to these challenges building on the strength of intelligent information, communication te ...

SPRINGER-VERLAG BERLIN2018

Isolation Forest Based Method for Low-Quality Synchrophasor Measurements and Early Events Detection

Xiaoying Tang, Tong Wu

This paper proposes an online data-driven approach that utilizes phasor measurement unit (PMU) data for early-event detection and low-quality data monitoring based on isolation forest (iForest). By skillfully selecting the feature subspaces, we design thre ...

IEEE2018

Learning-Based Hardware Design for Data Acquisition Systems

Cosimo Aprile

This multidisciplinary research work aims to investigate the optimized information extraction from signals or data volumes and to develop tailored hardware implementations that trade-off the complexity of data acquisition with that of data processing, conc ...

EPFL2018

QUASII: QUery-Aware Spatial Incremental Index

Anastasia Ailamaki, Darius Sidlauskas, Thomas Heinis, Mirjana Pavlovic

With large-scale simulations of increasingly detailed models and improvement of data acquisition technologies, massive amounts of data are easily and quickly created and collected. Traditional systems require indexes to be built before analytic queries can ...

2018

No data left behind: real-time insights from a complex data ecosystem

Anastasia Ailamaki, Manolis Karpathiotakis

The typical enterprise data architecture consists of several actively updated data sources (e.g., NoSQL systems, data warehouses), and a central data lake such as HDFS, in which all the data is periodically loaded through ETL processes. To simplify query p ...

2017