Publications associées à Hadoop

Communication trade-offs for Local-SGD with large step size

Synchronous mini-batch SGD is state-of-the-art for large-scale distributed machine learning. However, in practice, its convergence is bottlenecked by slow communication rounds between worker nodes. A natural solution to reduce communication is to use the " ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2019

Don't cry over spilled records: Memory elasticity of data-parallel applications and its application to cluster scheduling

Willy Zwaenepoel, Calin Iorgulescu, Syed Mohammad Aunn Raza, Florin Dinu, Wajih Ul Hassan

Understanding the performance of data-parallel workloads when resource-constrained has significant practical importance but unfortunately has received only limited attention. This paper identifies, quantifies and demonstrates memory elasticity, an intrinsi ...

The USENIX Association2017

Optimized Distributed Hyperparameter Search and Simulation for Lung Texture Classification in CT Using Hadoop

Adrien Raphaël Depeursinge

Many medical image analysis tasks require complex learning strategies to reach a quality of image-based decision support that is sufficient in clinical practice. The analysis of medical texture in tomographic images, for example of lung tissue, is no excep ...

2016

Reliable and Real-Time Distributed Abstractions

David Kozhaya

The celebrated distributed computing approach for building systems and services using multiple machines continues to expand to new domains. Computation devices nowadays have additional sensing and communication capabilities, while becoming, at the same tim ...

EPFL2016

Big-Data Streaming Applications Scheduling Based on Staged Multi-armed Bandits

David Atienza Alonso, Karim Kanoun, Cem Tekin

Several techniques have been recently proposed to adapt Big-Data streaming applications to existing many core platforms. Among these techniques, online reinforcement learning methods have been proposed that learn how to adapt at run-time the throughput and ...

Institute of Electrical and Electronics Engineers2016

Big-Data Streaming Applications Scheduling Based on Staged Multi-Armed Bandits

David Atienza Alonso, Karim Kanoun, Cem Tekin

Several techniques have been recently proposed to adapt Big-Data streaming applications to existing many core platforms. Among these techniques, online reinforcement learning methods have been proposed that learn how to adapt at run-time the throughput and ...

Institute of Electrical and Electronics Engineers2016

SigCO: Mining Significant Correlations via a Distributed Real-time Computation Engine

Karl Aberer, Jean Paul Calbimonte Perez, Tian Guo, Hao Zhuang

The dramatic rise of time-series data produced in a variety of contexts, such as stock markets, mobile sensing, sensor networks, data centre monitoring, etc., has fuelled the development of large-scale distributed real-time computation systems (e.g., Apach ...

2015

Online indexing and distributed querying model-view sensor data in the cloud

Karl Aberer, Tian Guo, Hao Zhuang

As various kinds of sensors penetrate our daily life (e.g., sensor networks for environmental monitoring, GPS for localization and navigation), the efficient management of massive amount of sensor data becomes increasingly important at present. Many sensor ...

Springer-Verlag Berlin2014

DiNoDB: Efficient Large-Scale Raw Data Analytics

Anastasia Ailamaki, Marko Vukolic, Ioannis Alagiannis, Erietta Liarou, Yongchao Tian

Modern big data workflows, found in e.g., machine learning use cases, often involve iterations of cycles of batch analytics and interactive analytics on temporary data. Whereas batch analytics solutions for large volumes of raw data are well established (e ...

2014

Model-View Sensor Data Management in the Cloud

Karl Aberer, Tian Guo

Infinite nature of sensor data poses a serious challenge for query processing even in a cloud infrastructure. Model-based sensor data approximation reduces the amount of data for query processing, but all modeled segments need to be scanned, in the worst c ...

2013

Model-View Sensor Data Management in the Cloud

Karl Aberer, Tian Guo

Infinite nature of sensor data poses a serious challenge for query processing even in a cloud infrastructure. Model-based sensor data approximation reduces the amount of data for query processing, but all modeled segments need to be scanned, in the worst c ...

IEEE2013

Scalability of write-ahead logging on multicore and multisocket hardware

Anastasia Ailamaki, Frederick Ryan Johnson, Manoussos Gavriil Athanassoulis, Radu Ioan Stoica, Ippokratis Pandis

The shift to multi-core and multi-socket hardware brings new challenges to database systems, as the software parallelism determines performance. Even though database systems traditionally accommodate simultaneous requests, a multitude of synchronization ba ...

2012

Generalized Universality

Rachid Guerraoui

State machine replication reduces distributed to centralized computing. Any sequential service, modeled by a state machine, can be replicated over any number of processes and made highly available to all of them. At the heart of this fundamental reduction ...

Springer Berlin Heidelberg2011

Adaptive query execution for data management in the cloud

Anastasia Ailamaki, Debabrata Dash, Adrian Daniel Popescu

A major component of many cloud services is query processing on data stored in the underlying cloud cluster. The traditional techniques for query processing on a cluster are those offered by parallel DBMS. These techniques however, cannot guarantee high pe ...

ACM Press2010

Aether: A scalable approach to logging

Anastasia Ailamaki, Frederick Ryan Johnson, Manoussos Gavriil Athanassoulis, Radu Ioan Stoica, Ippokratis Pandis

The shift to multi-core hardware brings new challenges to database systems, as the software parallelism determines performance. Even though database systems traditionally accommodate simultaneous requests, a multitude of synchronization barriers serialize ...

2010

Making Cluster Applications Energy-Aware

Dejan Kostic, Nedeljko Vasic, Vincent Salzgeber, Martin Barisits

Power consumption has become a critical issue in large scale clusters. Existing solutions for addressing the servers' energy consumption suggest ``shrinking'' the set of active machines, at least until the more power-proport-ional hardware devices become a ...

2009

Toward Massive Query Optimization in Large-Scale Distributed Stream Systems

Karl Aberer

Existing distributed stream systems adopt a tightly-coupled communication paradigm and focus on fine-tuning of operator placements to achieve communication efficiency. This kind of approach is hard to scale (both to the nodes in the network and the users). ...

2008