Publications associées à Word2vec

Examining European Press Coverage of the Covid-19 No-Vax Movement: An NLP Framework

This paper examines how the European press dealt with the no-vax reactions against the Covid-19 vaccine and the dis- and misinformation associated with this movement. Using a curated dataset of 1786 articles from 19 European newspapers on the anti-vaccine ...

ASSOC COMPUTING MACHINERY2023

Learnable latent embeddings for joint behavioural and neural analysis

Mackenzie Mathis, Steffen Schneider, Jin Hwa Lee

Mapping behavioural actions to neural activity is a fundamental goal of neuroscience. As our ability to record large neural and behavioural data increases, there is growing interest in modelling neural dynamics during adaptive behaviours to probe neural re ...

NATURE PORTFOLIO2023

4M: Massively Multimodal Masked Modeling

Amir Roshan Zamir, Oguzhan Fatih Kar, Roman Christian Bachmann, Shuqing Teresa Yeo, David Mizrahi

Current machine learning models for vision are often highly specialized and limited to a single modality and task. In contrast, recent large language models exhibit a wide range of capabilities, hinting at a possibility for similarly versatile models in co ...

Neural Information Processing Systems (Nips)2023

Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models

Karl Aberer, Rémi Philippe Lebret, Mohammadreza Banaei

Recent transformer language models achieve outstanding results in many natural language processing (NLP) tasks. However, their enormous size often makes them impractical on memory-constrained devices, requiring practitioners to compress them to smaller net ...

Assoc Computational Linguistics-Acl2023

Robustness, replicability and scalability in topic modelling

Omar Ballester Gonzalez, Orion B Penner

Approaches for estimating the similarity between individual publications are an area of long -standing interest in the scientometrics and informetrics communities. Traditional techniques have generally relied on references and other metadata, while text mi ...

ELSEVIER2022

Bias at a Second Glance: A Deep Dive into Bias for German Educational Peer-Review Data Modeling

Vinitra Swamy, Tanja Christina Käser Jacober, Thiemo Wambsganss

Natural Language Processing (NLP) has become increasingly utilized to provide adaptivity in educational applications. However, recent research has highlighted a variety of biases in pre-trained language models. While existing studies investigate bias in di ...

2022

Further results on latent discourse models and word embeddings

Youssef Allouah

We discuss some properties of generative models for word embeddings. Namely, (Arora et al., 2016) proposed a latent discourse model implying the concentration of the partition function of the word vectors. This concentration phenomenon led to an asymptotic ...

Microtome Publishing2021

Faster Parallel Training of Word Embeddings

Martin Jaggi

Word embeddings have gained increasing popularity in the recent years due to the Word2vec library and its extension fastText that uses subword information. In this paper, we aim at improving the execution speed of fastText training on homogeneous multi- an ...

IEEE COMPUTER SOC2021

Self-Supervised Neural Topic Modeling

Martin Jaggi

Topic models are useful tools for analyzing and interpreting the main underlying themes of large corpora of text. Most topic models rely on word co-occurrence for computing a topic, i.e., a weighted set of words that together represent a high-level semanti ...

Assoc Computational Linguistics-Acl2021

Learning computationally efficient static word and sentence representations

Prakhar Gupta

Most of the Natural Language Processing (NLP) algorithms involve use of distributed vector representations of linguistic units (primarily words and sentences) also known as embeddings in one way or another. These embeddings come in two flavours namely, sta ...

EPFL2021

Word-level Embeddings for Cross-Task Transfer Learning in Speech Processing

Pierre Anton Beckmann, Milos Cernak

Recent breakthroughs in deep learning often rely on representation learning and knowledge transfer. In recent years, unsupervised and self-supervised techniques for learning speech representation were developed to foster automatic speech recognition. Up to ...

EUROPEAN ASSOC SIGNAL SPEECH & IMAGE PROCESSING-EURASIP2021

Concept Discovery for The Interpretation of Landscape Scenicness

Devis Tuia

In this paper, we study how to extract visual concepts to understand landscape scenicness. Using visual feature representations from a Convolutional Neural Network (CNN), we learn a number of Concept Activation Vectors (CAV) aligned with semantic concepts ...

2020

Better Word Embeddings by Disentangling Contextual n-Gram Information

Martin Jaggi, Matteo Pagliardini, Prakhar Gupta

Pre-trained word vectors are ubiquitous in Natural Language Processing applications. In this paper, we show how training word embeddings jointly with bigram and even trigram embeddings, results in improved unigram embeddings. We claim that training word em ...

2019

Learning to Create Sentence Semantic Relation Graphs for Multi-Document Summarization

Boi Faltings, Diego Matteo Antognini

Linking facts across documents is a challenging task, as the language used to express the same information in a sentence can vary significantly, which complicates the task of multi-document summarization. Consequently, existing approaches heavily rely on h ...

2019

Aligning Multilingual Word Embeddings for Cross-Modal Retrieval Task

Karl Aberer, Rémi Philippe Lebret, Alireza Mohammadshahi

In this paper, we propose a new approach to learn multimodal multilingual embeddings for matching images and their relevant captions in two languages. We combine two existing objective functions to make images and captions close in a joint embedding space ...

Association for Computational Linguistics2019

Evolution of Topics and Novelty in Science

Omar Ballester Gonzalez, Orion B Penner

Methods of estimating the similarity between individual publications is an area of long-standing interest in the scientometrics community. Traditional methods have generally relied on references and other metadata, while text mining approaches based on tit ...

INT SOC SCIENTOMETRICS & INFORMETRICS-ISSI2019

Multimodal person recognition in audio-visual streams

Do Hoang Nam Le

Multimedia databases are growing rapidly in size in the digital age. To increase the value of these data and to enhance the user experience, there is a need to make these videos searchable through automatic indexing. Because people appearing and talking in ...

EPFL2019

Crosslingual Document Embedding as Reduced-Rank Ridge Regression

Martin Jaggi, Robert West, Martin Josifoski, Ivan Paskov

There has recently been much interest in extending vector-based word representations to multiple languages, such that words can be compared across languages. In this paper, we shift the focus from words to documents and introduce a method for embedding doc ...

2019

Feature Learning of Virus Genome Evolution With the Nucleotide Skip-Gram Neural Network

Hyunjin Shim

Recent studies reveal that even the smallest genomes such as viruses evolve through complex and stochastic processes, and the assumption of independent alleles is not valid in most applications. Advances in sequencing technologies produce multiple time-poi ...

2019

Simple Unsupervised Keyphrase Extraction using Sentence Embeddings

Martin Jaggi, Claudiu-Cristian Musat, Kamil Bennani-Smires

Keyphrase extraction is the task of automatically selecting a small set of phrases that best describe a given free text document. Keyphrases can be used for indexing, searching, aggregating and summarizing text documents, serving many automatic as well as ...

2018