arXiv:2410.16736v1 »Full PDF »Large language models (LLMs) have significantly benefited from training on
diverse, high-quality task-specific data, leading to impressive performance
across a range of downstream applications. Current methods often rely on
human-annotated data or predefined task templates to direct powerful LLMs in
synthesizing task-relevant data for effective model training. However, this
dependence on manually designed components may constrain the scope of generated
data, potentially overlooking critical edge cases or novel scenarios that could
challenge the model. In this paper, we present a novel approach, ReverseGen,
designed to automatically generate effective training samples that expose the
weaknesses of LLMs. Specifically, we introduce a dedicated proposer trained to
produce queries that lead target models to generate unsatisfactory responses.
These failure-inducing queries are then used to construct training data,
helping to address the models' shortcomings and improve overall performance.
Our approach is flexible and can be applied to models of various scales (3B,
7B, and 8B). We evaluate ReverseGen on three key applications (safety, honesty,
and math), demonstrating that our generated data is both highly effective and
diverse. Models fine-tuned with ReverseGen-generated data consistently
outperform those trained on human-annotated or general model-generated data,
offering a new perspective on data synthesis for task-specific LLM enhancement.Abstract
GDTS: Goal-Guided Diffusion Model with Tree Sampling for Multi-Modal
Pedestrian Trajectory Prediction
Accurate prediction of pedestrian trajectories is crucial for improving the
safety of autonomous driving. However, this task is generally nontrivial due to
the inherent stochasticity of human motion, which naturally requires the
predictor to generate multi-modal prediction. Previous works leverage various
generative methods, such as GAN and VAE, for pedestrian trajectory prediction.
Nevertheless, these methods may suffer from mode collapse and relatively
low-quality results. The denoising diffusion probabilistic model (DDPM) has
recently been applied to trajectory prediction due to its simple training
process and powerful reconstruction ability. However, current diffusion-based
methods do not fully utilize input information and usually require many
denoising iterations that lead to a long inference time or an additional
network for initialization. To address these challenges and facilitate the use
of diffusion models in multi-modal trajectory prediction, we propose GDTS, a
novel Goal-Guided Diffusion Model with Tree Sampling for multi-modal trajectory
prediction. Considering the "goal-driven" characteristics of human motion, GDTS
leverages goal estimation to guide the generation of the diffusion network. A
two-stage tree sampling algorithm is presented, which leverages common features
to reduce the inference time and improve accuracy for multi-modal prediction.
Experimental results demonstrate that our proposed framework achieves
comparable state-of-the-art performance with real-time inference speed in
public datasets.Abstract
TFDet: Target-Aware Fusion for RGB-T Pedestrian Detection
This paper has been accepted by IEEE T-NNLS journal. Please jump to
External DOI to view the offic...
Pedestrian detection plays a critical role in computer vision as it
contributes to ensuring traffic safety. Existing methods that rely solely on
RGB images suffer from performance degradation under low-light conditions due
to the lack of useful information. To address this issue, recent multispectral
detection approaches have combined thermal images to provide complementary
information and have obtained enhanced performances. Nevertheless, few
approaches focus on the negative effects of false positives caused by noisy
fused feature maps. Different from them, we comprehensively analyze the impacts
of false positives on the detection performance and find that enhancing feature
contrast can significantly reduce these false positives. In this paper, we
propose a novel target-aware fusion strategy for multispectral pedestrian
detection, named TFDet. TFDet achieves state-of-the-art performance on two
multispectral pedestrian benchmarks, KAIST and LLVIP. TFDet can easily extend
to multi-class object detection scenarios. It outperforms the previous best
approaches on two multispectral object detection benchmarks, FLIR and M3FD.
Importantly, TFDet has comparable inference efficiency to the previous
approaches, and has remarkably good detection performance even under low-light
conditions, which is a significant advancement for ensuring road safety.Abstract
arXiv:2407.21783v2 »Full PDF »Modern artificial intelligence (AI) systems are powered by foundation models.
This paper presents a new set of foundation models, called Llama 3. It is a
herd of language models that natively support multilinguality, coding,
reasoning, and tool usage. Our largest model is a dense Transformer with 405B
parameters and a context window of up to 128K tokens. This paper presents an
extensive empirical evaluation of Llama 3. We find that Llama 3 delivers
comparable quality to leading language models such as GPT-4 on a plethora of
tasks. We publicly release Llama 3, including pre-trained and post-trained
versions of the 405B parameter language model and our Llama Guard 3 model for
input and output safety. The paper also presents the results of experiments in
which we integrate image, video, and speech capabilities into Llama 3 via a
compositional approach. We observe this approach performs competitively with
the state-of-the-art on image, video, and speech recognition tasks. The
resulting models are not yet being broadly released as they are still under
development.Abstract
On ADMM in Heterogeneous Federated Learning: Personalization,
Robustness, and Fairness
arXiv admin note: text overlap with arXiv:2311.06756
Statistical heterogeneity is a root cause of tension among accuracy,
fairness, and robustness of federated learning (FL), and is key in paving a
path forward. Personalized FL (PFL) is an approach that aims to reduce the
impact of statistical heterogeneity by developing personalized models for
individual users, while also inherently providing benefits in terms of fairness
and robustness. However, existing PFL frameworks focus on improving the
performance of personalized models while neglecting the global model. Moreover,
these frameworks achieve sublinear convergence rates and rely on strong
assumptions. In this paper, we propose FLAME, an optimization framework by
utilizing the alternating direction method of multipliers (ADMM) to train
personalized and global models. We propose a model selection strategy to
improve performance in situations where clients have different types of
heterogeneous data. Our theoretical analysis establishes the global convergence
and two kinds of convergence rates for FLAME under mild assumptions. We
theoretically demonstrate that FLAME is more robust and fair than the
state-of-the-art methods on a class of linear problems. Our experimental
findings show that FLAME outperforms state-of-the-art methods in convergence
and accuracy, and it achieves higher test accuracy under various attacks and
performs more uniformly across clients.Abstract
Efficient k-means with Individual Fairness via Exponential Tilting
arXiv:2406.16557v1 »Full PDF »In location-based resource allocation scenarios, the distances between each
individual and the facility are desired to be approximately equal, thereby
ensuring fairness. Individually fair clustering is often employed to achieve
the principle of treating all points equally, which can be applied in these
scenarios. This paper proposes a novel algorithm, tilted k-means (TKM), aiming
to achieve individual fairness in clustering. We integrate the exponential
tilting into the sum of squared errors (SSE) to formulate a novel objective
function called tilted SSE. We demonstrate that the tilted SSE can generalize
to SSE and employ the coordinate descent and first-order gradient method for
optimization. We propose a novel fairness metric, the variance of the distances
within each cluster, which can alleviate the Matthew Effect typically caused by
existing fairness metrics. Our theoretical analysis demonstrates that the
well-known k-means++ incurs a multiplicative error of O(k log k), and we
establish the convergence of TKM under mild conditions. In terms of fairness,
we prove that the variance generated by TKM decreases with a scaled
hyperparameter. In terms of efficiency, we demonstrate the time complexity is
linear with the dataset size. Our experiments demonstrate that TKM outperforms
state-of-the-art methods in effectiveness, fairness, and efficiency.Abstract
NoisyGL: A Comprehensive Benchmark for Graph Neural Networks under Label
Noise
Graph Neural Networks (GNNs) exhibit strong potential in node classification
task through a message-passing mechanism. However, their performance often
hinges on high-quality node labels, which are challenging to obtain in
real-world scenarios due to unreliable sources or adversarial attacks.
Consequently, label noise is common in real-world graph data, negatively
impacting GNNs by propagating incorrect information during training. To address
this issue, the study of Graph Neural Networks under Label Noise (GLN) has
recently gained traction. However, due to variations in dataset selection, data
splitting, and preprocessing techniques, the community currently lacks a
comprehensive benchmark, which impedes deeper understanding and further
development of GLN. To fill this gap, we introduce NoisyGL in this paper, the
first comprehensive benchmark for graph neural networks under label noise.
NoisyGL enables fair comparisons and detailed analyses of GLN methods on noisy
labeled graph data across various datasets, with unified experimental settings
and interface. Our benchmark has uncovered several important insights that were
missed in previous research, and we believe these findings will be highly
beneficial for future studies. We hope our open-source benchmark library will
foster further advancements in this field. The code of the benchmark can be
found in https://github.com/eaglelab-zju/NoisyGL.Abstract
What Can Natural Language Processing Do for Peer Review?
arXiv:2405.06563v1 »Full PDF »The number of scientific articles produced every year is growing rapidly.
Providing quality control over them is crucial for scientists and, ultimately,
for the public good. In modern science, this process is largely delegated to
peer review -- a distributed procedure in which each submission is evaluated by
several independent experts in the field. Peer review is widely used, yet it is
hard, time-consuming, and prone to error. Since the artifacts involved in peer
review -- manuscripts, reviews, discussions -- are largely text-based, Natural
Language Processing has great potential to improve reviewing. As the emergence
of large language models (LLMs) has enabled NLP assistance for many new tasks,
the discussion on machine-assisted peer review is picking up the pace. Yet,
where exactly is help needed, where can NLP help, and where should it stand
aside? The goal of our paper is to provide a foundation for the future efforts
in NLP for peer-reviewing assistance. We discuss peer review as a general
process, exemplified by reviewing at AI conferences. We detail each step of the
process from manuscript submission to camera-ready revision, and discuss the
associated challenges and opportunities for NLP assistance, illustrated by
existing work. We then turn to the big challenges in NLP for peer review as a
whole, including data acquisition and licensing, operationalization and
experimentation, and ethical issues. To help consolidate community efforts, we
create a companion repository that aggregates key datasets pertaining to peer
review. Finally, we issue a detailed call for action for the scientific
community, NLP and AI researchers, policymakers, and funding bodies to help
bring the research in NLP for peer review forward. We hope that our work will
help set the agenda for research in machine-assisted scientific quality control
in the age of AI, within the NLP community and beyond.Abstract
ChatGPT has changed the AI community and an active research line is the
performance evaluation of ChatGPT. A key challenge for the evaluation is that
ChatGPT is still closed-source and traditional benchmark datasets may have been
used by ChatGPT as the training data. In this paper, (i) we survey recent
studies which uncover the real performance levels of ChatGPT in seven
categories of NLP tasks, (ii) review the social implications and safety issues
of ChatGPT, and (iii) emphasize key challenges and opportunities for its
evaluation. We hope our survey can shed some light on its blackbox manner, so
that researchers are not misleaded by its surface generation.Abstract
Ethical-Lens: Curbing Malicious Usages of Open-Source Text-to-Image
Models
The burgeoning landscape of text-to-image models, exemplified by innovations
such as Midjourney and DALLE 3, has revolutionized content creation across
diverse sectors. However, these advancements bring forth critical ethical
concerns, particularly with the misuse of open-source models to generate
content that violates societal norms. Addressing this, we introduce
Ethical-Lens, a framework designed to facilitate the value-aligned usage of
text-to-image tools without necessitating internal model revision. Ethical-Lens
ensures value alignment in text-to-image models across toxicity and bias
dimensions by refining user commands and rectifying model outputs. Systematic
evaluation metrics, combining GPT4-V, HEIM, and FairFace scores, assess
alignment capability. Our experiments reveal that Ethical-Lens enhances
alignment capabilities to levels comparable with or superior to commercial
models like DALLE 3, ensuring user-generated content adheres to ethical
standards while maintaining image quality. This study indicates the potential
of Ethical-Lens to ensure the sustainable development of open-source
text-to-image tools and their beneficial integration into society. Our code is
available at https://github.com/yuzhu-cai/Ethical-Lens.Abstract
16737 latest Fairness/Ethics + ML/AI papers
