arXiv:2406.00515v2 »Full PDF »Large Language Models (LLMs) have garnered remarkable advancements across
diverse code-related tasks, known as Code LLMs, particularly in code generation
that generates source code with LLM from natural language descriptions. This
burgeoning field has captured significant interest from both academic
researchers and industry professionals due to its practical significance in
software development, e.g., GitHub Copilot. Despite the active exploration of
LLMs for a variety of code tasks, either from the perspective of natural
language processing (NLP) or software engineering (SE) or both, there is a
noticeable absence of a comprehensive and up-to-date literature review
dedicated to LLM for code generation. In this survey, we aim to bridge this gap
by providing a systematic literature review that serves as a valuable reference
for researchers investigating the cutting-edge progress in LLMs for code
generation. We introduce a taxonomy to categorize and discuss the recent
developments in LLMs for code generation, covering aspects such as data
curation, latest advances, performance evaluation, ethical implications,
environmental impact, and real-world applications. In addition, we present a
historical overview of the evolution of LLMs for code generation and offer an
empirical comparison using the HumanEval, MBPP, and BigCodeBench benchmarks
across various levels of difficulty and types of programming tasks to highlight
the progressive enhancements in LLM capabilities for code generation. We
identify critical challenges and promising opportunities regarding the gap
between academia and practical development. Furthermore, we have established a
dedicated resource GitHub page (https://github.com/juyongjiang/CodeLLMSurvey)
to continuously document and disseminate the most recent advances in the field.Abstract
Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voters
How could LLMs influence our democracy? We investigate LLMs' political
leanings and the potential influence of LLMs on voters by conducting multiple
experiments in a U.S. presidential election context. Through a voting
simulation, we first demonstrate 18 open- and closed-weight LLMs' political
preference for a Democratic nominee over a Republican nominee. We show how this
leaning towards the Democratic nominee becomes more pronounced in
instruction-tuned models compared to their base versions by analyzing their
responses to candidate-policy related questions. We further explore the
potential impact of LLMs on voter choice by conducting an experiment with 935
U.S. registered voters. During the experiments, participants interacted with
LLMs (Claude-3, Llama-3, and GPT-4) over five exchanges. The experiment results
show a shift in voter choices towards the Democratic nominee following LLM
interaction, widening the voting margin from 0.7% to 4.6%, even though LLMs
were not asked to persuade users to support the Democratic nominee during the
discourse. This effect is larger than many previous studies on the
persuasiveness of political campaigns, which have shown minimal effects in
presidential elections. Many users also expressed a desire for further
political interaction with LLMs. Which aspects of LLM interactions drove these
shifts in voter choice requires further study. Lastly, we explore how a safety
method can make LLMs more politically neutral, while raising the question of
whether such neutrality is truly the path forward.Abstract
CIPHER: Cybersecurity Intelligent Penetration-testing Helper for Ethical
Researcher
Penetration testing, a critical component of cybersecurity, typically
requires extensive time and effort to find vulnerabilities. Beginners in this
field often benefit from collaborative approaches with the community or
experts. To address this, we develop CIPHER (Cybersecurity Intelligent
Penetration-testing Helper for Ethical Researchers), a large language model
specifically trained to assist in penetration testing tasks. We trained CIPHER
using over 300 high-quality write-ups of vulnerable machines, hacking
techniques, and documentation of open-source penetration testing tools.
Additionally, we introduced the Findings, Action, Reasoning, and Results (FARR)
Flow augmentation, a novel method to augment penetration testing write-ups to
establish a fully automated pentesting simulation benchmark tailored for large
language models. This approach fills a significant gap in traditional
cybersecurity Q\&A benchmarks and provides a realistic and rigorous standard
for evaluating AI's technical knowledge, reasoning capabilities, and practical
utility in dynamic penetration testing scenarios. In our assessments, CIPHER
achieved the best overall performance in providing accurate suggestion
responses compared to other open-source penetration testing models of similar
size and even larger state-of-the-art models like Llama 3 70B and Qwen1.5 72B
Chat, particularly on insane difficulty machine setups. This demonstrates that
the current capabilities of general LLMs are insufficient for effectively
guiding users through the penetration testing process. We also discuss the
potential for improvement through scaling and the development of better
benchmarks using FARR Flow augmentation results. Our benchmark will be released
publicly at https://github.com/ibndias/CIPHER.Abstract
UniGuard: Towards Universal Safety Guardrails for Jailbreak Attacks on
Multimodal Large Language Models
Multimodal large language models (MLLMs) have revolutionized vision-language
understanding but are vulnerable to multimodal jailbreak attacks, where
adversaries meticulously craft inputs to elicit harmful or inappropriate
responses. We propose UniGuard, a novel multimodal safety guardrail that
jointly considers the unimodal and cross-modal harmful signals. UniGuard is
trained such that the likelihood of generating harmful responses in a toxic
corpus is minimized, and can be seamlessly applied to any input prompt during
inference with minimal computational costs. Extensive experiments demonstrate
the generalizability of UniGuard across multiple modalities and attack
strategies. It demonstrates impressive generalizability across multiple
state-of-the-art MLLMs, including LLaVA, Gemini Pro, GPT-4, MiniGPT-4, and
InstructBLIP, thereby broadening the scope of our solution.Abstract
A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample
Perspective
arXiv:2411.00360v1 »Full PDF »Learning generalized models from biased data is an important undertaking
toward fairness in deep learning. To address this issue, recent studies attempt
to identify and leverage bias-conflicting samples free from spurious
correlations without prior knowledge of bias or an unbiased set. However,
spurious correlation remains an ongoing challenge, primarily due to the
difficulty in precisely detecting these samples. In this paper, inspired by the
similarities between mislabeled samples and bias-conflicting samples, we
approach this challenge from a novel perspective of mislabeled sample
detection. Specifically, we delve into Influence Function, one of the standard
methods for mislabeled sample detection, for identifying bias-conflicting
samples and propose a simple yet effective remedy for biased models by
leveraging them. Through comprehensive analysis and experiments on diverse
datasets, we demonstrate that our new perspective can boost the precision of
detection and rectify biased models effectively. Furthermore, our approach is
complementary to existing methods, showing performance improvement even when
applied to models that have already undergone recent debiasing techniques.Abstract
arXiv:2410.21276v1 »Full PDF »GPT-4o is an autoregressive omni model that accepts as input any combination
of text, audio, image, and video, and generates any combination of text, audio,
and image outputs. It's trained end-to-end across text, vision, and audio,
meaning all inputs and outputs are processed by the same neural network. GPT-4o
can respond to audio inputs in as little as 232 milliseconds, with an average
of 320 milliseconds, which is similar to human response time in conversation.
It matches GPT-4 Turbo performance on text in English and code, with
significant improvement on text in non-English languages, while also being much
faster and 50\% cheaper in the API. GPT-4o is especially better at vision and
audio understanding compared to existing models. In line with our commitment to
building AI safely and consistent with our voluntary commitments to the White
House, we are sharing the GPT-4o System Card, which includes our Preparedness
Framework evaluations. In this System Card, we provide a detailed look at
GPT-4o's capabilities, limitations, and safety evaluations across multiple
categories, focusing on speech-to-speech while also evaluating text and image
capabilities, and measures we've implemented to ensure the model is safe and
aligned. We also include third-party assessments on dangerous capabilities, as
well as discussion of potential societal impacts of GPT-4o's text and vision
capabilities.Abstract
Towards Automated Penetration Testing: Introducing LLM Benchmark,
Analysis, and Improvements
Main Paper 1-9 pages, Supplementary Materials: 10-17, 13 figures
Hacking poses a significant threat to cybersecurity, inflicting billions of
dollars in damages annually. To mitigate these risks, ethical hacking, or
penetration testing, is employed to identify vulnerabilities in systems and
networks. Recent advancements in large language models (LLMs) have shown
potential across various domains, including cybersecurity. However, there is
currently no comprehensive, open, end-to-end automated penetration testing
benchmark to drive progress and evaluate the capabilities of these models in
security contexts. This paper introduces a novel open benchmark for LLM-based
automated penetration testing, addressing this critical gap. We first evaluate
the performance of LLMs, including GPT-4o and Llama 3.1-405B, using the
state-of-the-art PentestGPT tool. Our findings reveal that while Llama 3.1
demonstrates an edge over GPT-4o, both models currently fall short of
performing fully automated, end-to-end penetration testing. Next, we advance
the state-of-the-art and present ablation studies that provide insights into
improving the PentestGPT tool. Our research illuminates the challenges LLMs
face in each aspect of Pentesting, e.g. enumeration, exploitation, and
privilege escalation. This work contributes to the growing body of knowledge on
AI-assisted cybersecurity and lays the foundation for future research in
automated penetration testing using large language models.Abstract
ETHIC: Evaluating Large Language Models on Long-Context Tasks with High
Information Coverage
Recent advancements in large language models (LLM) capable of processing
extremely long texts highlight the need for a dedicated evaluation benchmark to
assess their long-context capabilities. However, existing methods, like the
needle-in-a-haystack test, do not effectively assess whether these models fully
utilize contextual information, raising concerns about the reliability of
current evaluation techniques. To thoroughly examine the effectiveness of
existing benchmarks, we introduce a new metric called information coverage
(IC), which quantifies the proportion of the input context necessary for
answering queries. Our findings indicate that current benchmarks exhibit low
IC; although the input context may be extensive, the actual usable context is
often limited. To address this, we present ETHIC, a novel benchmark designed to
assess LLMs' ability to leverage the entire context. Our benchmark comprises
2,648 test instances spanning four long-context tasks with high IC scores in
the domains of books, debates, medicine, and law. Our evaluations reveal
significant performance drops in contemporary LLMs, highlighting a critical
challenge in managing long contexts. Our benchmark is available at
https://github.com/dmis-lab/ETHIC.Abstract
General Frameworks for Conditional Two-Sample Testing
We study the problem of conditional two-sample testing, which aims to
determine whether two populations have the same distribution after accounting
for confounding factors. This problem commonly arises in various applications,
such as domain adaptation and algorithmic fairness, where comparing two groups
is essential while controlling for confounding variables. We begin by
establishing a hardness result for conditional two-sample testing,
demonstrating that no valid test can have significant power against any single
alternative without proper assumptions. We then introduce two general
frameworks that implicitly or explicitly target specific classes of
distributions for their validity and power. Our first framework allows us to
convert any conditional independence test into a conditional two-sample test in
a black-box manner, while preserving the asymptotic properties of the original
conditional independence test. The second framework transforms the problem into
comparing marginal distributions with estimated density ratios, which allows us
to leverage existing methods for marginal two-sample testing. We demonstrate
this idea in a concrete manner with classification and kernel-based methods.
Finally, simulation studies are conducted to illustrate the proposed frameworks
in finite-sample scenarios.Abstract
HAICOSYSTEM: An Ecosystem for Sandboxing Safety Risks in Human-AI
Interactions
Both the second and third authors contributed equally
AI agents are increasingly autonomous in their interactions with human users
and tools, leading to increased interactional safety risks. We present
HAICOSYSTEM, a framework examining AI agent safety within diverse and complex
social interactions. HAICOSYSTEM features a modular sandbox environment that
simulates multi-turn interactions between human users and AI agents, where the
AI agents are equipped with a variety of tools (e.g., patient management
platforms) to navigate diverse scenarios (e.g., a user attempting to access
other patients' profiles). To examine the safety of AI agents in these
interactions, we develop a comprehensive multi-dimensional evaluation framework
that uses metrics covering operational, content-related, societal, and legal
risks. Through running 1840 simulations based on 92 scenarios across seven
domains (e.g., healthcare, finance, education), we demonstrate that HAICOSYSTEM
can emulate realistic user-AI interactions and complex tool use by AI agents.
Our experiments show that state-of-the-art LLMs, both proprietary and
open-sourced, exhibit safety risks in over 50\% cases, with models generally
showing higher risks when interacting with simulated malicious users. Our
findings highlight the ongoing challenge of building agents that can safely
navigate complex interactions, particularly when faced with malicious users. To
foster the AI agent safety ecosystem, we release a code platform that allows
practitioners to create custom scenarios, simulate interactions, and evaluate
the safety and performance of their agents.Abstract
16737 latest Fairness/Ethics + ML/AI papers
