WraAct: Convex Hull Approximation for General Activation Functions
Zhongkui Ma, Zihan Wang, Guangdong Bai
Efficiently constructing tight over-approximations for activation function hulls, with reported average 400x faster construction on benchmarked Sigmoid, Tanh, MaxPool, and related cases.
Overview
WraAct is an approach to efficiently constructing tight over-approximations for activation function hulls. OOPSLA’25 (the ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications) within SPLASH’25.
Key Idea
WraAct’s core idea is to introduce linear constraints to smooth out the fluctuations in the target function, by leveraging double-linear-piece (DLP) functions to simplify the local geometry. In this way, the problem is reduced to over-approximating DLP functions, which can be efficiently handled.
Supported Activation Functions
- ReLU — Rectified Linear Unit
- LeakyReLU — Leaky variant of the rectifier
- ELU — Exponential Linear Unit
- Sigmoid — Logistic sigmoid function
- Tanh — Hyperbolic tangent
- MaxPool — Max pooling, also available as a DLP variant
- Further activations by subclassing the
ActHullbase class
Performance Highlights
- 400x faster on average than SBLM+PDDM in the reported evaluation
- 150x tighter on average under the paper’s precision metric
- 50% fewer constraints on average
- Handles up to 8 input dimensions in under 10 seconds
- In the reported evaluation, handled ResNet-scale models with about 22k neurons in under one minute per sample
Verification Impact
On the paper’s 100-sample benchmark:
- Enhances single-neuron verification from under 10 to over 40 verified samples
- Outperforms multi-neuron verifier PRIMA with up to 20 additional verified samples
Comparison with WraLU
| Feature | WraLU (POPL’24) | WraAct (OOPSLA’25) |
|---|---|---|
| Activation | ReLU only | General (Sigmoid, Tanh, MaxPool, etc.) |
| Approach | Wrapping polytope | DLP-based smoothing |
| Multi-neuron | Yes | Yes |
| Scalability | Up to ~4D efficiently | Up to 8D in 10s |
Related Tools
- WraLU: ReLU-specific hull approximation (POPL’24)
- wraact (library): Unified Python library for hull approximation
Read the explanation
- What Makes a Hull Approximation Useful
Discussed in this article alongside WraLU.
- Activation Hulls: What Makes an Approximation Sound?
Discussed in this article alongside WraLU.
- Single-Neuron, Multivariate, and Multi-Neuron Relaxations
The chapter that places this work against the others in its family.
Software
The paper artifact above and the maintained library below are separate objects with separate versions. The article links the library; the repository is where its API is current.
Citation
@article{10.1145/3763086,
author = {Ma, Zhongkui and Wang, Zihan and Bai, Guangdong},
title = {Convex Hull Approximation for Activation Functions},
year = {2025},
issue_date = {October 2025},
publisher = {Association for Computing Machinery},
volume = {9},
number = {OOPSLA2},
url = {https://doi.org/10.1145/3763086},
doi = {10.1145/3763086},
month = oct,
articleno = {308},
numpages = {27}
}