Benchmark
non-incremental/QF_BV/20230221-oisc-gurtner/SRAI-SAFE-20-1024.smt2
Publications: "A Formally Verified Reduction of the RV32I ISA", Sonja Gurtner
(https://epub.jku.at/obvulihs/content/titleinfo/8237703)
The benchmarks come from the master thesis from Sonja Gurtner "A Formally Verified Reduction of the
RV32I ISA" about translating RISC-V instructions using only memory operations, jump, and sub. The
translation was done by Rosette, a tool to translate constraints from a Racket-like language to the
SMT-LIB.
There are two parameters in the translation:
- the bitwidth we are considering
- the upper bound on the number of loop iterations (for
correctness, it must be larger than the bitwidth)
One limitation of the work is that the translation was verified one replacement at a time. We verify
that addition (RISC-V instruction add) can be replaced by two subtraction (RISC-V instruction sub),
but subsequently use the usual addition. Thanks to this, verification scales up to large
bitwidth. Otherwise, the verification of such nested instructions is very challenging.
Naming convention: <instruction_to_replace>[-nested]-<bitwidth>-<upper_bound>[-<counterexample-cause>.smt2
| Benchmark |
|---|
| Size | 2781133 |
| Compressed Size | 430397 |
| License |
Creative Commons Attribution 4.0 International
(CC-BY-4.0)
|
| Category | industrial |
| First Occurrence | 2023-07-06 |
| Generated By | Sonja Gurtner and Mathias Fleury |
| Generated On | 2023-02-19 00:00:00 |
| Generator | Rosette |
| Dolmen OK | 1 |
| strict Dolmen OK | 1 |
| check-sat calls | 1 |
| Status | unsat |
| Inferred Status | unsat |
| Size | 2781124 |
| Compressed Size | 430402 |
| Max. Term Depth | 3 |
| Asserts | 4 |
| Declared Functions | 0 |
| Declared Constants | 140 |
| Declared Sorts | 0 |
| Defined Functions | 0 |
| Defined Recursive Functions | 0 |
| Defined Sorts | 0 |
| Constants | 55515 |
| Declared Datatypes | 0 |
Symbols
Bool | 34673 |
ite | 7795 |
not | 8206 |
or | 10260 |
and | 10429 |
= | 5751 |
BitVec | 20842 |
bvand | 1 |
bvor | 9 |
bvneg | 1039 |
bvadd | 5132 |
bvsmod | 2 |
bvult | 10 |
bvule | 4 |
bvslt | 1 |
bvsle | 12 |
bvshl | 5116 |
bvlshr | 1035 |
bvashr | 1 |
| |
Evaluations
| Evaluation |
Rating |
Solver |
Variant |
Result |
Wallclock |
CPU Time |
|
SMT-COMP 2023
|
0.33 (4/6) |
Bitwuzla |
Bitwuzla-fixed_default |
unsat ✅
|
60.22790
|
60.23220
|
| |
cvc5 |
cvc5-default-2023-05-16-ea045f305_sq |
unsat ✅
|
385.22500
|
385.10200
|
| |
STP |
STP 2022.4_default |
unsat ✅
|
78.12080
|
78.07520
|
| |
|
STP 2022.4_default |
unsat ✅
|
81.00020
|
80.99400
|
| |
UltimateEliminator |
UltimateIntBlastingWrapper+SMTInterpol_default |
unknown ❌
|
4.70691
|
12.09480
|
| |
Yices2 |
Yices 2 for SMTCOMP 2023_default |
unknown ❌
|
1200.12000
|
1200.01000
|
| |
Z3-Owl |
z3-Owl-Final_default |
unsat ✅
|
144.37700
|
144.35600
|
| |
|
z3-Owl-Final_default |
unsat ✅
|
160.62700
|
160.58900
|
|
SMT-COMP 2024
|
0.33 (4/6) |
Bitwuzla |
Bitwuzla |
unsat ✅
|
37.13871
|
37.03045
|
| |
cvc5 |
cvc5 |
unsat ✅
|
209.15398
|
209.04704
|
| |
SMTInterpol |
SMTInterpol |
unknown ❌
|
1202.21750
|
1265.76933
|
| |
STP |
STP |
unsat ✅
|
36.20807
|
36.09541
|
| |
Yices2 |
Yices2 |
unsat ✅
|
47.43640
|
47.33412
|
| |
Z3alpha |
Z3-alpha |
unknown ❌
|
1201.71383
|
1201.08795
|
|
SMT-COMP 2025
|
0.22 (7/9) |
Bitwuzla |
Bitwuzla |
unsat ✅
|
57.26647
|
57.13234
|
| |
|
Bitwuzla-MachBV-base |
unsat ✅
|
38.86613
|
38.73598
|
| |
Bitwuzla-MachBV |
Bitwuzla-MachBV |
unsat ✅
|
20.87172
|
20.74737
|
| |
BVDecide |
bv_decide |
unknown ❌
|
1201.40273
|
1200.92884
|
| |
|
bv_decide-nokernel |
unknown ❌
|
1201.38750
|
1201.04150
|
| |
cvc5 |
cvc5 |
unsat ✅
|
138.91541
|
138.77562
|
| |
SMTInterpol |
SMTInterpol |
unknown ❌
|
1201.66348
|
1253.65918
|
| |
Yices2 |
Yices2 |
unsat ✅
|
53.28465
|
53.15605
|
| |
Z3alpha |
Z3-alpha |
unsat ✅
|
169.45596
|
470.75648
|
| |
Z3 |
Z3-alpha-base |
unsat ✅
|
23.87815
|
23.75223
|
| |
|
Z3-Owl-base |
unsat ✅
|
106.27313
|
106.10375
|
| |
|
z3siri-base |
unsat ✅
|
24.16764
|
24.04136
|
| |
Z3-Owl |
Z3-Owl |
unsat ✅
|
83.03348
|
82.90335
|