Benchmark

non-incremental/QF_BV/tacas07/rfunit_flat-64.smt2 

This directory contains benchmarks used in the experimental results
reported in the following paper:

Title: Deciding Bit-Vector Arithmetic with Abstraction
Authors: R. E. Bryant, D. Kroening, J. Ouaknine, S. A. Seshia, O. Strichman, B. Brady

If you use these benchmarks, we will appreciate hearing from you by 
e-mail to sseshia@eecs.berkeley.edu. Additionally, the following sources 
must be acknowledged:

1) Y86_std.smt, Y86_btnft.smt: 
   Generated from a UCLID model created by R. E. Bryant, as described
   in the following paper:
    http://www.cs.cmu.edu/~bryant/pubdir/CMU-CS-05-195.pdf

2) s-40-50-bv.smt: 
   Generated by V. Ganapathy for finding format-string exploits in
   the WiSA project (http://www.cs.wisc.edu/wisa)
 
3) BBB-32.smt, rfunit_flat-64.smt:
   Generated from benchmarks generated by Z. Hanna of Intel.

All benchmarks are in the SMT-LIB format, as described at:
http://combination.cs.uiowa.edu/smtlib/
Benchmark
Size397821
Compressed Size46200
License Creative Commons Attribution 4.0 International (CC-BY-4.0)
Categoryindustrial
First Occurrence2007-07-03
Generated By—
Generated On—
Generator—
Dolmen OK1
strict Dolmen OK1
check-sat calls1
Query 1
Status sat
Inferred Status sat
Size 397813
Compressed Size46202
Max. Term Depth402
Asserts 1
Declared Functions0
Declared Constants2594
Declared Sorts 0
Defined Functions0
Defined Recursive Functions 0
Defined Sorts0
Constants0
Declared Datatypes0

Symbols

not1305 or3276 and3445 =512
let386 concat132 extract133 bvmul174
bvsub430 bvult258 bvule622 sign_extend132

Evaluations

Evaluation Rating Solver Variant Result Wallclock CPU Time
SMT-COMP 2008 0.14 (6/7) Beaver Beaver-1.0 sat ✅ 15.10000 —
Boolector Boolector sat ✅ 2.90000 —
CVC3 CVC3-1.5 sat ✅ 3.20000 —
MathSAT MathSAT-4.2 sat ✅ 0.90000 —
Spear Spear sat ✅ 131.50000 —
Spear v1.9 (fh-1-2) sat ✅ 47.50000 —
SWORD SWORD v0.2 unknown ❌ 1801.00000 —
Z3 Z3.2 sat ✅ 164.10000 —
SMT-COMP 2010 CVC3 CVC3 2.3 sat ✅ 1.40000 —
SONOLAR SONOLAR r252 sat ✅ 5.40000 —
STP simplifyingSTP sat ✅ 5.70000 —
SMT Evaluation 2013 4Simp 4Simp-SMT-COMP-2012 default sat ✅ 3.13969 —
Abziz abziz_portfolio_2011_minfeatures_2 default sat ✅ 0.09380 —
abziz_portfolio_2011_solvers_2 default sat ✅ 0.08896 —
abziz_portfolio_2012_minfeatures default sat ✅ 0.09497 —
abziz_portfolio_2012_solvers default sat ✅ 0.09264 —
AbzizPortfolio-SMT-COMP-2012 default sat ✅ 0.08946 —
Boolector Boolector-1.5.118-SMT-EVAL-2013 default sat ✅ 0.75513 —
Boolector-SMT-COMP-2011 default sat ✅ 2.87568 —
Boolector-SMT-COMP-2012 default sat ✅ 0.52177 —
CVC3 CVC3-SMT-COMP-2010 default sat ✅ 1.05514 —
CVC3-SMT-COMP-2011 default sat ✅ 0.89896 —
CVC3-SMT-COMP-2012 default sat ✅ 0.91395 —
CVC4 CVC4-SMT-COMP-2012-Resubmission default sat ✅ 19.73890 —
CVC4-SMT-EVAL-2013 default sat ✅ 0.41524 —
MathSAT MathSAT5-5.2.6-SMT-EVAL-2013 default sat ✅ 0.76017 —
MathSAT5-HeavyBV-SMT-COMP-2012 default sat ✅ 12.35370 —
MathSAT5-SMT-COMP-2011 default sat ✅ 0.66964 —
MathSAT5-SMT-COMP-2012 default sat ✅ 0.69528 —
SONOLAR SONOLAR-2013-05-15-SMT-EVAL-2013 default sat ✅ 1.46443 —
SONOLAR-SMT-COMP-2010 default sat ✅ 1.94410 —
SONOLAR-SMT-COMP-2011 default sat ✅ 1.55916 —
SONOLAR-SMT-COMP-2012 default sat ✅ 1.52785 —
STP STP2-SMT-COMP-2011 default sat ✅ 0.09785 —
STP2-SMT-COMP-2012 default sat ✅ 0.10434 —
simplifyingSTP-SMT-COMP-2010 default sat ✅ 2.70373 —
Z3 Z3-4.3.2.a054b099c1d6-x64-debian-6.0.6-SMT-EVAL-2013 default sat ✅ 0.07611 —
Z3-SMT-COMP-2011 default sat ✅ 0.08440 —
SMT-COMP 2014 4Simp 4Simp - 2014 default sat ✅ 2.00402 1.99869
Abziz abziz_portfolio_all_features default sat ✅ 0.16588 0.16097
abziz_portfolio_min_features default sat ✅ 0.17522 0.16197
Boolector Boolector boolector sat ✅ 6.53393 6.52801
CVC4 CVC4 f7118b2 default sat ✅ 0.37208 0.59591
MathSAT MathSAT-5.2.12-Main default sat ✅ 0.78875 0.77988
SONOLAR sonolar_smtcomp-2014 default sat ✅ 2.14020 2.13767
STP stp-cryptominisat4 default sat ✅ 0.10611 0.09699
Yices2 Yices-2.2.1-smtcomp2014 default sat ✅ 0.18548 0.17697
Z3 Z3-4.3.2.a054b099c1d6-x64-debian-6.0.6-SMT-COMP-2014 default sat ✅ 0.07794 0.06999
SMT-COMP 2015 0.14 (6/7) Boolector Boolector SMT15 QF_BV final boolector_qf_bv sat ✅ 6.24087 6.24305
CVC4 CVC4-master-2015-06-15-9b32405-main default sat ✅ 0.37766 0.61791
CVC4-experimental-2015-06-15-ff5745a-main default sat ✅ 0.38219 0.62790
MathSAT MathSat 5.3.6 main smtcomp2015_main sat ✅ 2.70378 2.70359
SMT-RAT SMT-RAT-final default unknown ❌ 287.01700 287.01000
STP stp-cryptominisat4 default sat ✅ 0.09828 0.09698
stp-cmsat4-v15 default sat ✅ 0.09721 0.09599
stp-cmsat4-mt-v15 default sat ✅ 0.09663 0.09598
stp-minisat-v15 default sat ✅ 0.09615 0.09498
Yices2 Yices default sat ✅ 0.17568 0.17497
Z3 z3 4.4.0 default sat ✅ 0.08715 0.08699
SMT-COMP 2016 ABC ABC_default default_abc sat ✅ 2.73669 2.73791
ABC_glucose glucose sat ✅ 19.31620 19.32650
Boolector Boolector boolector sat ✅ 13.47860 13.48650
Boolector preprop boolector sat ✅ 0.11307 0.11308
CVC4 CVC4-master-2016-05-27-cfef263-main default sat ✅ 7.62163 15.13270
MapleSTP MapleSTP default sat ✅ 0.08841 0.08843
MapleSTP-mt default sat ✅ 0.08841 0.08841
MathSAT mathsat-5.3.11-linux-x86_64-Main default sat ✅ 2.28898 2.29183
MinkeyRink Minkeyrink 2016 default sat ✅ 0.12554 0.12559
Q3B Q3B default sat ✅ 0.49133 1.20333
STP stp-cms-exp-2016 default sat ✅ 0.09255 0.09254
stp-cms-mt-2016 default sat ✅ 0.09140 0.09161
stp-cms-st-2016 default sat ✅ 0.09307 0.09309
stp-minisat-st-2016 default sat ✅ 0.09239 0.09240
Yices2 Yices-2.4.2 default sat ✅ 0.20242 0.20249
Z3 z3-4.4.1 default sat ✅ 0.08392 0.08520
SMT-COMP 2017 Boolector Boolector+CaDiCaL SMT17 final boolector sat ✅ 0.14281 0.14113
Boolector SMT17 final boolector sat ✅ 0.14036 0.13954
CVC4 CVC4-smtcomp2017-main default sat ✅ 0.41334 0.67712
MathSAT mathsat-5.4.1-linux-x86_64-Main default sat ✅ 2.42360 2.42033
MinkeyRink MinkeyRink 2017.3a default sat ✅ 0.15071 0.14896
Q3B Q3B default sat ✅ 0.34349 0.75972
STP stp_st default sat ✅ 0.09896 0.09818
stp_mt default sat ✅ 0.09808 0.09801
Yices2 Yices2-Main default sat ✅ 0.22714 0.21935
Z3 z3-4.5.0 default sat ✅ 0.09239 0.09134
SMT-COMP 2018 Boolector Boolector_default sat ✅ 0.12468 0.12460
CVC4 master-2018-06-10-b19c840-competition-default_default sat ✅ 5.54218 5.54179
MathSAT mathsat-5.5.2-linux-x86_64-Main_default sat ✅ 2.29910 2.29989
MinkeyRink Minkeyrink MT_mt sat ✅ 0.16375 0.16379
Minkeyrink ST_st sat ✅ 0.16430 0.16430
STP STP-CMS-st-2018_default-no-stderr sat ✅ 0.09151 0.09140
STP-CMS-mt-2018_multicore-no-stderr sat ✅ 0.09138 0.09143
STP-Riss-st-2018_riss-no-stderr sat ✅ 0.09151 0.09158
Yices2 Yices 2.6.0_default sat ✅ 0.17542 0.17533
Z3 z3-4.7.1_default sat ✅ 0.09186 0.09180
SMT-COMP 2020 Bitwuzla Bitwuzla-fixed_default sat ✅ 0.77026 0.77021
Boolector Boolector-wrapped-sq_default sat ✅ 0.14183 0.14184
Poolector-wrapped-sq_default sat ✅ 0.27910 0.40836
CVC4 CVC4-sq-final_default sat ✅ 3.73477 3.73499
LazyBV2Int LazyBV2Int20200523_default.sh sat ✅ 8.89120 8.89159
MathSAT MathSAT5_default.sh sat ✅ 0.69827 0.69818
MinkeyRink MinkeyRink Solver 2020.3.1_default sat ✅ 0.14217 0.14227
MinkeyRink Solver 2020.3_default sat ✅ 0.14439 0.14442
STP STP_default sat ✅ 0.09135 0.09146
STP ++ Mergsat v1_default sat ✅ 0.08669 0.08832
Yices2 Yices 2.6.2 bug fix_default sat ✅ 0.01460 0.01370
Z3 z3-4.8.8_default sat ✅ 0.22747 0.22736
SMT-COMP 2022 Bitwuzla Bitwuzla-fixed_default sat ✅ 0.75863 0.75854
Bitwuzla-wrapped_default sat ✅ 0.75944 0.75931
cvc5 cvc5-default-2022-07-02-b15e116-wrapped_sq sat ✅ 6.93216 6.93225
MathSAT MathSAT-5.6.8_default sat ✅ 0.80163 0.80163
STP STP 2022.4_default sat ✅ 0.10559 0.10562
Yices2 Yices 2.6.2 for SMTCOMP 2021_default sat ✅ 0.01395 0.01390
Z3++BV z3++bv_0702_default sat ✅ 2.04964 2.04938
Z3 z3-4.8.17_default sat ✅ 0.21925 0.22100
SMT-COMP 2024 Bitwuzla Bitwuzla sat ✅ 1.12710 1.02715
cvc5 cvc5 sat ✅ 6.67661 6.57404
SMTInterpol SMTInterpol sat ✅ 47.13847 69.68459
STP STP sat ✅ 1.08372 0.98404
Yices2 Yices2 sat ✅ 0.22039 0.12066
Z3alpha Z3-alpha sat ✅ 14.91961 14.81164
SMT-COMP 2025 Bitwuzla Bitwuzla sat ✅ 0.63678 0.51815
Bitwuzla-MachBV-base sat ✅ 0.92056 0.78992
Bitwuzla-MachBV Bitwuzla-MachBV sat ✅ 0.81988 0.70245
BVDecide bv_decide sat ✅ 11.12610 10.98891
bv_decide-nokernel sat ✅ 6.76882 6.62328
cvc5 cvc5 sat ✅ 5.38120 5.26595
SMTInterpol SMTInterpol sat ✅ 6.85685 17.11103
Yices2 Yices2 sat ✅ 0.28463 0.16072
Z3alpha Z3-alpha sat ✅ 0.40457 0.35524
Z3 Z3-alpha-base sat ✅ 0.30543 0.18765
Z3-Owl-base sat ✅ 0.39670 0.28352
z3siri-base sat ✅ 0.30384 0.18304
Z3-Owl Z3-Owl sat ✅ 0.85268 0.71904